You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Wei ZHOU <us...@gmail.com> on 2013/05/30 18:03:19 UTC
[MERGE] disk_io_throttling to MASTER
Hi,
I would like to merge disk_io_throttling branch into master.
If nobody object, I will merge into master in 48 hours.
The purpose is :
Virtual machines are running on the same storage device (local storage or
share strage). Because of the rate limitation of device (such as iops), if
one VM has large disk operation, it may affect the disk performance of
other VMs running on the same storage device.
It is neccesary to set the maximum rate and limit the disk I/O of VMs.
The feature includes:
(1) set the maximum rate of VMs (in disk_offering, and global configuration)
(2) change the maximum rate of VMs
(3) limit the disk rate (total bps and iops)
JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
FS (I will update later) :
https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
Merge check list :-
* Did you check the branch's RAT execution success?
Yes
* Are there new dependencies introduced?
No
* What automated testing (unit and integration) is included in the new
feature?
Unit tests are added.
* What testing has been done to check for potential regressions?
(1) set the bytes rate and IOPS rate on CloudStack UI.
(2) VM operations, including
deploy, stop, start, reboot, destroy, expunge. migrate, restore
(3) Volume operations, including
Attach, Detach
To review the code, you can try
git diff c30057635d04a2396f84c588127d7ebe42e503a7
f2e5591b710d04cc86815044f5823e73a4a58944
Best regards,
Wei
[1]
https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
[2] refs/heads/disk_io_throttling
[3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301
- VM Disk I/O Throttling)
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
For now, this support KVM only.
2013/5/30 Wei ZHOU <us...@gmail.com>
> Hi,
> I would like to merge disk_io_throttling branch into master.
> If nobody object, I will merge into master in 48 hours.
> The purpose is :
>
> Virtual machines are running on the same storage device (local storage or
> share strage). Because of the rate limitation of device (such as iops), if
> one VM has large disk operation, it may affect the disk performance of
> other VMs running on the same storage device.
> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>
> The feature includes:
>
> (1) set the maximum rate of VMs (in disk_offering, and global
> configuration)
> (2) change the maximum rate of VMs
> (3) limit the disk rate (total bps and iops)
> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> FS (I will update later) :
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> Merge check list :-
>
> * Did you check the branch's RAT execution success?
> Yes
>
> * Are there new dependencies introduced?
> No
>
> * What automated testing (unit and integration) is included in the new
> feature?
> Unit tests are added.
>
> * What testing has been done to check for potential regressions?
> (1) set the bytes rate and IOPS rate on CloudStack UI.
> (2) VM operations, including
> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> (3) Volume operations, including
> Attach, Detach
>
> To review the code, you can try
> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> f2e5591b710d04cc86815044f5823e73a4a58944
>
> Best regards,
> Wei
>
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> [2] refs/heads/disk_io_throttling
> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301 - VM Disk I/O Throttling)
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"My thinking is that, independent of CloudStack, an operator allocates a
chunk of a SAN to CloudStack, and exposes it through a LUN. They simply
want to turn control of that LUN over to CloudStack, but not allow
CloudStack to allocate anymore LUNs."
This is OK and is supported today in CloudStack. In fact, this is how
SolidFire's CloudStack users use CloudStack: They manually allocate a large
SAN volume (LUN) and expose it as Primary Storage. This SAN volume then
houses multiple VMs and data disks.
This is not ideal, however, because although we can guarantee QoS to the
SAN volume, that does not translate into QoS for any given VM or data disk
on the SAN volume.
On Tue, Jun 4, 2013 at 1:35 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> The weird part is that the iSCSI type is today only used (as far as I
> know) in regards to XenServer (when you have not PreSetup an SR).
>
> If you want to use your iSCSI volume from VMware, it uses the vmfs type.
>
> If you want to use your iSCSI volume from KVM, it uses the
> SharedMountPoint type.
>
> So, I suppose mine and Edison's thinking here was to make a new type of
> storage to describe this dynamic ability Edison added into the storage
> framework. Maybe it should be more specificy, though: Dynamic_iSCSI versus,
> say, Dynamic_FC.
>
>
> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> "The storage device itself shouldn't know or care that it is being used
>> for a Xen SR -- simply be able to answer questions about it is storing."
>>
>> I see...so your concern here is that the SolidFire plug-in needs to call
>> itself "Dynamic" storage so that the hypervisor logic knows to treat it as
>> such.
>>
>> I'm totally open to removing that constraint and just calling it iSCSI or
>> whatever. We would just need a way for the hypervisor attach logic to
>> detect this new requirement and perform the necessary activities.
>>
>>
>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> See my responses in-line.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>> > I'm trying to picture this:
>>> >
>>> > "Finally, while CloudStack may be able to manage a device, an operator
>>> may
>>> > chose to leave it unmanaged by CloudStack (e.g. the device is shared by
>>> > many services, and the operator has chosen to dedicate only a portion
>>> of it
>>> > to CloudStack). Does my reasoning make sense?"
>>> >
>>> > I guess I'm not sure how creating a SAN volume via the plug-in (before
>>> an
>>> > attach request to the hypervisor) would work unless the hypervisor
>>> consumes
>>> > the SAN volume in the form of, say, an SR.
>>>
>>> My thinking is that, independent of CloudStack, an operator allocates a
>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They simply
>>> want to turn control of that LUN over to CloudStack, but not allow
>>> CloudStack to allocate anymore LUNs.
>>>
>>> >
>>> > As the attach logic stands prior to my changes, we would be passing in
>>> a
>>> > SAN volume that does not have the necessary hypervisor support (like
>>> an SR)
>>> > and the logic will fail.
>>> >
>>> > Are you thinking we should maybe have the storage framework itself
>>> detect
>>> > that such a SAN volume needs support from the hypervisor side and have
>>> it
>>> > call into the agent code specifically to create the SR before the
>>> attach
>>> > logic runs in the agent code?
>>>
>>> I think the hypervisor management plugin should have a rich enough
>>> interface to storage to determine available for volume storage. For Xen,
>>> this interface would allow the interrogation of the device to determine the
>>> SR is present. The storage device itself shouldn't know or care that it
>>> is being used for a Xen SR -- simply be able to answer questions about it
>>> is storing.
>>>
>>> >
>>> >
>>> > On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com
>>> >> wrote:
>>> >
>>> >> So, the flow is as follows:
>>> >>
>>> >> * The admin registers the SolidFire driver (which is a type of
>>> so-called
>>> >> Dynamic storage). Once this is done, a new Primary Storage shows up
>>> in the
>>> >> applicable zone.
>>> >>
>>> >> * The admin creates a Disk Offering that references the storage tag
>>> of the
>>> >> newly created Primary Storage.
>>> >>
>>> >> * The end user creates a CloudStack volume. This leads to a new row
>>> in the
>>> >> cloud.volumes table.
>>> >>
>>> >> * The end user attaches the CloudStack volume to a VM (attach disk).
>>> This
>>> >> leads to the storage framework calling the plug-in to create a new
>>> volume
>>> >> on its storage system (in my case, a SAN). The plug-in also updates
>>> the
>>> >> cloud.volumes row with applicable data (like the IQN of the SAN
>>> volume).
>>> >> This plug-in code is only invoked if the CloudStack volume is in the
>>> >> 'Allocated' state. After the attach, the volume will be in the 'Ready'
>>> >> state (even after a detach disk) and the plug-in code will not be
>>> called
>>> >> again to create this SAN volume.
>>> >>
>>> >> * The hypervisor-attach logic is run and detects the CloudStack
>>> volume to
>>> >> attach needs "assistance" in the form of a hypervisor data structure
>>> (ex.
>>> >> an SR on XenServer).
>>> >>
>>> >>
>>> >> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>> >> mike.tutkowski@solidfire.com> wrote:
>>> >>
>>> >>> "To ensure that we are in sync on terminology, volume, in these
>>> >>> definitions, refers to the physical allocation on the device,
>>> correct?"
>>> >>>
>>> >>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>> >>>
>>> >>> To refer to the 'volume' the end user can make in CloudStack, I try
>>> to
>>> >>> use 'CloudStack volume'.
>>> >>>
>>> >>>
>>> >>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>> >>> mike.tutkowski@solidfire.com> wrote:
>>> >>>
>>> >>>> Hi John,
>>> >>>>
>>> >>>> What you say here may very well make sense, but I'm having a hard
>>> time
>>> >>>> envisioning it.
>>> >>>>
>>> >>>> Perhaps we should draw Edison in on this conversation as he was the
>>> >>>> initial person to suggest the approach I took.
>>> >>>>
>>> >>>> What do you think?
>>> >>>>
>>> >>>> Thanks!
>>> >>>>
>>> >>>>
>>> >>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
>>> >wrote:
>>> >>>>
>>> >>>>> Mike,
>>> >>>>>
>>> >>>>> It feels like we are combining two distinct concepts -- storage
>>> device
>>> >>>>> management and storage protocols. In both cases, we are
>>> communicating with
>>> >>>>> ISCSI, but one allows the system to create/delete volumes
>>> (Dynamic) on the
>>> >>>>> device while the other requires the volume to be volume to be
>>> managed
>>> >>>>> outside of the CloudStack context. To ensure that we are in sync
>>> on
>>> >>>>> terminology, volume, in these definitions, refers to the physical
>>> >>>>> allocation on the device, correct? Minimally, we must be able to
>>> >>>>> communicate with a storage device to move bits from one place to
>>> another,
>>> >>>>> read bits, delete bits, etc. Optionally, a storage device may be
>>> able to
>>> >>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
>>> device
>>> >>>>> onto which we store a Xen SR, and we can have a managed SolidFire
>>> iSCSI
>>> >>>>> device on which CloudStack is capable of allocating LUNs and
>>> storing
>>> >>>>> volumes. Finally, while CloudStack may be able to manage a
>>> device, an
>>> >>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
>>> device is
>>> >>>>> shared by many services, and the operator has chosen to dedicate
>>> only a
>>> >>>>> portion of it to CloudStack). Does my reasoning make sense?
>>> >>>>>
>>> >>>>> Assuming my thoughts above are reasonable, it seems appropriate to
>>> >>>>> strip the management concerns from StoragePoolType, add the notion
>>> of a
>>> >>>>> storage device with an attached driver that indicates whether or
>>> not is
>>> >>>>> managed by CloudStack, and establish a abstraction representing a
>>> physical
>>> >>>>> allocation on a device separate that is associated with a volume.
>>> With
>>> >>>>> these notions in place, hypervisor drivers can declare which
>>> protocols they
>>> >>>>> support and when they encounter a device managed by CloudStack,
>>> utilize the
>>> >>>>> management operations exposed by the driver to automate
>>> allocation. If
>>> >>>>> these thoughts/concepts make sense, then we can sit down and drill
>>> down to
>>> >>>>> a more detailed design.
>>> >>>>>
>>> >>>>> Thanks,
>>> >>>>> -John
>>> >>>>>
>>> >>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>> >>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>
>>> >>>>>> Here is the difference between the current iSCSI type and the
>>> Dynamic
>>> >>>>> type:
>>> >>>>>>
>>> >>>>>> iSCSI type: The admin has to go in and create a Primary Storage
>>> based
>>> >>>>> on
>>> >>>>>> the iSCSI type. At this point in time, the iSCSI volume must
>>> exist on
>>> >>>>> the
>>> >>>>>> storage system (it is pre-allocated). Future CloudStack volumes
>>> are
>>> >>>>> created
>>> >>>>>> as VDIs on the SR that was created behind the scenes.
>>> >>>>>>
>>> >>>>>> Dynamic type: The admin has to go in and create Primary Storage
>>> based
>>> >>>>> on a
>>> >>>>>> plug-in that will create and delete volumes on its storage system
>>> >>>>>> dynamically (as is enabled via the storage framework). When a user
>>> >>>>> wants to
>>> >>>>>> attach a CloudStack volume that was created, the framework tells
>>> the
>>> >>>>>> plug-in to create a new volume. After this is done, the attach
>>> logic
>>> >>>>> for
>>> >>>>>> the hypervisor in question is called. No hypervisor data structure
>>> >>>>> exists
>>> >>>>>> at this point because the volume was just created. The hypervisor
>>> data
>>> >>>>>> structure must be created.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>> >>>>> mike.tutkowski@solidfire.com
>>> >>>>>>> wrote:
>>> >>>>>>
>>> >>>>>>> These are new terms, so I should probably have defined them up
>>> front
>>> >>>>> for
>>> >>>>>>> you. :)
>>> >>>>>>>
>>> >>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>>> creates a
>>> >>>>>>> volume on a SAN), then a hypervisor data structure is created to
>>> >>>>> consume
>>> >>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>>> structure
>>> >>>>> is
>>> >>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>>> >>>>> hypervisor
>>> >>>>>>> data structure as needed. In these cases, the attach logic
>>> assumes
>>> >>>>> the
>>> >>>>>>> hypervisor data structure is already in place and simply attaches
>>> >>>>> the VDI
>>> >>>>>>> on the hypervisor data structure to the VM in question.
>>> >>>>>>>
>>> >>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>>> >>>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN,
>>> >>>>> but the
>>> >>>>>>> SAN itself). The hypervisor data structure must be created when
>>> an
>>> >>>>> attach
>>> >>>>>>> volume is performed because these types of volumes have not been
>>> >>>>> pre-hooked
>>> >>>>>>> up to such a hypervisor data structure by an admin. Once the
>>> attach
>>> >>>>> logic
>>> >>>>>>> creates, say, an SR on XenServer for this volume, it attaches the
>>> >>>>> one and
>>> >>>>>>> only VDI within the SR to the VM in question.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jburwell@basho.com
>>> >
>>> >>>>> wrote:
>>> >>>>>>>
>>> >>>>>>>> Mike,
>>> >>>>>>>>
>>> >>>>>>>> The current implementation of the Dynamic type attach behavior
>>> >>>>> works in
>>> >>>>>>>> terms of Xen ISCSI which why I ask about the difference.
>>> Another
>>> >>>>> way to
>>> >>>>>>>> ask the question -- what is the definition of a Dynamic storage
>>> >>>>> pool type?
>>> >>>>>>>>
>>> >>>>>>>> Thanks,
>>> >>>>>>>> -John
>>> >>>>>>>>
>>> >>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>> >>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
>>> >>>>> when you
>>> >>>>>>>>> want to set up Primary Storage that is directly based on an
>>> iSCSI
>>> >>>>>>>> target.
>>> >>>>>>>>> This allows you to skip the step of going to the hypervisor and
>>> >>>>>>>> creating a
>>> >>>>>>>>> storage repository based on that iSCSI target as CloudStack
>>> does
>>> >>>>> that
>>> >>>>>>>> part
>>> >>>>>>>>> for you. I think this is only supported for XenServer. For all
>>> >>>>> other
>>> >>>>>>>>> hypervisors, you must first go to the hypervisor and perform
>>> this
>>> >>>>> step
>>> >>>>>>>>> manually.
>>> >>>>>>>>>
>>> >>>>>>>>> I don't really know what RBD is.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>> jburwell@basho.com>
>>> >>>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>> Mike,
>>> >>>>>>>>>>
>>> >>>>>>>>>> Reading through the code, what is the difference between the
>>> >>>>> ISCSI and
>>> >>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>> >>>>>>>>>>
>>> >>>>>>>>>> Thanks,
>>> >>>>>>>>>> -John
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>> >>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> This new type of storage is defined in the
>>> >>>>> Storage.StoragePoolType
>>> >>>>>>>> class
>>> >>>>>>>>>>> (called Dynamic):
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> public static enum StoragePoolType {
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Filesystem(false), // local directory
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> ISO(false), // for iso image
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> CLVM(true),
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> RBD(true),
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> SharedMountPoint(true),
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up
>>> by
>>> >>>>>>>>>>> customers.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> OCFS2(true),
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>> SolidFire)
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> boolean shared;
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> StoragePoolType(boolean shared) {
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> this.shared = shared;
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> }
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> public boolean isShared() {
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> return shared;
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> }
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> }
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>> >>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> For example, let's say another storage company wants to
>>> >>>>> implement a
>>> >>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
>>> would be
>>> >>>>>>>> dynamic,
>>> >>>>>>>>>>>> zone-wide storage, as well. They would need only implement a
>>> >>>>> storage
>>> >>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>> >>>>> hypervisor-attach
>>> >>>>>>>>>> logic
>>> >>>>>>>>>>>> to support their plug-in.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>>> specific.
>>> >>>>> It is
>>> >>>>>>>> not.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>>> zone-wide
>>> >>>>>>>> storage
>>> >>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>>> storage).
>>> >>>>> This
>>> >>>>>>>>>> kind of
>>> >>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>>> changes.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Edison created a new framework that supported the creation
>>> and
>>> >>>>>>>> deletion
>>> >>>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
>>> >>>>> Portland
>>> >>>>>>>>>> back
>>> >>>>>>>>>>>>> in April, we realized that it was not complete. We realized
>>> >>>>> there
>>> >>>>>>>> was
>>> >>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
>>> >>>>> attach
>>> >>>>>>>> logic
>>> >>>>>>>>>> was
>>> >>>>>>>>>>>>> changed to recognize this new type of storage and create
>>> the
>>> >>>>>>>>>> appropriate
>>> >>>>>>>>>>>>> hypervisor data structure.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>> >>>>> jburwell@basho.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> It is generally odd to me that any operation in the
>>> Storage
>>> >>>>> layer
>>> >>>>>>>>>> would
>>> >>>>>>>>>>>>>> understand or care about details. I expect to see the
>>> Storage
>>> >>>>>>>>>> services
>>> >>>>>>>>>>>>>> expose a set of operations that can be composed/driven by
>>> the
>>> >>>>>>>>>> Hypervisor
>>> >>>>>>>>>>>>>> implementations to allocate space/create structures per
>>> their
>>> >>>>>>>> needs.
>>> >>>>>>>>>> If
>>> >>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>>> >>>>> massive
>>> >>>>>>>>>> n-to-n
>>> >>>>>>>>>>>>>> problem that will make the system increasingly difficult
>>> to
>>> >>>>>>>> maintain
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>>> SolidFire
>>> >>>>> code
>>> >>>>>>>> is
>>> >>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>>> XenServer,
>>> >>>>> the
>>> >>>>>>>> admin
>>> >>>>>>>>>>>>>> had
>>> >>>>>>>>>>>>>>> to first create a volume on the storage system, then go
>>> into
>>> >>>>> the
>>> >>>>>>>>>>>>>> hypervisor
>>> >>>>>>>>>>>>>>> to set up a data structure to make use of the volume
>>> (ex. a
>>> >>>>>>>> storage
>>> >>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs
>>> and
>>> >>>>> data
>>> >>>>>>>> disks
>>> >>>>>>>>>>>>>> then
>>> >>>>>>>>>>>>>>> shared this storage system's volume.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
>>> longer
>>> >>>>> be so
>>> >>>>>>>>>>>>>> static
>>> >>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
>>> >>>>> storage
>>> >>>>>>>>>> system's
>>> >>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
>>> Quality
>>> >>>>> of
>>> >>>>>>>>>>>>>> Service).
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
>>> >>>>> create
>>> >>>>>>>> and
>>> >>>>>>>>>>>>>> delete
>>> >>>>>>>>>>>>>>> volumes as needed.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> The problem that the storage framework did not address
>>> is in
>>> >>>>>>>> creating
>>> >>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>>> >>>>> performing an
>>> >>>>>>>>>>>>>>> attach/detach.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so.
>>> I've
>>> >>>>> got
>>> >>>>>>>>>>>>>> XenServer
>>> >>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox
>>> and
>>> >>>>> can
>>> >>>>>>>>>> submit
>>> >>>>>>>>>>>>>> this
>>> >>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Does that help a bit? :)
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Hi John,
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>>> agnostic.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>>> >>>>> code).
>>> >>>>>>>> The
>>> >>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
>>> volume
>>> >>>>> as
>>> >>>>>>>>>>>>>> needed,
>>> >>>>>>>>>>>>>>> but
>>> >>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>>> hypervisor,
>>> >>>>> the
>>> >>>>>>>> attach
>>> >>>>>>>>>>>>>>> logic
>>> >>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked
>>> on
>>> >>>>>>>> zone-wide
>>> >>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>> (where the volume has just been created) and create,
>>> say, a
>>> >>>>>>>> storage
>>> >>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware)
>>> to
>>> >>>>> make
>>> >>>>>>>> use
>>> >>>>>>>>>>>>>> of the
>>> >>>>>>>>>>>>>>>> volume that was just created.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> I've been spending most of my time recently making the
>>> >>>>> attach
>>> >>>>>>>> logic
>>> >>>>>>>>>>>>>> work
>>> >>>>>>>>>>>>>>>> in the agent code.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Does that clear it up?
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>> >>>>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>>> hypervisor
>>> >>>>>>>> specific?
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>>> hypervisors
>>> >>>>> that
>>> >>>>>>>>>>>>>>> CloudStack
>>> >>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get
>>> KVM
>>> >>>>> in.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending
>>> on
>>> >>>>> what
>>> >>>>>>>> we do
>>> >>>>>>>>>>>>>>> with
>>> >>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>> >>>>> VMware in
>>> >>>>>>>> my
>>> >>>>>>>>>>>>>>>>> sandbox,
>>> >>>>>>>>>>>>>>>>>> as well.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>> >>>>>>>> Offerings.
>>> >>>>>>>>>>>>>> I
>>> >>>>>>>>>>>>>>>>> plan to
>>> >>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
>>> Damage <
>>> >>>>>>>>>>>>>>> kelcey@bbits.ca
>>> >>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch
>>> cycle
>>> >>>>> post
>>> >>>>>>>> 4.2?
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>> >>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mike.tutkowski@solidfire.com
>>> >
>>> >>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>> >>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>> >>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non
>>> issue
>>> >>>>> as I
>>> >>>>>>>> don't
>>> >>>>>>>>>>>>>>>>> support
>>> >>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>> >>>>>>>> ustcweizhou@gmail.com>
>>> >>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> John,
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
>>> users
>>> >>>>> need
>>> >>>>>>>> to
>>> >>>>>>>>>>>>>>>>> calculate
>>> >>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>> >>>>> service_offering
>>> >>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation.
>>> Of
>>> >>>>> course
>>> >>>>>>>>>>>>>> it is
>>> >>>>>>>>>>>>>>>>>>> better
>>> >>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>>> individual
>>> >>>>> volume,
>>> >>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>> >>>>> later. Do
>>> >>>>>>>> you
>>> >>>>>>>>>>>>>>>>> think it
>>> >>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Wei,
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>> >>>>> ustcweizhou@gmail.com
>>> >>>>>>>>>
>>> >>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
>>> adding
>>> >>>>> more.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>>> >>>>> rather
>>> >>>>>>>> than
>>> >>>>>>>>>>>>>>> IOPS
>>> >>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>>> >>>>> time.
>>> >>>>>>>>>>>>>>>>>>> disk_io_stat
>>> >>>>>>>>>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>>> read/write
>>> >>>>> for
>>> >>>>>>>> an
>>> >>>>>>>>>>>>>>>>>>> individual
>>> >>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more
>>> for
>>> >>>>>>>> volumes
>>> >>>>>>>>>>>>>> with
>>> >>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations
>>> (
>>> >>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
>>> >>>>> users
>>> >>>>>>>> would
>>> >>>>>>>>>>>>>> like
>>> >>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>>> >>>>> operator
>>> >>>>>>>>>>>>>> implement
>>> >>>>>>>>>>>>>>>>>>> such a
>>> >>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>>> >>>>> setting
>>> >>>>>>>>>>>>>> IOPS/BPS
>>> >>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>> >>>>> command
>>> >>>>>>>> line.
>>> >>>>>>>>>>>>>>>>>>> However,
>>> >>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>>> parameters
>>> >>>>> of a
>>> >>>>>>>>>>>>>> created
>>> >>>>>>>>>>>>>>>>>>>> offering
>>> >>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize
>>> for
>>> >>>>> being
>>> >>>>>>>>>>>>>>> unclear.
>>> >>>>>>>>>>>>>>>>>>> Can
>>> >>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a
>>> set of
>>> >>>>>>>>>>>>>> interfaces
>>> >>>>>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>>> resource?
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>>> >>>>> commit
>>> >>>>>>>> Mike's
>>> >>>>>>>>>>>>>>> patch
>>> >>>>>>>>>>>>>>>>>>>>> after
>>> >>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>> >>>>>>>> limitation in
>>> >>>>>>>>>>>>>>>>>>>> hypervisor
>>> >>>>>>>>>>>>>>>>>>>>>> type, I think.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
>>> >>>>> Mike
>>> >>>>>>>>>>>>>> regarding
>>> >>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
>>> that
>>> >>>>>>>>>>>>>>>>>>> disk_io_throttling
>>> >>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I
>>> think
>>> >>>>> we need
>>> >>>>>>>>>>>>>> closer
>>> >>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have
>>> have
>>> >>>>>>>> solidfire
>>> >>>>>>>>>>>>>>> track
>>> >>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> - Wei
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>>> following:
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>> >>>>> captured
>>> >>>>>>>> in
>>> >>>>>>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>>> common
>>> >>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>> >>>>> expressing
>>> >>>>>>>>>>>>>>> offerings).
>>> >>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>> >>>>> reasoning
>>> >>>>>>>>>>>>>> about
>>> >>>>>>>>>>>>>>>>>>> IOP
>>> >>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>> >>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>> >>>>> configurations
>>> >>>>>>>>>>>>>> between
>>> >>>>>>>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>> >>>>> particular,
>>> >>>>>>>> a
>>> >>>>>>>>>>>>>>>>> scenario
>>> >>>>>>>>>>>>>>>>>>>>> where
>>> >>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>> >>>>> configured
>>> >>>>>>>> for
>>> >>>>>>>>>>>>>> a VM
>>> >>>>>>>>>>>>>>>>>>>> than a
>>> >>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>> >>>>> Another
>>> >>>>>>>>>>>>>> scenario
>>> >>>>>>>>>>>>>>>>>>>> could a
>>> >>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>>> storage
>>> >>>>>>>> device at
>>> >>>>>>>>>>>>>>>>>>> creation
>>> >>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>> >>>>>>>> introduces
>>> >>>>>>>>>>>>>>> logical
>>> >>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling
>>> the
>>> >>>>> max
>>> >>>>>>>>>>>>>> number of
>>> >>>>>>>>>>>>>>>>>>> IOPS
>>> >>>>>>>>>>>>>>>>>>>>> from
>>> >>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from
>>> the
>>> >>>>> storage
>>> >>>>>>>>>>>>>> system
>>> >>>>>>>>>>>>>>>>>>>> side.
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>> >>>>>>>>>>>>>> jburwell@basho.com
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>>> >>>>> until all
>>> >>>>>>>>>>>>>>>>>>> functional
>>> >>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to
>>> turn
>>> >>>>> over to
>>> >>>>>>>>>>>>>> test.
>>> >>>>>>>>>>>>>>>>>>>> Until
>>> >>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>> >>>>> addressed
>>> >>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>> re-reviewed
>>> >>>>>>>>>>>>>>>>>>>>>>> by
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>> >>>>> satisfied.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment
>>> the
>>> >>>>>>>> SolidFire
>>> >>>>>>>>>>>>>>> patch
>>> >>>>>>>>>>>>>>>>> (
>>> >>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>> >>>>> understand
>>> >>>>>>>> it
>>> >>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>> work
>>> >>>>>>>>>>>>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>>> >>>>> ensure
>>> >>>>>>>> we
>>> >>>>>>>>>>>>>> don't
>>> >>>>>>>>>>>>>>>>>>>> have a
>>> >>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>> >>>>> SolidFire are
>>> >>>>>>>>>>>>>>>>>>>> unnecessarily
>>> >>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>> >>>>>>>> ustcweizhou@gmail.com
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>>> >>>>> Monday,
>>> >>>>>>>> I
>>> >>>>>>>>>>>>>> will
>>> >>>>>>>>>>>>>>>>>>>> revert
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> them.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
>>> again
>>> >>>>>>>> that we
>>> >>>>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>>>>>>> change
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>>> >>>>> release
>>> >>>>>>>> a
>>> >>>>>>>>>>>>>>> version
>>> >>>>>>>>>>>>>>>>>>>> with
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling
>>> and
>>> >>>>> that's
>>> >>>>>>>>>>>>>> great!
>>> >>>>>>>>>>>>>>>>> We
>>> >>>>>>>>>>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day
>>> #1.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
>>> support
>>> >>>>> read
>>> >>>>>>>> and
>>> >>>>>>>>>>>>>> write
>>> >>>>>>>>>>>>>>>>>>>> IOPS.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> I
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>>> >>>>> don't see
>>> >>>>>>>>>>>>>> how we
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> correctly
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>>> Therefore, we
>>> >>>>>>>> would
>>> >>>>>>>>>>>>>> be
>>> >>>>>>>>>>>>>>>>> stuck
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> with a
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>> >>>>>>>>>>>>>> compatibility.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>>> >>>>> that it
>>> >>>>>>>>>>>>>> will go
>>> >>>>>>>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the
>>> 4.2
>>> >>>>>>>> release
>>> >>>>>>>>>>>>>> so
>>> >>>>>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>>> 4.2
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
>>> >>>>> Total
>>> >>>>>>>> IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze
>>> date.
>>> >>>>> But if
>>> >>>>>>>> the
>>> >>>>>>>>>>>>>>>>> window
>>> >>>>>>>>>>>>>>>>>>>>> moves
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary
>>> to do
>>> >>>>>>>> that.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>>> association
>>> >>>>>>>> between
>>> >>>>>>>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander
>>> <
>>> >>>>>>>>>>>>>> wido@widodh.nl
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>>> >>>>> decided to
>>> >>>>>>>>>>>>>> ignore
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>>> >>>>> network
>>> >>>>>>>>>>>>>>> throttling
>>> >>>>>>>>>>>>>>>>>>>> did
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes
>>> as
>>> >>>>> well.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk
>>> I/O
>>> >>>>>>>> completely
>>> >>>>>>>>>>>>>>>>>>>> different.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>>> situations
>>> >>>>> then
>>> >>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>>> >>>>> could be
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I
>>> will
>>> >>>>>>>>>>>>>> implement
>>> >>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> after
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
>>> >>>>> goes
>>> >>>>>>>> into
>>> >>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>> 4.2
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release
>>> with
>>> >>>>> only
>>> >>>>>>>> total
>>> >>>>>>>>>>>>>>> IOps
>>> >>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> fix
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch
>>> into
>>> >>>>>>>> master.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>>> >>>>> hours.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>>> >>>>> device
>>> >>>>>>>>>>>>>> (local
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> storage or
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>>> >>>>> device
>>> >>>>>>>>>>>>>> (such as
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect
>>> the
>>> >>>>> disk
>>> >>>>>>>>>>>>>>>>> performance
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and
>>> limit
>>> >>>>> the
>>> >>>>>>>> disk
>>> >>>>>>>>>>>>>> I/O
>>> >>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>>> >>>>> between
>>> >>>>>>>>>>>>>> Read
>>> >>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>>> different
>>> >>>>> rate
>>> >>>>>>>> for
>>> >>>>>>>>>>>>>> Read
>>> >>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>>> collect
>>> >>>>> both
>>> >>>>>>>> the
>>> >>>>>>>>>>>>>> Read
>>> >>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go
>>> for a
>>> >>>>> global
>>> >>>>>>>>>>>>>> value.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then
>>> Read
>>> >>>>>>>> IOps,
>>> >>>>>>>>>>>>>> so it
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> seems
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would
>>> set
>>> >>>>> a
>>> >>>>>>>> lower
>>> >>>>>>>>>>>>>>> value
>>> >>>>>>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> write
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
>>> think
>>> >>>>> it
>>> >>>>>>>> would
>>> >>>>>>>>>>>>>> be
>>> >>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> great
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>>> >>>>> support
>>> >>>>>>>>>>>>>> both,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> implementing
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>>> different
>>> >>>>> values
>>> >>>>>>>> for
>>> >>>>>>>>>>>>>>> read
>>> >>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that
>>> as the
>>> >>>>>>>> total
>>> >>>>>>>>>>>>>>> limit.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>> disk_offering,
>>> >>>>> and
>>> >>>>>>>>>>>>>> global
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>> >>>>>>>>>>>>>> issues.apache.org/****
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>> >>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> >>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>> >>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> >>>>>>>> https://cwiki.
>>> >>>>>>>>>>>>>> **
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> >>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>
>>> >>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>>> success?
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Yes
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> No
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration)
>>> is
>>> >>>>>>>> included
>>> >>>>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> new
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> feature?
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>>> potential
>>> >>>>>>>>>>>>>> regressions?
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>>> CloudStack
>>> >>>>> UI.
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>> >>>>> migrate,
>>> >>>>>>>>>>>>>> restore
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> git diff
>>> >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> [1]
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>> >>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> >>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>> >>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> >>>>>>>> https://cwiki.
>>> >>>>>>>>>>>>>> **
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> >>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>
>>> >>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> [3]
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>> >>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>> >>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>> >>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> **<
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>> >>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>> >>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>>>>> cloud<
>>> >>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play
>>> >>>>>>
>>> >>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>> >
>>> >>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> --
>>> >>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> --
>>> >>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>> o: 303.746.7302
>>> >>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>> *â„¢*
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> *Mike Tutkowski*
>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>> o: 303.746.7302
>>> >>>>>>> Advancing the way the world uses the cloud<
>>> >>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>> *â„¢*
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> *Mike Tutkowski*
>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>> o: 303.746.7302
>>> >>>>>> Advancing the way the world uses the
>>> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>> *â„¢*
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> *Mike Tutkowski*
>>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>> e: mike.tutkowski@solidfire.com
>>> >>>> o: 303.746.7302
>>> >>>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>> *â„¢*
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Mike Tutkowski*
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>> e: mike.tutkowski@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>> *â„¢*
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> *Mike Tutkowski*
>>> >> *Senior CloudStack Developer, SolidFire Inc.*
>>> >> e: mike.tutkowski@solidfire.com
>>> >> o: 303.746.7302
>>> >> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >> *â„¢*
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I guess you could say that prior to 4.2, the hypervisor's attach and detach
logic assumed un-managed (static) storage.
As of 4.2, the possibility of managed (dynamic) storage is now present and
is currently being denoted by the type "Dynamic" in the StoragePoolType
class.
On Tue, Jun 4, 2013 at 3:28 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> "So, in essence, the SolidFire plugin introduces the notion of a managed
> iSCSI device and provisioned IOPS."
>
> Technically, the SolidFire plug-in just introduces the notion of
> provisioned storage IOPS.
>
> The storage framework that leverages the plug-in was incomplete, so I had
> to try to add in the notion of a managed iSCSI device.
>
> I appreciate all the time you've been spending on this. :) Do you have a
> recommendation as to how we should accomplish what you're looking for?
>
> Thanks!
>
>
> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> So, in essence, the SolidFire plugin introduces the notion of a managed
>> iSCSI device and provisioned IOPS.
>>
>> I want to see a separation of the management capabilities (i.e. can a
>> device be managed/does an operator want it managed by CloudStack) from the
>> storage protocol. Ideally, we should end up with a semantic that will
>> allow any type of storage device to be managed. I also want to make
>> progress on decoupling the storage types from the hypervisor definitions.
>>
>> Thanks,
>> -John
>>
>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > Hi John,
>> >
>> > No problem. Answers are below in red.
>> >
>> > Thanks!
>> >
>> >
>> > On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Mike,
>> >>
>> >> Could you please answer the following questions for me with regards to
>> the
>> >> operation of the SolidFire plugin:
>> >>
>> >> What is the cardinality between iSCSI LUNs and SAN volumes?
>> >>
>> > Each SAN volume is equivalent to a single LUN (LUN 0).
>> > 1 SAN volume : 1 LUN
>> >
>> >> What is the cardinality between SAN Volumes and CloudStack Volumes?
>> >>
>> > 1 SAN volume : 1 CloudStack volume
>> >
>> >> Are the LUN(s) created by the management server or externally by the
>> >> operator?
>> >>
>> > When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN)
>> is
>> > created by the management server (via the plug-in) the first time the
>> > CloudStack volume is attached to a hypervisor.
>> >
>> > If you don't want to use the SolidFire plug-in, but still want to use a
>> > SolidFire volume (LUN), you can do this already today (prior to 4.2).
>> The
>> > admin manually creates the SAN volume and - in this case - multiple VMs
>> and
>> > data disks can share this SAN volume. While you can do this today, it is
>> > not useful if you want to enforce storage QoS.
>> >
>> >> Are the SAN volumes by the management server or externally by the
>> operator?
>> >>
>> > When the SolidFire plug-in is used, the SAN volumes are completely
>> managed
>> > by the management server (via the plug-in). There is no admin
>> interaction.
>> > This allows for a 1:1 mapping between a SAN volume and a CloudStack
>> volume,
>> > which is necessary for any storage vendor that supports true QoS.
>> >
>> >>
>> >> I would like to clarify how these pieces are related and expected to
>> >> operate.
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >> wrote:
>> >>
>> >>> "In particular, how do we ensure that multiple VMs with provisioned
>> IOPS
>> >>> won't be cut off by the underlying storage."
>> >>>
>> >>> In the storage QoS world, we need to map a single SAN volume (LUN) to
>> a
>> >>> single CloudStack volume. We cannot have multiple CloudStack volumes
>> >>> sharing a single SAN volume and still guarantee QoS.
>> >>>
>> >>> If the user wants to have a single SAN volume house more than one
>> >>> CloudStack volume, then can do that today without any of my plug-in
>> code.
>> >>>
>> >>>
>> >>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com
>> >>>> wrote:
>> >>>
>> >>>> "The administrator will allocate a SAN volume for CloudStack's use
>> onto
>> >>>> which CloudStack volumes will be created."
>> >>>>
>> >>>> I think we crossed e-mails. :)
>> >>>>
>> >>>> Check out my recent e-mail on this.
>> >>>>
>> >>>>
>> >>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>>
>> >>>>> Mike,
>> >>>>>
>> >>>>> You are coming to the part which concerns me -- concepts from the
>> >>>>> hypervisor are leaking into storage layer.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> -John
>> >>>>>
>> >>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> The weird part is that the iSCSI type is today only used (as far
>> as I
>> >>>>> know)
>> >>>>>> in regards to XenServer (when you have not PreSetup an SR).
>> >>>>>>
>> >>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
>> >> type.
>> >>>>>>
>> >>>>>> If you want to use your iSCSI volume from KVM, it uses the
>> >>>>> SharedMountPoint
>> >>>>>> type.
>> >>>>>>
>> >>>>>> So, I suppose mine and Edison's thinking here was to make a new
>> type
>> >> of
>> >>>>>> storage to describe this dynamic ability Edison added into the
>> storage
>> >>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
>> >>>>> versus,
>> >>>>>> say, Dynamic_FC.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com
>> >>>>>>> wrote:
>> >>>>>>
>> >>>>>>> "The storage device itself shouldn't know or care that it is being
>> >> used
>> >>>>>>> for a Xen SR -- simply be able to answer questions about it is
>> >>>>> storing."
>> >>>>>>>
>> >>>>>>> I see...so your concern here is that the SolidFire plug-in needs
>> to
>> >>>>> call
>> >>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
>> treat
>> >>>>> it as
>> >>>>>>> such.
>> >>>>>>>
>> >>>>>>> I'm totally open to removing that constraint and just calling it
>> >> iSCSI
>> >>>>> or
>> >>>>>>> whatever. We would just need a way for the hypervisor attach
>> logic to
>> >>>>>>> detect this new requirement and perform the necessary activities.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
>> >>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> See my responses in-line.
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> I'm trying to picture this:
>> >>>>>>>>>
>> >>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
>> >>>>> operator
>> >>>>>>>> may
>> >>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
>> >> shared
>> >>>>> by
>> >>>>>>>>> many services, and the operator has chosen to dedicate only a
>> >> portion
>> >>>>>>>> of it
>> >>>>>>>>> to CloudStack). Does my reasoning make sense?"
>> >>>>>>>>>
>> >>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
>> >>>>> (before
>> >>>>>>>> an
>> >>>>>>>>> attach request to the hypervisor) would work unless the
>> hypervisor
>> >>>>>>>> consumes
>> >>>>>>>>> the SAN volume in the form of, say, an SR.
>> >>>>>>>>
>> >>>>>>>> My thinking is that, independent of CloudStack, an operator
>> >> allocates
>> >>>>> a
>> >>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN.
>> They
>> >>>>> simply
>> >>>>>>>> want to turn control of that LUN over to CloudStack, but not
>> allow
>> >>>>>>>> CloudStack to allocate anymore LUNs.
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> As the attach logic stands prior to my changes, we would be
>> passing
>> >>>>> in a
>> >>>>>>>>> SAN volume that does not have the necessary hypervisor support
>> >> (like
>> >>>>> an
>> >>>>>>>> SR)
>> >>>>>>>>> and the logic will fail.
>> >>>>>>>>>
>> >>>>>>>>> Are you thinking we should maybe have the storage framework
>> itself
>> >>>>>>>> detect
>> >>>>>>>>> that such a SAN volume needs support from the hypervisor side
>> and
>> >>>>> have
>> >>>>>>>> it
>> >>>>>>>>> call into the agent code specifically to create the SR before
>> the
>> >>>>> attach
>> >>>>>>>>> logic runs in the agent code?
>> >>>>>>>>
>> >>>>>>>> I think the hypervisor management plugin should have a rich
>> enough
>> >>>>>>>> interface to storage to determine available for volume storage.
>> For
>> >>>>> Xen,
>> >>>>>>>> this interface would allow the interrogation of the device to
>> >>>>> determine the
>> >>>>>>>> SR is present. The storage device itself shouldn't know or care
>> >>>>> that it
>> >>>>>>>> is being used for a Xen SR -- simply be able to answer questions
>> >>>>> about it
>> >>>>>>>> is storing.
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> So, the flow is as follows:
>> >>>>>>>>>>
>> >>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
>> >>>>>>>> so-called
>> >>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage
>> shows
>> >> up
>> >>>>> in
>> >>>>>>>> the
>> >>>>>>>>>> applicable zone.
>> >>>>>>>>>>
>> >>>>>>>>>> * The admin creates a Disk Offering that references the storage
>> >> tag
>> >>>>> of
>> >>>>>>>> the
>> >>>>>>>>>> newly created Primary Storage.
>> >>>>>>>>>>
>> >>>>>>>>>> * The end user creates a CloudStack volume. This leads to a new
>> >> row
>> >>>>> in
>> >>>>>>>> the
>> >>>>>>>>>> cloud.volumes table.
>> >>>>>>>>>>
>> >>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
>> >> disk).
>> >>>>>>>> This
>> >>>>>>>>>> leads to the storage framework calling the plug-in to create a
>> new
>> >>>>>>>> volume
>> >>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
>> >> updates
>> >>>>> the
>> >>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
>> >>>>>>>> volume).
>> >>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is
>> in
>> >> the
>> >>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
>> >>>>> 'Ready'
>> >>>>>>>>>> state (even after a detach disk) and the plug-in code will not
>> be
>> >>>>>>>> called
>> >>>>>>>>>> again to create this SAN volume.
>> >>>>>>>>>>
>> >>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
>> >>>>> volume
>> >>>>>>>> to
>> >>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>> >> structure
>> >>>>>>>> (ex.
>> >>>>>>>>>> an SR on XenServer).
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in
>> these
>> >>>>>>>>>>> definitions, refers to the physical allocation on the device,
>> >>>>>>>> correct?"
>> >>>>>>>>>>>
>> >>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>> >>>>>>>>>>>
>> >>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack,
>> I
>> >>>>> try to
>> >>>>>>>>>>> use 'CloudStack volume'.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> What you say here may very well make sense, but I'm having a
>> >> hard
>> >>>>>>>> time
>> >>>>>>>>>>>> envisioning it.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
>> was
>> >>>>> the
>> >>>>>>>>>>>> initial person to suggest the approach I took.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> What do you think?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>> >> jburwell@basho.com
>> >>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> It feels like we are combining two distinct concepts --
>> storage
>> >>>>>>>> device
>> >>>>>>>>>>>>> management and storage protocols. In both cases, we are
>> >>>>>>>> communicating with
>> >>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
>> >>>>> (Dynamic)
>> >>>>>>>> on the
>> >>>>>>>>>>>>> device while the other requires the volume to be volume to
>> be
>> >>>>>>>> managed
>> >>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
>> >>>>> sync on
>> >>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
>> >> physical
>> >>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
>> able
>> >> to
>> >>>>>>>>>>>>> communicate with a storage device to move bits from one
>> place
>> >> to
>> >>>>>>>> another,
>> >>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device
>> may
>> >> be
>> >>>>>>>> able to
>> >>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
>> >> iSCSI
>> >>>>>>>> device
>> >>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>> >> SolidFire
>> >>>>>>>> iSCSI
>> >>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
>> >>>>> storing
>> >>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
>> >>>>> device,
>> >>>>>>>> an
>> >>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
>> >> the
>> >>>>>>>> device is
>> >>>>>>>>>>>>> shared by many services, and the operator has chosen to
>> >> dedicate
>> >>>>>>>> only a
>> >>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>> appropriate
>> >>>>> to
>> >>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
>> >>>>> notion
>> >>>>>>>> of a
>> >>>>>>>>>>>>> storage device with an attached driver that indicates
>> whether
>> >> or
>> >>>>>>>> not is
>> >>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>> >> representing a
>> >>>>>>>> physical
>> >>>>>>>>>>>>> allocation on a device separate that is associated with a
>> >> volume.
>> >>>>>>>> With
>> >>>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
>> >>>>>>>> protocols they
>> >>>>>>>>>>>>> support and when they encounter a device managed by
>> CloudStack,
>> >>>>>>>> utilize the
>> >>>>>>>>>>>>> management operations exposed by the driver to automate
>> >>>>> allocation.
>> >>>>>>>> If
>> >>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
>> >>>>> drill
>> >>>>>>>> down to
>> >>>>>>>>>>>>> a more detailed design.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
>> the
>> >>>>>>>> Dynamic
>> >>>>>>>>>>>>> type:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
>> >> Storage
>> >>>>>>>> based
>> >>>>>>>>>>>>> on
>> >>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume
>> must
>> >>>>> exist
>> >>>>>>>> on
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
>> >> volumes
>> >>>>> are
>> >>>>>>>>>>>>> created
>> >>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
>> >> Storage
>> >>>>>>>> based
>> >>>>>>>>>>>>> on a
>> >>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
>> >>>>> system
>> >>>>>>>>>>>>>> dynamically (as is enabled via the storage framework).
>> When a
>> >>>>> user
>> >>>>>>>>>>>>> wants to
>> >>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
>> >> tells
>> >>>>>>>> the
>> >>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
>> attach
>> >>>>>>>> logic
>> >>>>>>>>>>>>> for
>> >>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
>> >>>>> structure
>> >>>>>>>>>>>>> exists
>> >>>>>>>>>>>>>> at this point because the volume was just created. The
>> >>>>> hypervisor
>> >>>>>>>> data
>> >>>>>>>>>>>>>> structure must be created.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> These are new terms, so I should probably have defined
>> them
>> >> up
>> >>>>>>>> front
>> >>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>> you. :)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an
>> admin
>> >>>>>>>> creates a
>> >>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
>> created
>> >>>>> to
>> >>>>>>>>>>>>> consume
>> >>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>> >>>>>>>> structure
>> >>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
>> this
>> >>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
>> >>>>> assumes
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
>> >>>>> attaches
>> >>>>>>>>>>>>> the VDI
>> >>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
>> Instead
>> >> of
>> >>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume
>> on a
>> >>>>> SAN,
>> >>>>>>>>>>>>> but the
>> >>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
>> >>>>> when an
>> >>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>> volume is performed because these types of volumes have
>> not
>> >>>>> been
>> >>>>>>>>>>>>> pre-hooked
>> >>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
>> the
>> >>>>>>>> attach
>> >>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
>> attaches
>> >>>>> the
>> >>>>>>>>>>>>> one and
>> >>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>> >>>>> jburwell@basho.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
>> >> behavior
>> >>>>>>>>>>>>> works in
>> >>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
>> >>>>> Another
>> >>>>>>>>>>>>> way to
>> >>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
>> >>>>> storage
>> >>>>>>>>>>>>> pool type?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>> >>>>> XenServer
>> >>>>>>>>>>>>> when you
>> >>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based
>> on an
>> >>>>>>>> iSCSI
>> >>>>>>>>>>>>>>>> target.
>> >>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>> hypervisor
>> >>>>> and
>> >>>>>>>>>>>>>>>> creating a
>> >>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>> CloudStack
>> >>>>> does
>> >>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>> part
>> >>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
>> For
>> >>>>> all
>> >>>>>>>>>>>>> other
>> >>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
>> >> perform
>> >>>>>>>> this
>> >>>>>>>>>>>>> step
>> >>>>>>>>>>>>>>>>> manually.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I don't really know what RBD is.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>> >>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
>> between
>> >> the
>> >>>>>>>>>>>>> ISCSI and
>> >>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>> >>>>>>>>>>>>> Storage.StoragePoolType
>> >>>>>>>>>>>>>>>> class
>> >>>>>>>>>>>>>>>>>>> (called Dynamic):
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>> overlay
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> CLVM(true),
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> RBD(true),
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
>> >> up
>> >>>>>>>> by
>> >>>>>>>>>>>>>>>>>>> customers.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> OCFS2(true),
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>> >>>>>>>> SolidFire)
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> boolean shared;
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> this.shared = shared;
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> public boolean isShared() {
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> return shared;
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants
>> to
>> >>>>>>>>>>>>> implement a
>> >>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature.
>> It
>> >>>>> would
>> >>>>>>>> be
>> >>>>>>>>>>>>>>>> dynamic,
>> >>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>> >>>>> implement a
>> >>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>> >>>>>>>>>>>>> hypervisor-attach
>> >>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>> to support their plug-in.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>> >>>>> specific.
>> >>>>>>>>>>>>> It is
>> >>>>>>>>>>>>>>>> not.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>> >>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>> >>>>>>>> storage).
>> >>>>>>>>>>>>> This
>> >>>>>>>>>>>>>>>>>> kind of
>> >>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
>> framework
>> >>>>>>>> changes.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
>> >>>>> creation
>> >>>>>>>> and
>> >>>>>>>>>>>>>>>> deletion
>> >>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
>> >> him
>> >>>>> in
>> >>>>>>>>>>>>> Portland
>> >>>>>>>>>>>>>>>>>> back
>> >>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
>> >>>>> realized
>> >>>>>>>>>>>>> there
>> >>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes
>> unless
>> >> the
>> >>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
>> >> create
>> >>>>> the
>> >>>>>>>>>>>>>>>>>> appropriate
>> >>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>> >>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
>> >>>>> Storage
>> >>>>>>>>>>>>> layer
>> >>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see
>> the
>> >>>>>>>> Storage
>> >>>>>>>>>>>>>>>>>> services
>> >>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>> composed/driven
>> >>>>> by
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>>>> Hypervisor
>> >>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
>> >> per
>> >>>>>>>> their
>> >>>>>>>>>>>>>>>> needs.
>> >>>>>>>>>>>>>>>>>> If
>> >>>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
>> >> with a
>> >>>>>>>>>>>>> massive
>> >>>>>>>>>>>>>>>>>> n-to-n
>> >>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>> >>>>> difficult to
>> >>>>>>>>>>>>>>>> maintain
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>> >>>>>>>> SolidFire
>> >>>>>>>>>>>>> code
>> >>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>> >>>>>>>> XenServer,
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> admin
>> >>>>>>>>>>>>>>>>>>>>>> had
>> >>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
>> then
>> >> go
>> >>>>>>>> into
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
>> volume
>> >>>>> (ex.
>> >>>>>>>> a
>> >>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
>> >> VMs
>> >>>>> and
>> >>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>>> disks
>> >>>>>>>>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need
>> no
>> >>>>>>>> longer
>> >>>>>>>>>>>>> be so
>> >>>>>>>>>>>>>>>>>>>>>> static
>> >>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
>> between
>> >> a
>> >>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>> system's
>> >>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
>> storage
>> >>>>>>>> Quality
>> >>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>> Service).
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>> >>>>> dynamically
>> >>>>>>>>>>>>> create
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>> delete
>> >>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
>> >> address
>> >>>>> is
>> >>>>>>>> in
>> >>>>>>>>>>>>>>>> creating
>> >>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
>> when
>> >>>>>>>>>>>>> performing an
>> >>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do
>> so.
>> >>>>> I've
>> >>>>>>>>>>>>> got
>> >>>>>>>>>>>>>>>>>>>>>> XenServer
>> >>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
>> >> sandbox
>> >>>>>>>> and
>> >>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>> submit
>> >>>>>>>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>> >>>>> agnostic.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
>> >>>>> agent
>> >>>>>>>>>>>>> code).
>> >>>>>>>>>>>>>>>> The
>> >>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
>> create a
>> >>>>>>>> volume
>> >>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>> needed,
>> >>>>>>>>>>>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>> >>>>> hypervisor,
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>> >>>>> invoked on
>> >>>>>>>>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
>> create,
>> >>>>>>>> say, a
>> >>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>> >>>>> VMware) to
>> >>>>>>>>>>>>> make
>> >>>>>>>>>>>>>>>> use
>> >>>>>>>>>>>>>>>>>>>>>> of the
>> >>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently
>> making
>> >> the
>> >>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>> >>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>> >>>>> hypervisor
>> >>>>>>>>>>>>>>>> specific?
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>> >>>>> hypervisors
>> >>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2
>> to
>> >>>>> get
>> >>>>>>>> KVM
>> >>>>>>>>>>>>> in.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>> >>>>> Depending on
>> >>>>>>>>>>>>> what
>> >>>>>>>>>>>>>>>> we do
>> >>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
>> working
>> >> for
>> >>>>>>>>>>>>> VMware in
>> >>>>>>>>>>>>>>>> my
>> >>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>> >>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards
>> to
>> >>>>> Disk
>> >>>>>>>>>>>>>>>> Offerings.
>> >>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>> plan to
>> >>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
>> >>>>> Damage
>> >>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
>> patch
>> >>>>> cycle
>> >>>>>>>>>>>>> post
>> >>>>>>>>>>>>>>>> 4.2?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>> >>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
>> MASTER
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
>> >> mine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
>> non
>> >>>>> issue
>> >>>>>>>>>>>>> as I
>> >>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
>> now,
>> >>>>>>>> users
>> >>>>>>>>>>>>> need
>> >>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>>> calculate
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>> >>>>>>>>>>>>> service_offering
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>> >>>>> calculation.
>> >>>>>>>> Of
>> >>>>>>>>>>>>> course
>> >>>>>>>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> better
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>> >>>>> individual
>> >>>>>>>>>>>>> volume,
>> >>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work
>> on
>> >> it
>> >>>>>>>>>>>>> later. Do
>> >>>>>>>>>>>>>>>> you
>> >>>>>>>>>>>>>>>>>>>>>>>>> think it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>> features.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>> >>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying
>> to
>> >>>>>>>> adding
>> >>>>>>>>>>>>> more.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>> >>>>> statistics
>> >>>>>>>>>>>>> rather
>> >>>>>>>>>>>>>>>> than
>> >>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
>> you
>> >>>>> have
>> >>>>>>>>>>>>> time.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>> >>>>>>>> read/write
>> >>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
>> >> more
>> >>>>>>>> for
>> >>>>>>>>>>>>>>>> volumes
>> >>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>> >>>>> operations (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
>> imagine
>> >>>>> our
>> >>>>>>>>>>>>> users
>> >>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>> like
>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
>> Could an
>> >>>>>>>>>>>>> operator
>> >>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>> >>>>> supports
>> >>>>>>>>>>>>> setting
>> >>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
>> >> through
>> >>>>>>>>>>>>> command
>> >>>>>>>>>>>>>>>> line.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>> >>>>> parameters
>> >>>>>>>>>>>>> of a
>> >>>>>>>>>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>> >> apologize
>> >>>>>>>> for
>> >>>>>>>>>>>>> being
>> >>>>>>>>>>>>>>>>>>>>>>> unclear.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
>> with a
>> >>>>> set
>> >>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>> interfaces
>> >>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by
>> a
>> >>>>>>>> resource?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
>> better
>> >> to
>> >>>>>>>>>>>>> commit
>> >>>>>>>>>>>>>>>> Mike's
>> >>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
>> consider
>> >> the
>> >>>>>>>>>>>>>>>> limitation in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>> >> response
>> >>>>> to
>> >>>>>>>>>>>>> Mike
>> >>>>>>>>>>>>>>>>>>>>>> regarding
>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
>> >> think
>> >>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire,
>> but I
>> >>>>> think
>> >>>>>>>>>>>>> we need
>> >>>>>>>>>>>>>>>>>>>>>> closer
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
>> >> have
>> >>>>>>>> have
>> >>>>>>>>>>>>>>>> solidfire
>> >>>>>>>>>>>>>>>>>>>>>>> track
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
>> >> issue.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>> >>>>> following:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>> policies
>> >> be
>> >>>>>>>>>>>>> captured
>> >>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> common
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>> purposes,
>> >>>>>>>>>>>>> expressing
>> >>>>>>>>>>>>>>>>>>>>>>> offerings).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface
>> model
>> >> for
>> >>>>>>>>>>>>> reasoning
>> >>>>>>>>>>>>>>>>>>>>>> about
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>> >>>>>>>>>>>>> configurations
>> >>>>>>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled?
>> In
>> >>>>>>>>>>>>> particular,
>> >>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
>> IOPS
>> >>>>>>>>>>>>> configured
>> >>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>> a VM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>> >> deliver.
>> >>>>>>>>>>>>> Another
>> >>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and
>> a
>> >>>>>>>> storage
>> >>>>>>>>>>>>>>>> device at
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
>> >>>>> device
>> >>>>>>>>>>>>>>>> introduces
>> >>>>>>>>>>>>>>>>>>>>>>> logical
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
>> Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>> >> controlling
>> >>>>> the
>> >>>>>>>>>>>>> max
>> >>>>>>>>>>>>>>>>>>>>>> number of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
>> >> from
>> >>>>> the
>> >>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>> system
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
>> Burwell
>> >> <
>> >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
>> >>>>> merged
>> >>>>>>>>>>>>> until all
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is
>> ready to
>> >>>>> turn
>> >>>>>>>>>>>>> over to
>> >>>>>>>>>>>>>>>>>>>>>> test.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops
>> issue
>> >> is
>> >>>>>>>>>>>>> addressed
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has
>> been
>> >>>>>>>>>>>>> satisfied.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>> >> intersect/compliment
>> >>>>> the
>> >>>>>>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>> (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
>> As I
>> >>>>>>>>>>>>> understand
>> >>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
>> >> like
>> >>>>> to
>> >>>>>>>>>>>>> ensure
>> >>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM
>> and
>> >>>>>>>>>>>>> SolidFire are
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>> wido@widodh.nl
>> >>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
>> >>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
>> reason
>> >>>>> till
>> >>>>>>>>>>>>> Monday,
>> >>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
>> >> mention
>> >>>>>>>> again
>> >>>>>>>>>>>>>>>> that we
>> >>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that
>> we
>> >>>>> never
>> >>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>> version
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>> >> throttling
>> >>>>>>>> and
>> >>>>>>>>>>>>> that's
>> >>>>>>>>>>>>>>>>>>>>>> great!
>> >>>>>>>>>>>>>>>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy
>> from
>> >> day
>> >>>>>>>> #1.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
>> >>>>> support
>> >>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
>> >> because
>> >>>>> I
>> >>>>>>>>>>>>> don't see
>> >>>>>>>>>>>>>>>>>>>>>> how we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>> >>>>>>>> Therefore, we
>> >>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>> >>>>> backwards
>> >>>>>>>>>>>>>>>>>>>>>> compatibility.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
>> >> now
>> >>>>> so
>> >>>>>>>>>>>>> that it
>> >>>>>>>>>>>>>>>>>>>>>> will go
>> >>>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
>> before
>> >>>>> the
>> >>>>>>>> 4.2
>> >>>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
>> instead
>> >>>>> of
>> >>>>>>>>>>>>> Total
>> >>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
>> freeze
>> >>>>> date.
>> >>>>>>>>>>>>> But if
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> window
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>> >> necessary
>> >>>>>>>> to do
>> >>>>>>>>>>>>>>>> that.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>> >>>>>>>> association
>> >>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>> >>>>> Hollander <
>> >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>> Finally I
>> >>>>>>>>>>>>> decided to
>> >>>>>>>>>>>>>>>>>>>>>> ignore
>> >>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
>> because
>> >>>>> the
>> >>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>> throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
>> >>>>> bytes as
>> >>>>>>>>>>>>> well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
>> >> disk
>> >>>>> I/O
>> >>>>>>>>>>>>>>>> completely
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>> >>>>>>>> situations
>> >>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
>> >> work.
>> >>>>> It
>> >>>>>>>>>>>>> could be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
>> >> freeze,
>> >>>>> I
>> >>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss.
>> But
>> >> if
>> >>>>> it
>> >>>>>>>>>>>>> goes
>> >>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
>> release
>> >>>>> with
>> >>>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>> IOps
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
>> >>>>> branch
>> >>>>>>>> into
>> >>>>>>>>>>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into
>> master
>> >> in
>> >>>>> 48
>> >>>>>>>>>>>>> hours.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
>> >>>>> storage
>> >>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>> (local
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>> >> limitation
>> >>>>> of
>> >>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>> (such as
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
>> >> affect
>> >>>>>>>> the
>> >>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>>>>> performance
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
>> >> device.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
>> and
>> >>>>> limit
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>> I/O
>> >>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>> >>>>> difference
>> >>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>> >>>>> different
>> >>>>>>>>>>>>> rate
>> >>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side
>> you
>> >>>>>>>> collect
>> >>>>>>>>>>>>> both
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you
>> only go
>> >>>>> for
>> >>>>>>>> a
>> >>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>> value.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
>> expensive
>> >>>>> then
>> >>>>>>>> Read
>> >>>>>>>>>>>>>>>> IOps,
>> >>>>>>>>>>>>>>>>>>>>>> so it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
>> >>>>> would
>> >>>>>>>> set
>> >>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>> lower
>> >>>>>>>>>>>>>>>>>>>>>>> value
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
>> point I
>> >>>>>>>> think
>> >>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
>> >> place
>> >>>>> to
>> >>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>> both,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>> >>>>> different
>> >>>>>>>>>>>>> values
>> >>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
>> >> that
>> >>>>> as
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>> limit.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
>> this
>> >>>>> way?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>> >>>>>>>> disk_offering,
>> >>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
>> iops)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>> https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>
>> >>>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
>> execution
>> >>>>>>>> success?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>> >>>>> integration)
>> >>>>>>>> is
>> >>>>>>>>>>>>>>>> included
>> >>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>> >>>>>>>> potential
>> >>>>>>>>>>>>>>>>>>>>>> regressions?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>> >>>>>>>> CloudStack
>> >>>>>>>>>>>>> UI.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
>> >> expunge.
>> >>>>>>>>>>>>> migrate,
>> >>>>>>>>>>>>>>>>>>>>>> restore
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> >>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>
>> >>>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>> >>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>> >>>
>> >>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the
>> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the cloud<
>> >> http://solidfire.com/solution/overview/?video=play>
>> >>>> *â„¢*
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Yes, we are on the same page regarding the managed flag. From my way of thinking, the managed flag should remove the need for StoragePoolType#Dynamic and allow SolidFire devices to be defined as ISCSI -- greatly simplifying the hypervisor end of the business. Is my expectation correct?
Thanks,
-John
On Jun 10, 2013, at 2:10 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> I believe we're on the same page, John.
>
> I would add a new parameter to the createStoragePool API command (let's call it "managed" and it can be true or false). If you pass in "true", but the driver does not support being managed by CloudStack, an exception is thrown by the driver. If you pass in "false", but the driver does not support being managed outside of CloudStack, an exception is thrown by the driver. It is possible one could write a driver that can be managed both inside or outside of CloudStack (the SolidFire driver requires being managed only by CloudStack).
>
> This managed value would end up in the storage_pool table in a new column ("managed").
>
> An example usage: After the storage framework calls on the driver's createAsync method, it will check to see if the driver is managed. If the driver is managed, the storage framework will invoke the proper (new) command of the hypervisor in use to set up, say, the SR for the SAN volume.
>
> There may be other cases when the isManaged() method is invoked on the driver and - if true is returned - the storage framework sends a particular command to the hypervisor in use.
>
> ********************
>
> As far as reworking the default storage plug-in (which sends commands to hypervisors) for 4.2, I would check with Edison on his schedule.
>
> Thanks!
>
>
> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I want to make sure we are on the same page regarding isManaged. I see the addition of the following operations:
>
> * DataStoreDriver#supportsManagement() : boolean -- indicates whether or not the driver support/implements the management functions
> * DataStore#isManaged() : boolean -- indicates whether or not a device is managed by CloudStack
>
> Only DataStores associated with DataStoreDrivers that support management would be able to have the management flag enabled. This behavior allows operators to declare their intention to allow CloudStack to manage a device. This behavior would need to be exposed as flag on the HTTP API. Additionally, this flag would only be settable when a device is defined/created in the system (i.e. once a device is managed by CloudStack, you can't unmanaged it without dissociating it from CloudStack). In the future, we may consider relaxing that rule, but, in the short term, it greatly decreases the alternative flows.
>
> In terms of removing the dependencies from Storage to Hypervisor, I would really like to see this accomplished in 4.2. It will be much more difficult to address this issue post 4.2 because we will have more plugins to address. I also see this issue as a critical architectural issues that for the management server. I recommend we identify the dependencies that need to be inverted, and see if we can divide the work across a few more resources.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>
>> Sounds good, Edison :)
>>
>> I can implement John's isManaged() logic and resubmit my code for review.
>>
>> Perhaps in 4.3 I could address the direct-attach scenario. If it requires some refactoring of the storage framework, then I could do that.
>>
>> Also, the default storage plug-in does have hypervisor logic in it in 4.2. I recommend we leave this as is for 4.2 and it can be refactored in 4.3 with the above work I mentioned.
>>
>> Are you OK with that, guys?
>>
>>
>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com> wrote:
>> Ok, I am Ok with the way you are talking about in 4.2.
>>
>>
>>
>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> Sent: Sunday, June 09, 2013 8:59 PM
>> To: dev@cloudstack.apache.org
>> Cc: Edison Su; John Burwell
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>>
>>
>> Another thing we might want to consider for 4.2 is that the way the code is currently implemented in master, I don't think it's even possible to write a plug-in that connects storage directly to a VM because the CloudStack storage framework will call the "attach" method of the hypervisor in use and this method will fail because it assumes there is (talking Xen here) an SR existent and in this situation there isn't one.
>>
>>
>>
>> My thinking is that we should ignore the direct-attach-of-volume-to-VM use case for 4.2. I think it will require too much of a rewrite at this point in development (fewer than three weeks left until code freeze).
>>
>>
>>
>> If the plug-in implements an isManaged() method (as John suggested), the storage framework can invoke the necessary commands on the hypervisor in use.
>>
>>
>>
>> For example, the createAsync method is called. The storage plug-in creates a SAN volume and updates the CS DB. The storage framework then asks the driver if it's managed. If it is, the storage framework executes a command against the hypervisor in use to have it create, say, an SR (with a single VDI that takes up the entire SR). Now, when the storage framework executes the "attach" command against the hypervisor, this command will work, as well (because that logic assumes the prior existence of an SR and it will, in fact, be present).
>>
>>
>>
>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
>>
>> Hi Edison and John,
>>
>>
>>
>> I wanted to point out something I thought was relevant to this conversation.
>>
>>
>>
>> My plug-in supports both XenServer and ESX(i).
>>
>>
>>
>> These two hypervisors handle their respective hypervisor data structures (XenServer SR or ESX(i) datastore) quite differently.
>>
>>
>>
>> In XenServer, the SR that resides on the SAN LUN is simply "forgotten" when you issue a remove-SR command to XenServer.
>>
>>
>>
>> When you later want to access the contents of that data, it is still safely stored on the LUN and can be accessed from XenServer (from the same or different cluster (what XenServer calls a resource pool)) by creating a new SR that is based on the same IQN and LUN.
>>
>>
>>
>> XenServer SRs exist at the cluster level.
>>
>>
>>
>> In ESX(i), the datastore that resides on the SAN LUN is actually destroyed when you issue a remove-datastore command to ESX(i). In other words, the contents of the SAN LUN are destroyed.
>>
>>
>>
>> ESX(i) datastores exist at the datacenter level.
>>
>>
>>
>> The XenServer behavior (not destroying the contents of the LUN when the SR is removed) is desirable; the ESX(i) behavior (destroying the contents of the LUN when the datastore is removed) is not desirable.
>>
>>
>>
>> I can get around the ESX(i) behavior by removing the referenced IQN from each host in the cluster when the CS storage framework notifies my plug-in of a detach-volume event. When the CS volume is later attached again (either to the same or a different ESX(i) cluster), I can have each host in the relevant ESX(i) cluster reference the IQN and the datastore will be back again.
>>
>>
>>
>> This sequence of events is critical for my plug-in to support both environments:
>>
>> XenServer flow (events for Driver class):
>>
>> createAsync (called right before first attach) = creates SAN volume
>>
>> preAttachVolume = creates SR and creates VDI or just introduces SR
>>
>> (CS volume gets attached and later detached)
>>
>> postDetachVolume = removes SR (this does not delete data on SAN volume)
>>
>> deleteAsync (can be called when volume is not attached) = deletes the SAN volume
>>
>> VMware flow (events for Driver class):
>>
>> createAsync (called right before first attach) = creates SAN volume, creates datastore, and creates VMDK file
>>
>> preAttachVolume = adds iSCSI target to each host in cluster
>>
>> (CS volume gets attached and later detached)
>>
>> postDetachVolume = removes iSCSI target from each host in cluster
>>
>> deleteAsync (can be called when volume is not attached) = deletes datastore and deletes the SAN volume
>>
>> ********************
>>
>> So, the way I have it written up here, the driver is talking to the hypervisor. As John has pointed out, this may not be ideal.
>>
>>
>>
>> Whatever we decide on, it needs to be able to support this kind of flow. :)
>>
>>
>>
>> We could go John's route, where the driver says if it's managed or not.
>>
>>
>>
>> If it is, then the storage framework could send the necessary messages to the hypervisor in question at the right times (just migrate the hypervisor logic above from the driver to the right parts of the storage framework).
>>
>>
>>
>> This is fine, except that it does not resolve Edison's use case of desiring to have the storage LUN potentially connected directly to a VM.
>>
>>
>>
>> I'm not sure we should design around that use case, though. Is it common enough? Has anyone asked for it? Is it even desirable? How are usage statistics gathered for such volumes? It's nice to offer the flexibility, but is it a good idea in this case?
>>
>>
>>
>> Thanks!
>>
>>
>>
>>
>>
>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>
>>
>> > -----Original Message-----
>> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > Sent: Friday, June 07, 2013 4:08 PM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >
>> > Hi John,
>> >
>> > Yes, you are correct that Xen must layer an SR on top of the iSCSI volume
>> > to use it. Same story for VMware: ESX(i) must layer a datastore on top of
>> > the iSCSI volume to use it. I look at it like they are layering a clustered
>> > file system on the SAN volume so the hypervisors can share access to the
>> > contents of the volume.
>> >
>> > In 4.1, by the time the hypervisor-attach-volume logic is called, the SR or
>> > datastore has already been created (usually manually by an admin).
>> >
>> > This pre-setup of the, say, SR is, of course, not acceptable in an
>> > environment where each CloudStack volume that a user creates is mapped
>> > to a
>> > single SAN volume (via the SR).
>> >
>> > The question comes down to who should allocate the SR dynamically.
>> >
>> > We could have the storage framework ask the storage plug-in if it is
>> > managed. If it is, then the storage framework could send a message to the
>> > hypervisor in question to create (let's talk Xen here) the SR ahead of
>> > time. Then, when the storage framework next sends the attach-volume
>> > command
>> > to the hypervisor, it should work without changes to that attach-volume
>> > logic (because - from the point of view of the attach logic - the SR is
>> > already existent, as expected).
>> >
>> > Now, as Edison has pointed out, this limits the power of the storage
>> > plug-in. A storage plug-in in this model cannot directly attach storage to
>> > a VM (it must go through the hypervisor). Perhaps that is OK. We need to
>> > make a call on that.
>>
>>
>> Yes, I think we need to address this usage case, and won't limit the power of the storage plug-in.
>> John, what's your idea to fit this usage case into your abstraction?
>>
>>
>> >
>> > If we want to give more power to the storage vendor, we would have to
>> > have
>> > the storage framework call into the storage plug-in to send the appropriate
>> > attach commands to the hypervisor.
>> >
>> > Let's discuss and come to a consensus (or at least agree on a path) as soon
>> > as we can. :)
>> >
>> > Thanks!
>> >
>> >
>> >
>> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com> wrote:
>> >
>> > > Mike,
>> > >
>> > > My understanding is the Xen expects an SR structure on all ISCSI devices
>> > > -- at least that is how I read the code in your patch. Is my understanding
>> > > correct? If so, the Xen plugin should be able to query the storage device
>> > > to determine the presence of the SR structure and create it if it does not
>> > > exist. Am I missing something in the implementation that makes that type
>> > > of implementation impossible?
>> > >
>> > > Thanks,
>> > > -John
>> > >
>> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <mi...@solidfire.com>
>> > > wrote:
>> > >
>> > > > Yeah, if a storage vendor wanted to attach a volume directly to a VM in
>> > > 4.2
>> > > > today, it would probably fail because the attach-volume logic assumes
>> > the
>> > > > existence of the necessary hypervisor data structure (ex. SR on Xen).
>> > > >
>> > > > If we wanted to enable such an attach, we could do it the way Edison
>> > > > suggests.
>> > > >
>> > > >
>> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com> wrote:
>> > > >
>> > > >>
>> > > >>
>> > > >>> -----Original Message-----
>> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > > >>> Sent: Friday, June 07, 2013 2:37 PM
>> > > >>> To: dev@cloudstack.apache.org
>> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> > > >>>
>> > > >>> As we only have three weeks until feature freeze, we should come to
>> > a
>> > > >>> consensus on this design point as soon as possible.
>> > > >>>
>> > > >>> Right now, if the storage framework asks my driver it is is managed, it
>> > > >>> will say 'yes.' This means the framework will tell the driver to
>> > > perform
>> > > >>> its management activities. This then means the driver will call into
>> > > the
>> > > >>> host (it doesn't know which hypervisor, by the way) to perform the
>> > > >> activity
>> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
>> > > >>>
>> > > >>> The driver doesn't know which hypervisor it's talking to, it just
>> > > sends a
>> > > >>> message to the host to perform the necessary pre-attach work.
>> > > >>
>> > > >> Could we just expose a method like "attachVolume/dettachVolume" on
>> > the
>> > > >> PrimaryDataStoreDriver driver?
>> > > >> For most of cases, the implementation of each driver would just send
>> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
>> > the
>> > > >> implementation in a base class, so that can be shared by all of this
>> > > >> drivers), and in each hypervisor resource code, which may just call
>> > > >> hypervisor's api to attach the volume to VM, while for certain storage,
>> > > >> like SolidFire, may need to create a SR first, and create volume on it,
>> > > >> then call hypervisor's API to attach volume to VM.
>> > > >> While for some other storage vendor, may want to bypass hypervisor
>> > > during
>> > > >> attaching volume, so inside driver's attachvolume implementation, the
>> > > >> driver can do some magic, such as, directly talk to an agent inside VM
>> > > >> instance, then create a disk inside VM.
>> > > >>
>> > > >> How do you guys think?
>> > > >>
>> > > >>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
>> > > wrote:
>> > > >>>
>> > > >>>>
>> > > >>>>
>> > > >>>>> -----Original Message-----
>> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
>> > > >>>>> To: dev@cloudstack.apache.org
>> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> > > >>>>>
>> > > >>>>> Hi John,
>> > > >>>>>
>> > > >>>>> How's about this:
>> > > >>>>>
>> > > >>>>> The driver can implement an isManaged() method. The
>> > > >>> VolumeManagerImpl
>> > > >>>>> can
>> > > >>>>> call into the driver to see if its managed. If it is, the
>> > > >>>> VolumeManagerImpl
>> > > >>>>> (which is responsible for calling into the hypervisor to attach the
>> > > >> disk)
>> > > >>>>> can call into the hypervisor to create the necessary hypervisor data
>> > > >>>>> structure (ex. for XenServer, a storage repository).
>> > > >>>>
>> > > >>>> The problem here is that storage vendor may work differently with
>> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
>> > > >> other
>> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
>> > directly
>> > > >> to
>> > > >>>> VM instance, see the discuss(
>> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>> > > >>>
>> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>> > > >>> %3E
>> > > >>>> ).
>> > > >>>> So I would let storage provider to implement attach disk to VM,
>> > > >> instead of
>> > > >>>> implemented by cloudstack itself.
>> > > >>>>
>> > > >>>>
>> > > >>>>>
>> > > >>>>> If that's what you're going for, that works for me. By the way,
>> > > >> Edison's
>> > > >>>>> default storage plug-in (which handles the default storage behavior
>> > > >> in
>> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
>> > > >>>>> hypervisors. You might want to contact him and inform him of your
>> > > >>>> concerns
>> > > >>>>> or that logic (as is) will make it to production.
>> > > >>>>>
>> > > >>>>> Please let me know if what I wrote in above (for my solution) is OK
>> > > >> with
>> > > >>>>> you. :)
>> > > >>>>>
>> > > >>>>> Thanks!
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
>> > > >>> wrote:
>> > > >>>>>
>> > > >>>>>> Mike,
>> > > >>>>>>
>> > > >>>>>> Please see my responses in-line below.
>> > > >>>>>>
>> > > >>>>>> Thanks,
>> > > >>>>>> -John
>> > > >>>>>>
>> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>> > > >>>> mike.tutkowski@solidfire.com>
>> > > >>>>>> wrote:
>> > > >>>>>>
>> > > >>>>>>> Hey John,
>> > > >>>>>>>
>> > > >>>>>>> I still have a bit more testing I'd like to do before I build up
>> > > >> a
>> > > >>>> patch
>> > > >>>>>>> file, but this is the gist of what I've done:
>> > > >>>>>>>
>> > > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
>> > tells
>> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
>> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
>> > > >>>>> managed.
>> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
>> > > >> whatever
>> > > >>>>>>> activity is required. In my case, this includes sending a
>> > > >> message to
>> > > >>>> the
>> > > >>>>>>> host where the VM is running to have, say XenServer, add a
>> > > >> storage
>> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
>> > > >> SAN
>> > > >>>>>> volume,
>> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can
>> > > >> on
>> > > >>>> the
>> > > >>>>>>> storage repository). After this, the normal attach-volume
>> > > >> message is
>> > > >>>> sent
>> > > >>>>>>> to the host by VolumeManagerImpl.
>> > > >>>>>>
>> > > >>>>>> There should be **no** code from a storage driver to a hypervisor.
>> > > >> I
>> > > >>>>>> apologize for the repetition, but we simply can not have hypervisor
>> > > >>>>>> specific code in the storage layer. The circular dependencies
>> > > >> between
>> > > >>>> the
>> > > >>>>>> two layers are not sustainable in the long term. Either the
>> > > >>>> VirtualManager
>> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>> > > >>> coordinate
>> > > >>>>>> volume creation and then populating the SR. Ideally, we can
>> > > >>>> generalize the
>> > > >>>>>> process flow for attaching volumes such that the Xen hypervisor
>> > > >> plugin
>> > > >>>>>> would only implement callbacks to perform the attach action and
>> > > >> create
>> > > >>>>> the
>> > > >>>>>> structure and SR. To my mind, the SolidFire driver should only be
>> > > >>>>>> allocating space and providing information about contents (e.g.
>> > > >> space
>> > > >>>>>> available, space consumed, streams to a URI, file handle for a URI,
>> > > >>>> etc)
>> > > >>>>>> and capabilities.
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>>>
>> > > >>>>>>> * The reverse is performed for a detach-volume command.
>> > > >>>>>>>
>> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
>> > > >>>>>> Edison's
>> > > >>>>>>> default driver simply does a "return false;". We could add a new
>> > > >>>>>> parameter
>> > > >>>>>>> to the createStoragePool API command, if we want, to remove
>> > the
>> > > >>>>>> hard-coded
>> > > >>>>>>> return values in the drivers (although my driver will probably
>> > > >> just
>> > > >>>>>> ignore
>> > > >>>>>>> this parameter and always return true since it wouldn't make
>> > > >> sense
>> > > >>>> for it
>> > > >>>>>>> to ever return false). We'd need another column in the
>> > > >> storage_pool
>> > > >>>>> table
>> > > >>>>>>> to store this value.
>> > > >>>>>>
>> > > >>>>>> Yes, I think we should have a parameter added to the
>> > > >>> createStoragePool
>> > > >>>>>> surfaced to the HTTP API that allows DataStores to be configured
>> > > >> for
>> > > >>>>>> management when their underlying drivers support it. To simplify
>> > > >>>> things,
>> > > >>>>>> this flag should only be mutable when the DataStore is created. It
>> > > >>>> would be
>> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
>> > > >>>> creation.
>> > > >>>>>>
>> > > >>>>>>>
>> > > >>>>>>> Sound like I'm in sync with what you were thinking?
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>> > > >>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>> wrote:
>> > > >>>>>>>
>> > > >>>>>>>> I agree, John. Just wanted to point out that I have a working
>> > > >> GUI
>> > > >>>> for
>> > > >>>>>> you
>> > > >>>>>>>> to review (in that document), if you'd like to check it out.
>> > > >>>>>>>>
>> > > >>>>>>>> Thanks!
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>> > > >> jburwell@basho.com>
>> > > >>>>>> wrote:
>> > > >>>>>>>>
>> > > >>>>>>>>> Mike,
>> > > >>>>>>>>>
>> > > >>>>>>>>> I would like the UIs of two features reviewed together to
>> > > >> ensure
>> > > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
>> > > >> and
>> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
>> > > >>>> confusion,
>> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
>> > > >> help
>> > > >>>>>>>>> minimize any potential confusion.
>> > > >>>>>>>>>
>> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work itself
>> > > >> if
>> > > >>>> it
>> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
>> > > >>>> throttled
>> > > >>>>>>>>> IOPs or storage provisioned IOPs.
>> > > >>>>>>>>>
>> > > >>>>>>>>> Thanks,
>> > > >>>>>>>>> -John
>> > > >>>>>>>>>
>> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>> > > >>>>>>>>> <mi...@solidfire.com> wrote:
>> > > >>>>>>>>>
>> > > >>>>>>>>>> Hi John,
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
>> > to
>> > > >>> you
>> > > >>>>> the
>> > > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
>> > > >> IOPS.
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
>> > > >> and we
>> > > >>>>>>>>> decided
>> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
>> > Offering
>> > > >>>> dialog.
>> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
>> > should
>> > > >>> be
>> > > >>>>> able
>> > > >>>>>> to
>> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only use
>> > > >> Max
>> > > >>>>>> IOPS).
>> > > >>>>>>>>>> These fields are optional and can be ignored for storage that
>> > > >>>> does not
>> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
>> > > >>>> admin can
>> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
>> > > >> IOPS.
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
>> > > >> mine,
>> > > >>>> as
>> > > >>>>>> is
>> > > >>>>>>>>>> decided by the admin).
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> I'm not sure what we can do about these two features
>> > > >> expressing
>> > > >>>> the
>> > > >>>>>>>>> speed
>> > > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
>> > > >>>> QoS in
>> > > >>>>>>>>> any
>> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
>> > > >>>> IOPS).
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Thanks
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>> > > >>> <jb...@basho.com>
>> > > >>>>>>>>> wrote:
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
>> > > >>> Storage/Hypervisor
>> > > >>>>>> layer
>> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
>> > > >>>> hypervisor
>> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
>> > > >> following
>> > > >>>>>> issues
>> > > >>>>>>>>>>> outstanding:
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>> > > >>> read/write
>> > > >>>>>>>>> values
>> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
>> > > >> IOPS.
>> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
>> > > >>> storage
>> > > >>>>>> volumes
>> > > >>>>>>>>>>> with provisioned IOPS?
>> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
>> > > >>>> provisioned
>> > > >>>>>> IOPS
>> > > >>>>>>>>>>> that will permit providers to discriminate these guaranteed
>> > > >>>>>> operations
>> > > >>>>>>>>> in
>> > > >>>>>>>>>>> the event they want to bill for it differently?
>> > > >>>>>>>>>>> 4. What is the user experience for throttled and provisioned
>> > > >>> IOPS
>> > > >>>>>> that
>> > > >>>>>>>>>>> minimizes confusion of these concepts?
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
>> > > >>>> throttled
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
>> > > >>>> items 1
>> > > >>>>>>>>> and 2.
>> > > >>>>>>>>>>> Since the two facilities would not be permitted to operate
>> > > >>>> together,
>> > > >>>>>>>>> they
>> > > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
>> > > >> can be
>> > > >>>>>>>>> resolved
>> > > >>>>>>>>>>> with an additional flag or two on the usage records. As for
>> > > >>>> Item 4,
>> > > >>>>>> I
>> > > >>>>>>>>> am
>> > > >>>>>>>>>>> not familiar with how these two enhancements are
>> > depicted in
>> > > >>> the
>> > > >>>>> user
>> > > >>>>>>>>>>> interface. I think we need to review the UI enhancements
>> > for
>> > > >>>> both
>> > > >>>>>>>>>>> enhancements and ensure they are consistent.
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> Do these solutions make sense?
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>> -John
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>> > <us...@gmail.com>
>> > > >>>>> wrote:
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>> John and Mike,
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
>> > 2780/2729,
>> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>> > > >>> CLOUDSTACK-
>> > > >>>>> 2875 ) this
>> > > >>>>>>>>> week.
>> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
>> > > >> ask
>> > > >>>> for
>> > > >>>>>>>>> second
>> > > >>>>>>>>>>>> merge request after testing.
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>>> I believe I understand where you're going with this, John,
>> > > >> and
>> > > >>>>> have
>> > > >>>>>>>>> been
>> > > >>>>>>>>>>>>> re-working this section of code today.
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> Thanks for the comments,
>> > > >>>>>>>>>>>>> Mike
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>> > > >>>>> <jb...@basho.com>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> See my responses in-line below.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks for the response.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
>> > whether or
>> > > >>>>> not a
>> > > >>>>>>>>> Volume
>> > > >>>>>>>>>>> is
>> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>> > > >>> actual
>> > > >>>>>>>>> isManaged
>> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
>> > > >> resources
>> > > >>>> for
>> > > >>>>>> the
>> > > >>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>> if it is managed."
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
>> > method
>> > > >>>>> added to
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
>> > > >> then the
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
>> > > >>> method
>> > > >>>>> (which
>> > > >>>>>>>>>>> would
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
>> > and
>> > > >>> this
>> > > >>>>>> would
>> > > >>>>>>>>>>> call
>> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
>> > on
>> > > >>>>>>>>> XenServer,
>> > > >>>>>>>>>>> an
>> > > >>>>>>>>>>>>> SR?
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>> > > >>>>> DataStoreDriver.
>> > > >>>>>>>>> Since
>> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
>> > I
>> > > >>>> would
>> > > >>>>>> add
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
>> > straw
>> > > >>> man
>> > > >>>>> term
>> > > >>>>>>>>> --
>> > > >>>>>>>>>>>>>> something other than volume) methods to
>> > > >>>>>>>>>>> allocate/create/deallocate/delete
>> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
>> > > >>> property
>> > > >>>>> of
>> > > >>>>>>>>>>>>> DataStore
>> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
>> > > >>> DataStoreDriver
>> > > >>>>> can be
>> > > >>>>>>>>>>>>> managed.
>> > > >>>>>>>>>>>>>> This approach allows operators to override
>> > manageability
>> > > >> of
>> > > >>>>>> devices.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
>> > > >> plugin
>> > > >>>>>> would
>> > > >>>>>>>>> be
>> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
>> > to
>> > > >>>>> create any
>> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There should
>> > > >> be no
>> > > >>>>>>>>>>>>> dependencies
>> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
>> > > >> earlier,
>> > > >>>> such
>> > > >>>>>>>>>>>>> circular
>> > > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
>> > mess.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks again, John
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>> > > >>>>> <jb...@basho.com>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
>> > that
>> > > >>>>>> supports n
>> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
>> > > >> hypervisors.
>> > > >>>>>> Such
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
>> > > >>> beast.
>> > > >>>>>>>>>>>>> Therefore, my
>> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
>> > the
>> > > >>>>> Storage
>> > > >>>>>> layer
>> > > >>>>>>>>>>>>> towards
>> > > >>>>>>>>>>>>>>> this separation.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
>> > whether
>> > > >>> or
>> > > >>>>> not a
>> > > >>>>>>>>> Volume
>> > > >>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
>> > an
>> > > >>> actual
>> > > >>>>>>>>>>> isManaged
>> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
>> > > >> resources
>> > > >>>> for
>> > > >>>>>> the
>> > > >>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
>> > > >> to
>> > > >>>>>> indicate
>> > > >>>>>>>>>>>>> whether
>> > > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
>> > > >> think
>> > > >>>> we
>> > > >>>>>>>>> need to
>> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
>> > > >>>>>> something
>> > > >>>>>>>>>>>>> that is
>> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
>> > > >>> managing
>> > > >>>>> the
>> > > >>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
>> > unmanaged).
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
>> > in
>> > > >>> the
>> > > >>>>> Xen
>> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
>> > Storage
>> > > >>>>> layer to
>> > > >>>>>>>>>>> perform
>> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
>> > > >> file,
>> > > >>>> etc).
>> > > >>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>> > > >>>> hypervisor, as
>> > > >>>>>>>>> such,
>> > > >>>>>>>>>>>>>>> should be confined to its implementation. From my
>> > > >>>>> perspective,
>> > > >>>>>>>>>>> nothing
>> > > >>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
>> > From
>> > > >>> its
>> > > >>>>>>>>>>>>> perspective,
>> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
>> > to
>> > > >>>> query
>> > > >>>>>> the
>> > > >>>>>>>>>>> data
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
>> > level
>> > > >>>>> layers
>> > > >>>>>>>>> (e.g.
>> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
>> > composition
>> > > >>> of
>> > > >>>>>>>>>>> operations
>> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
>> > Volume
>> > > >>>>> storage
>> > > >>>>>>>>>>> protocol
>> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
>> > implemented
>> > > >> in
>> > > >>>> my
>> > > >>>>>>>>> patch,
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>> could do the following:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>> > > >>>>>>>>> VolumeManagerImpl
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
>> > > >>>>> hypervisor.
>> > > >>>>>> Before
>> > > >>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
>> > dealing
>> > > >>> with
>> > > >>>>>>>>> Dynamic
>> > > >>>>>>>>>>> (or
>> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
>> > > >>>> "Create
>> > > >>>>>>>>> SR"
>> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
>> > would
>> > > >>> then
>> > > >>>>>>>>> proceed
>> > > >>>>>>>>>>> to
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
>> > > >> add
>> > > >>>>>> another
>> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
>> > > >> particular
>> > > >>>>> type
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>> storage.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> What do you think?
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> Hey John,
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
>> > outlines
>> > > >>> how
>> > > >>>>> the
>> > > >>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
>> > > >> use
>> > > >>>> to
>> > > >>>>>>>>> you, as
>> > > >>>>>>>>>>>>> well,
>> > > >>>>>>>>>>>>>>>> as you perform the code review.
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> I have shared this document with you (you should
>> > have
>> > > >>>>> received
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> Talk to you later!
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>> > > >>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
>> > thoughts,
>> > > >>> but
>> > > >>>>>> want
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
>> > > >>>>>>>>>>>>> disk_io_throttle,
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
>> > > >>>>> coherent
>> > > >>>>>>>>> form
>> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
>> > the
>> > > >>>> notion
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
>> > > >> the
>> > > >>>>> notion
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
>> > was
>> > > >>>>>>>>> incomplete,
>> > > >>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>> I had
>> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
>> > > >> device.
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
>> > > >> this.
>> > > >>>> :)
>> > > >>>>> Do
>> > > >>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>> have a
>> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
>> > > >>> what
>> > > >>>>> you're
>> > > >>>>>>>>> looking
>> > > >>>>>>>>>>>>>>>>>> for?
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>> > > >>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
>> > the
>> > > >>>> notion
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
>> > > >>>>> capabilities
>> > > >>>>>>>>> (i.e.
>> > > >>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
>> > > >>> managed
>> > > >>>>> by
>> > > >>>>>>>>> CloudStack)
>> > > >>>>>>>>>>>>>>>>>> from the
>> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
>> > a
>> > > >>>>> semantic
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
>> > I
>> > > >>> also
>> > > >>>>> want
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
>> > the
>> > > >>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> definitions.
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>> > > >>>>>>>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
>> > questions
>> > > >>> for
>> > > >>>>> me
>> > > >>>>>> with
>> > > >>>>>>>>>>>>>>>>>> regards to
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
>> > and
>> > > >>> SAN
>> > > >>>>>> volumes?
>> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
>> > (LUN
>> > > >>> 0).
>> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
>> > and
>> > > >>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> Volumes?
>> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
>> > > >>> server or
>> > > >>>>>>>>> externally
>> > > >>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> operator?
>> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
>> > > >>> volume
>> > > >>>>> (same
>> > > >>>>>> as a
>> > > >>>>>>>>>>> SAN
>> > > >>>>>>>>>>>>>>>>>> LUN) is
>> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
>> > plug-in)
>> > > >>>>> the
>> > > >>>>>> first
>> > > >>>>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
>> > but
>> > > >>>> still
>> > > >>>>>>>>> want
>> > > >>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> use a
>> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
>> > > >>> today
>> > > >>>>>> (prior
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> 4.2). The
>> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
>> > in
>> > > >>> this
>> > > >>>>> case -
>> > > >>>>>>>>>>>>>>>>>> multiple VMs
>> > > >>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
>> > you
>> > > >>> can do
>> > > >>>>> this
>> > > >>>>>>>>>>>>> today,
>> > > >>>>>>>>>>>>>>>>>> it is
>> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
>> > server
>> > > >>> or
>> > > >>>>> externally
>> > > >>>>>>>>> by
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> operator?
>> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
>> > > >>> volumes
>> > > >>>>> are
>> > > >>>>>>>>>>>>> completely
>> > > >>>>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
>> > There
>> > > >>> is
>> > > >>>>> no
>> > > >>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>> interaction.
>> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
>> > > >>> volume
>> > > >>>>> and a
>> > > >>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
>> > > >>>>> supports
>> > > >>>>>> true
>> > > >>>>>>>>>>> QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
>> > > >> related
>> > > >>>>> and
>> > > >>>>>>>>>>>>> expected
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> operate.
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
>> > multiple
>> > > >>> VMs
>> > > >>>>> with
>> > > >>>>>>>>>>>>>>>>>> provisioned
>> > > >>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
>> > > >>> single
>> > > >>>>> SAN
>> > > >>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>> (LUN) to a
>> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
>> > > >>> multiple
>> > > >>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
>> > guarantee
>> > > >>> QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
>> > volume
>> > > >>> house
>> > > >>>>> more
>> > > >>>>>>>>> than
>> > > >>>>>>>>>>>>> one
>> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
>> > > >>> without
>> > > >>>>> any of
>> > > >>>>>> my
>> > > >>>>>>>>>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>>>> code.
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
>> > Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
>> > volume
>> > > >>> for
>> > > >>>>>>>>>>> CloudStack's
>> > > >>>>>>>>>>>>>>>>>> use
>> > > >>>>>>>>>>>>>>>>>>>> onto
>> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
>> > Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
>> > concerns me
>> > > >>> --
>> > > >>>>>> concepts
>> > > >>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
>> > Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
>> > today
>> > > >>>> only
>> > > >>>>>> used
>> > > >>>>>>>>>>> (as
>> > > >>>>>>>>>>>>>>>>>> far as
>> > > >>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
>> > not
>> > > >>>>> PreSetup an
>> > > >>>>>>>>> SR).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
>> > from
>> > > >>>>> VMware, it
>> > > >>>>>>>>> uses
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> vmfs
>> > > >>>>>>>>>>>>>>>>>>>>>> type.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
>> > from
>> > > >>> KVM,
>> > > >>>>> it uses
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
>> > > >>> here
>> > > >>>>> was to
>> > > >>>>>>>>> make a
>> > > >>>>>>>>>>>>>>>>>> new type
>> > > >>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
>> > > >>> Edison
>> > > >>>>> added
>> > > >>>>>>>>> into
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
>> > > >>> specificy,
>> > > >>>>> though:
>> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
>> > > >>> Tutkowski
>> > > >>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
>> > know
>> > > >>> or
>> > > >>>>> care
>> > > >>>>>> that
>> > > >>>>>>>>> it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>> used
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
>> > > >>>>> questions
>> > > >>>>>>>>> about
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
>> > > >>>>> SolidFire
>> > > >>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>> needs to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
>> > > >>> hypervisor
>> > > >>>>> logic
>> > > >>>>>>>>>>> knows
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>> treat
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
>> > constraint
>> > > >>>>> and just
>> > > >>>>>>>>>>>>>>>>>> calling it
>> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
>> > for
>> > > >>> the
>> > > >>>>> hypervisor
>> > > >>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
>> > perform
>> > > >>> the
>> > > >>>>> necessary
>> > > >>>>>>>>>>>>>>>>>> activities.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
>> > > >>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>> > > >>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
>> > able to
>> > > >>>>> manage a
>> > > >>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>> > > >>> CloudStack
>> > > >>>>> (e.g. the
>> > > >>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>> shared
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
>> > > >>> chosen
>> > > >>>>> to
>> > > >>>>>>>>> dedicate
>> > > >>>>>>>>>>>>>>>>>> only a
>> > > >>>>>>>>>>>>>>>>>>>>>> portion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
>> > > >>> make
>> > > >>>>> sense?"
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
>> > SAN
>> > > >>>>> volume via
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
>> > would
>> > > >>>>> work unless
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
>> > an
>> > > >>> SR.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
>> > > >>>>> CloudStack, an
>> > > >>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>> allocates
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
>> > > >>> exposes it
>> > > >>>>>> through
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> LUN. They
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
>> > to
>> > > >>>>> CloudStack,
>> > > >>>>>>>>> but
>> > > >>>>>>>>>>>>>>>>>> not allow
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
>> > > >>>>> changes, we
>> > > >>>>>>>>> would
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>> passing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
>> > > >>>>> necessary
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>> (like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
>> > have
>> > > >>> the
>> > > >>>>> storage
>> > > >>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>> itself
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
>> > support
>> > > >>> from
>> > > >>>>> the
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> side and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
>> > to
>> > > >>> create
>> > > >>>>> the
>> > > >>>>>>>>> SR
>> > > >>>>>>>>>>>>>>>>>> before the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
>> > plugin
>> > > >>>>> should
>> > > >>>>>> have a
>> > > >>>>>>>>>>>>> rich
>> > > >>>>>>>>>>>>>>>>>> enough
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
>> > > >>> available
>> > > >>>>> for
>> > > >>>>>> volume
>> > > >>>>>>>>>>>>>>>>>> storage.
>> > > >>>>>>>>>>>>>>>>>>>> For
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>> > > >>> interrogation of
>> > > >>>>> the
>> > > >>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
>> > itself
>> > > >>>>> shouldn't
>> > > >>>>>>>>> know
>> > > >>>>>>>>>>>>>>>>>> or care
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
>> > able
>> > > >>> to
>> > > >>>>>> answer
>> > > >>>>>>>>>>>>>>>>>> questions
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
>> > > >>> driver
>> > > >>>>> (which
>> > > >>>>>>>>> is a
>> > > >>>>>>>>>>>>>>>>>> type of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
>> > done, a
>> > > >>> new
>> > > >>>>> Primary
>> > > >>>>>>>>>>>>>>>>>> Storage shows
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
>> > > >>> that
>> > > >>>>>> references
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>> tag
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
>> > CloudStack
>> > > >>>>> volume. This
>> > > >>>>>>>>> leads
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> a new
>> > > >>>>>>>>>>>>>>>>>>>>>> row
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
>> > CloudStack
>> > > >>>>> volume to a
>> > > >>>>>>>>> VM
>> > > >>>>>>>>>>>>>>>>>> (attach
>> > > >>>>>>>>>>>>>>>>>>>>>> disk).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
>> > calling
>> > > >>> the
>> > > >>>>> plug-in
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> create a
>> > > >>>>>>>>>>>>>>>>>>>> new
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
>> > > >>> SAN).
>> > > >>>>> The
>> > > >>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>> also
>> > > >>>>>>>>>>>>>>>>>>>>>> updates
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
>> > data
>> > > >>>>> (like the
>> > > >>>>>>>>> IQN of
>> > > >>>>>>>>>>>>>>>>>> the SAN
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
>> > the
>> > > >>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> volume is in
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
>> > the
>> > > >>>>> volume
>> > > >>>>>>>>> will be
>> > > >>>>>>>>>>>>>>>>>> in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
>> > the
>> > > >>>>> plug-in
>> > > >>>>>>>>> code
>> > > >>>>>>>>>>>>>>>>>> will not
>> > > >>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
>> > and
>> > > >>>>> detects
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
>> > form
>> > > >>> of a
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
>> > > >>>>> terminology,
>> > > >>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>> in these
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
>> > > >>>>> allocation on
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
>> > > >>> mean
>> > > >>>>> 'SAN
>> > > >>>>>>>>> volume'.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
>> > user
>> > > >>> can
>> > > >>>>> make in
>> > > >>>>>>>>>>>>>>>>>> CloudStack, I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
>> > well
>> > > >>> make
>> > > >>>>> sense, but
>> > > >>>>>>>>> I'm
>> > > >>>>>>>>>>>>>>>>>> having a
>> > > >>>>>>>>>>>>>>>>>>>>>> hard
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
>> > in on
>> > > >>>>> this
>> > > >>>>>>>>> conversation
>> > > >>>>>>>>>>>>>>>>>> as he
>> > > >>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>> > > >>> approach I
>> > > >>>>> took.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
>> > > >>> John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
>> > two
>> > > >>>>> distinct
>> > > >>>>>>>>> concepts
>> > > >>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
>> > protocols.
>> > > >>> In
>> > > >>>>> both
>> > > >>>>>>>>> cases, we
>> > > >>>>>>>>>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
>> > system to
>> > > >>>>>> create/delete
>> > > >>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
>> > the
>> > > >>>>> volume to
>> > > >>>>>> be
>> > > >>>>>>>>>>>>>>>>>> volume to be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
>> > context.
>> > > >>> To
>> > > >>>>> ensure
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>> are in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
>> > > >>>>> definitions,
>> > > >>>>>>>>> refers to
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> physical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
>> > correct?
>> > > >>>>> Minimally,
>> > > >>>>>> we
>> > > >>>>>>>>>>>>> must
>> > > >>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>> able
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
>> > device
>> > > >>> to
>> > > >>>>> move bits
>> > > >>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>> one place
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>> > > >>> Optionally, a
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
>> > Therefore,
>> > > >>>>> we can have
>> > > >>>>>> a
>> > > >>>>>>>>>>>>>>>>>> unmanaged
>> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
>> > and
>> > > >>> we
>> > > >>>>> can have a
>> > > >>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
>> > > >>> capable
>> > > >>>>> of
>> > > >>>>>>>>> allocating
>> > > >>>>>>>>>>>>>>>>>> LUNs and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
>> > CloudStack
>> > > >>> may
>> > > >>>>> be able
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> manage a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
>> > > >>>>> unmanaged by
>> > > >>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> (e.g.
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
>> > the
>> > > >>>>> operator has
>> > > >>>>>>>>> chosen
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
>> > Does
>> > > >>> my
>> > > >>>>>> reasoning
>> > > >>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>> sense?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
>> > are
>> > > >>>>> reasonable, it
>> > > >>>>>>>>> seems
>> > > >>>>>>>>>>>>>>>>>>>> appropriate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
>> > concerns
>> > > >>> from
>> > > >>>>>>>>> StoragePoolType,
>> > > >>>>>>>>>>>>>>>>>> add the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
>> > attached
>> > > >>> driver
>> > > >>>>> that
>> > > >>>>>>>>>>> indicates
>> > > >>>>>>>>>>>>>>>>>> whether
>> > > >>>>>>>>>>>>>>>>>>>>>> or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
>> > > >>> establish
>> > > >>>>> a
>> > > >>>>>>>>> abstraction
>> > > >>>>>>>>>>>>>>>>>>>>>> representing a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
>> > that
>> > > >>> is
>> > > >>>>>>>>> associated
>> > > >>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>> volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
>> > hypervisor
>> > > >>>>> drivers can
>> > > >>>>>>>>>>>>> declare
>> > > >>>>>>>>>>>>>>>>>> which
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
>> > encounter
>> > > >>> a
>> > > >>>>> device
>> > > >>>>>> managed
>> > > >>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
>> > exposed
>> > > >>> by
>> > > >>>>> the driver to
>> > > >>>>>>>>>>>>> automate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
>> > > >>> sense,
>> > > >>>>> then we can
>> > > >>>>>>>>> sit
>> > > >>>>>>>>>>>>>>>>>> down and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
>> > between
>> > > >>> the
>> > > >>>>> current
>> > > >>>>>> iSCSI
>> > > >>>>>>>>>>>>> type
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
>> > go in
>> > > >>> and
>> > > >>>>> create
>> > > >>>>>> a
>> > > >>>>>>>>>>>>>>>>>> Primary
>> > > >>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
>> > time,
>> > > >>>>> the
>> > > >>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>> volume must
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
>> > allocated).
>> > > >>>>> Future
>> > > >>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
>> > created
>> > > >>>>> behind the
>> > > >>>>>>>>>>>>> scenes.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
>> > to
>> > > >>> go in
>> > > >>>>> and
>> > > >>>>>> create
>> > > >>>>>>>>>>>>>>>>>> Primary
>> > > >>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
>> > delete
>> > > >>>>> volumes on
>> > > >>>>>>>>> its
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
>> > the
>> > > >>>>> storage
>> > > >>>>>>>>>>>>>>>>>> framework). When
>> > > >>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
>> > that
>> > > >>> was
>> > > >>>>> created,
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>>>> tells
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
>> > volume.
>> > > >>> After
>> > > >>>>> this is
>> > > >>>>>>>>> done,
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
>> > called.
>> > > >>>>> No
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
>> > volume
>> > > >>> was
>> > > >>>>> just
>> > > >>>>>>>>> created.
>> > > >>>>>>>>>>>>> The
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
>> > PM,
>> > > >>> Mike
>> > > >>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
>> > should
>> > > >>>>> probably
>> > > >>>>>> have
>> > > >>>>>>>>>>>>>>>>>> defined them
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
>> > > >>> pre-
>> > > >>>>> allocated
>> > > >>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>> an admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
>> > > >>> hypervisor
>> > > >>>>> data
>> > > >>>>>>>>> structure
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
>> > SR),
>> > > >>> then
>> > > >>>>> that
>> > > >>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
>> > Disks
>> > > >>> (VDI)
>> > > >>>>> are later
>> > > >>>>>>>>>>>>> placed
>> > > >>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
>> > > >>> these
>> > > >>>>> cases, the
>> > > >>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
>> > > >>> already
>> > > >>>>> in place
>> > > >>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> simply
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
>> > structure
>> > > >>> to
>> > > >>>>> the VM in
>> > > >>>>>>>>>>>>>>>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
>> > that is
>> > > >>> not
>> > > >>>>>>>>>>> pre-allocated.
>> > > >>>>>>>>>>>>>>>>>> Instead
>> > > >>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
>> > could
>> > > >>> be a
>> > > >>>>> SAN
>> > > >>>>>> (not
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> volume on
>> > > >>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
>> > data
>> > > >>>>> structure
>> > > >>>>>>>>> must be
>> > > >>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
>> > because
>> > > >>> these
>> > > >>>>> types of
>> > > >>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>> have not
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
>> > > >>>>> structure by an
>> > > >>>>>>>>>>> admin.
>> > > >>>>>>>>>>>>>>>>>> Once
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
>> > XenServer
>> > > >>> for
>> > > >>>>> this
>> > > >>>>>>>>> volume,
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> attaches
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
>> > VM
>> > > >>> in
>> > > >>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
>> > PM,
>> > > >>>>> John Burwell
>> > > >>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>> > implementation of
>> > > >>> the
>> > > >>>>> Dynamic
>> > > >>>>>> type
>> > > >>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>> behavior
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
>> > why I
>> > > >>> ask
>> > > >>>>> about the
>> > > >>>>>>>>>>>>>>>>>> difference.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
>> > the
>> > > >>>>> definition
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> Dynamic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
>> > > >>> Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
>> > type
>> > > >>> is
>> > > >>>>>> uniquely
>> > > >>>>>>>>>>> used
>> > > >>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
>> > > >>> Storage
>> > > >>>>> that is
>> > > >>>>>>>>> directly
>> > > >>>>>>>>>>>>>>>>>> based on
>> > > >>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
>> > > >>> step of
>> > > >>>>> going
>> > > >>>>>> to
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
>> > on
>> > > >>> that
>> > > >>>>> iSCSI
>> > > >>>>>>>>> target as
>> > > >>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
>> > > >>>>> supported for
>> > > >>>>>>>>>>>>>>>>>> XenServer.
>> > > >>>>>>>>>>>>>>>>>>>> For
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
>> > first go
>> > > >>> to
>> > > >>>>> the
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>> perform
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
>> > RBD is.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> > 2:13
>> > > >>> PM,
>> > > >>>>> John
>> > > >>>>>> Burwell
>> > > >>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
>> > code,
>> > > >>> what
>> > > >>>>> is the
>> > > >>>>>>>>>>> difference
>> > > >>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
>> > isn't
>> > > >>> RBD
>> > > >>>>> considered
>> > > >>>>>>>>>>>>> Dynamic?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
>> > PM,
>> > > >>> Mike
>> > > >>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
>> > storage is
>> > > >>>>> defined in
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>> > > >>>>> StoragePoolType {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
>> > local
>> > > >>>>> directory
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > NetworkFilesystem(true),
>> > > >>> //
>> > > >>>>> NFS or CIFS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
>> > shared
>> > > >>> LUN,
>> > > >>>>> with a
>> > > >>>>>>>>>>> clusterfs
>> > > >>>>>>>>>>>>>>>>>>>> overlay
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
>> > ZFS
>> > > >>>>> Comstar
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
>> > image
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
>> > XenServer
>> > > >>> local
>> > > >>>>> LVM SR
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > SharedMountPoint(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
>> > VMware
>> > > >>> VMFS
>> > > >>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
>> > > >>>>> XenServer, Storage
>> > > >>>>>>>>> Pool
>> > > >>>>>>>>>>>>>>>>>> is set
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
>> > XenServer
>> > > >>> local
>> > > >>>>> EXT SR
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
>> > dynamic,
>> > > >>>>> zone-wide
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > StoragePoolType(boolean
>> > > >>>>> shared) {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
>> > isShared() {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> > 1:41
>> > > >>> PM,
>> > > >>>>> Mike
>> > > >>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
>> > > >>> another
>> > > >>>>> storage
>> > > >>>>>>>>>>> company
>> > > >>>>>>>>>>>>>>>>>> wants
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
>> > > >>> Quality
>> > > >>>>> of
>> > > >>>>>> Service
>> > > >>>>>>>>>>>>>>>>>> feature. It
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
>> > well.
>> > > >>>>> They would
>> > > >>>>>>>>> need
>> > > >>>>>>>>>>>>> only
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
>> > the
>> > > >>>>> necessary
>> > > >>>>>>>>> changes to
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
>> > in.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> > at
>> > > >>> 1:39
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
>> > the
>> > > >>>>> XenServer code
>> > > >>>>>> is
>> > > >>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
>> > attach
>> > > >>> logic is
>> > > >>>>> now aware
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>> dynamic,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
>> > > >>>>> implementation of
>> > > >>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>> kind of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
>> > > >>> with
>> > > >>>>> Edison's
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
>> > new
>> > > >>>>> framework that
>> > > >>>>>>>>>>> supported
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
>> > dynamically.
>> > > >>>>> However, when I
>> > > >>>>>>>>>>>>>>>>>> visited with
>> > > >>>>>>>>>>>>>>>>>>>>>> him
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
>> > that
>> > > >>> it
>> > > >>>>> was not
>> > > >>>>>>>>>>>>>>>>>> complete. We
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
>> > could
>> > > >>> do
>> > > >>>>> with these
>> > > >>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>> unless
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
>> > recognize
>> > > >>> this
>> > > >>>>> new type of
>> > > >>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>> create
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>> > structure.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> > at
>> > > >>> 1:28
>> > > >>>>> PM, John
>> > > >>>>>>>>> Burwell
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
>> > to me
>> > > >>>>> that any
>> > > >>>>>>>>>>> operation
>> > > >>>>>>>>>>>>>>>>>> in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
>> > > >>> about
>> > > >>>>> details. I
>> > > >>>>>>>>> expect
>> > > >>>>>>>>>>>>>>>>>> to see
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
>> > > >>> operations
>> > > >>>>> that can be
>> > > >>>>>>>>>>>>>>>>>>>> composed/driven
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
>> > > >>>>> allocate
>> > > >>>>>> space/create
>> > > >>>>>>>>>>>>>>>>>> structures
>> > > >>>>>>>>>>>>>>>>>>>>>> per
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
>> > > >>>>> dependency, we are
>> > > >>>>>>>>> going
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> end
>> > > >>>>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
>> > make
>> > > >>> the
>> > > >>>>> system
>> > > >>>>>>>>>>>>> increasingly
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
>> > > >>>>> understanding that the
>> > > >>>>>>>>> Xen
>> > > >>>>>>>>>>>>>>>>>> specific
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>> > > >>>>> CitrixResourceBase
>> > > >>>>>> class?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013 at
>> > > >>> 3:12
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
>> > in a
>> > > >>> bit
>> > > >>>>> more
>> > > >>>>>>>>> detail:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
>> > aside
>> > > >>> from
>> > > >>>>> one setup
>> > > >>>>>>>>>>>>> method
>> > > >>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
>> > > >>> volume on
>> > > >>>>> the
>> > > >>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> system,
>> > > >>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>> go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
>> > > >>> structure
>> > > >>>>> to make
>> > > >>>>>> use
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>> > > >>> XenServer
>> > > >>>>> or a
>> > > >>>>>> datastore
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> ESX(i)).
>> > > >>>>>>>>>>>>>>>>>>>>>> VMs
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
>> > storage
>> > > >>>>> system's volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
>> > > >>> storage
>> > > >>>>> framework,
>> > > >>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> need
>> > > >>>>>>>>>>>>>>>>>>>> no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
>> > > >>> create
>> > > >>>>> a 1:1
>> > > >>>>>>>>>>>>> relationship
>> > > >>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
>> > VM's
>> > > >>> data
>> > > >>>>> disk
>> > > >>>>>>>>> (necessary
>> > > >>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
>> > a
>> > > >>> plug-
>> > > >>>>> in that is
>> > > >>>>>>>>> called
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
>> > needed.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
>> > the
>> > > >>>>> storage
>> > > >>>>>> framework
>> > > >>>>>>>>> did
>> > > >>>>>>>>>>>>>>>>>> not
>> > > >>>>>>>>>>>>>>>>>>>>>> address
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>> > > >>> hypervisor-
>> > > >>>>> specific data
>> > > >>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>> when
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
>> > case,
>> > > >>> I've
>> > > >>>>> been
>> > > >>>>>>>>> enhancing
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> to do
>> > > >>>>>>>>>>>>>>>>>>>> so.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
>> > > >>>>> submitted. I've got
>> > > >>>>>>>>> ESX(i)
>> > > >>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> my
>> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
>> > 4.2
>> > > >>>>> freeze date.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
>> > bit? :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013 at
>> > > >>>>> 1:03 PM, Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
>> > plug-in -
>> > > >>> by
>> > > >>>>> itself - is
>> > > >>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
>> > the
>> > > >>>>> volume-attach
>> > > >>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>> (in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
>> > into
>> > > >>> the
>> > > >>>>> plug-in to
>> > > >>>>>>>>> have
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> create a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
>> > > >>> comes to
>> > > >>>>> attach the
>> > > >>>>>>>>> volume
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
>> > > >>> enough
>> > > >>>>> to recognize
>> > > >>>>>>>>> it's
>> > > >>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
>> > volume
>> > > >>> has
>> > > >>>>> just been
>> > > >>>>>>>>> created)
>> > > >>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> create,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
>> > > >>>>> XenServer) or a
>> > > >>>>>>>>> datastore
>> > > >>>>>>>>>>>>>>>>>> (for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
>> > just
>> > > >>>>> created.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
>> > spending
>> > > >>> most
>> > > >>>>> of my time
>> > > >>>>>>>>>>>>> recently
>> > > >>>>>>>>>>>>>>>>>> making
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
>> > code.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
>> > up?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013
>> > > >>> at
>> > > >>>>> 12:48 PM,
>> > > >>>>>> John
>> > > >>>>>>>>>>>>>>>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
>> > > >>> why
>> > > >>>>> the the storage
>> > > >>>>>>>>>>>>> driver
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
>> > at
>> > > >>> 1:21
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
>> > ultimately I
>> > > >>>>> would like to
>> > > >>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>> all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
>> > think
>> > > >>> I'm
>> > > >>>>> just out of
>> > > >>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>> for 4.2
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
>> > > >>> plug-in
>> > > >>>>> supports
>> > > >>>>>>>>>>>>> XenServer.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
>> > > >>> feature
>> > > >>>>> freeze, I
>> > > >>>>>>>>> have
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> working
>> > > >>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
>> > be
>> > > >>> clear,
>> > > >>>>> this is
>> > > >>>>>> all
>> > > >>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> regards
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > Compute
>> > > >>>>> Offerings post
>> > > >>>>>> 4.2.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > > >>> 2013
>> > > >>>>> at 11:14 AM,
>> > > >>>>>>>>> Kelcey
>> > > >>>>>>>>>>>>>>>>>> Jamison
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
>> > plan
>> > > >>> on
>> > > >>>>> supporting
>> > > >>>>>>>>> KVM in
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
>> > > >>>>> Message -----
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
>> > > >>>>> Tutkowski" <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>> > > >>>>> dev@cloudstack.apache.org
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
>> > Monday,
>> > > >>> June
>> > > >>>>> 3, 2013
>> > > >>>>>>>>> 10:12:32 AM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
>> > > >>> [MERGE]
>> > > >>>>>>>>>>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>> MASTER
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
>> > > >>> merging
>> > > >>>>> Wei's feature
>> > > >>>>>>>>>>> first,
>> > > >>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>> mine.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
>> > is
>> > > >>> for
>> > > >>>>> KVM only,
>> > > >>>>>>>>> then
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> is a
>> > > >>>>>>>>>>>>>>>>>>>> non
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
>> > 3,
>> > > >>> 2013
>> > > >>>>> at 8:55 AM,
>> > > >>>>>> Wei
>> > > >>>>>>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
>> > billing,
>> > > >>> as
>> > > >>>>> no one works
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> billing
>> > > >>>>>>>>>>>>>>>>>>>> now,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
>> > by
>> > > >>>>> themselves. They
>> > > >>>>>>>>> can
>> > > >>>>>>>>>>>>> get
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > disk_offering of
>> > > >>> a
>> > > >>>>> VMs and
>> > > >>>>>> volumes
>> > > >>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
>> > the
>> > > >>>>> exact
>> > > >>>>>> limitation
>> > > >>>>>>>>>>>>> value
>> > > >>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>> > limitation
>> > > >>> for
>> > > >>>>> nics as
>> > > >>>>>> well.
>> > > >>>>>>>>> I
>> > > >>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
>> > I/O
>> > > >>>>> throttling?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
>> > > >>>>> misunstand the second
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
>> > > >>> what
>> > > >>>>> you said about
>> > > >>>>>> the
>> > > >>>>>>>>>>> two
>> > > >>>>>>>>>>>>>>>>>>>> features.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>> > John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>> > 2013,
>> > > >>> at
>> > > >>>>> 2:13 AM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > Mike
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
>> > Mike's
>> > > >>>>> aswer helps you.
>> > > >>>>>>>>> I am
>> > > >>>>>>>>>>>>>>>>>> trying
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
>> > think
>> > > >>> billing
>> > > >>>>> should
>> > > >>>>>>>>> depend
>> > > >>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> IO
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > limitation.
>> > > >>> Please
>> > > >>>>> review
>> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
>> > > >>>>>>>>>>>>>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
>> > IO
>> > > >>>>> statistics
>> > > >>>>>> including
>> > > >>>>>>>>>>>>>>>>>> bytes/iops
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
>> > > >>> machine.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
>> > the
>> > > >>> AWS
>> > > >>>>> model,
>> > > >>>>>> customers
>> > > >>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>> billed
>> > > >>>>>>>>>>>>>>>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > provisioned
>> > > >>> IOPS,
>> > > >>>>> as well as,
>> > > >>>>>> for
>> > > >>>>>>>>>>>>> those
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://aws.amazon.com/ebs/).
>> > > >>>>>> I
>> > > >>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>> imagine
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
>> > > >>> employ
>> > > >>>>> similar cost
>> > > >>>>>>>>>>> models.
>> > > >>>>>>>>>>>>>>>>>> Could
>> > > >>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
>> > model
>> > > >>> in
>> > > >>>>> the current
>> > > >>>>>>>>> patch?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
>> > you
>> > > >>> mean
>> > > >>>>> IOPS runtime
>> > > >>>>>>>>>>> change?
>> > > >>>>>>>>>>>>>>>>>> KVM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > limitation for
>> > > >>> a
>> > > >>>>> running
>> > > >>>>>> virtual
>> > > >>>>>>>>>>>>>>>>>> machine
>> > > >>>>>>>>>>>>>>>>>>>>>> through
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > CloudStack
>> > > >>> does
>> > > >>>>> not support
>> > > >>>>>>>>>>> changing
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > (computer
>> > > >>>>> offering or disk
>> > > >>>>>>>>>>>>> offering).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
>> > the
>> > > >>>>> Java interface
>> > > >>>>>>>>> level.
>> > > >>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>> apologize
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
>> > > >>>>> generalize allocation
>> > > >>>>>>>>>>>>>>>>>> algorithms
>> > > >>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
>> > the
>> > > >>>>> service guarantees
>> > > >>>>>>>>>>>>>>>>>> provided by a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
>> > > >>> good
>> > > >>>>> question.
>> > > >>>>>>>>> Maybe it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>> better
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> disk_io_throttling as Mike
>> > > >>>>>>>>> needs to
>> > > >>>>>>>>>>>>>>>>>> consider
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
>> > think.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
>> > expand
>> > > >>> on
>> > > >>>>> my thoughts
>> > > >>>>>> in a
>> > > >>>>>>>>>>>>> later
>> > > >>>>>>>>>>>>>>>>>>>>>> response
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
>> > points
>> > > >>>>> between these two
>> > > >>>>>>>>>>>>>>>>>> features. I
>> > > >>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
>> > to be
>> > > >>>>> merged before
>> > > >>>>>>>>>>>>>>>>>> SolidFire, but
>> > > >>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > coordination
>> > > >>>>> between the
>> > > >>>>>> branches
>> > > >>>>>>>>>>>>>>>>>> (possibly
>> > > >>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> disk_io_throttling)
>> > > >>>>> to
>> > > >>>>>>>>> coordinate on
>> > > >>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>> issue.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>> > > >>> John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > things I
>> > > >>>>> want to
>> > > >>>>>> understand
>> > > >>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
>> > there
>> > > >>>>> value in
>> > > >>>>>> capturing
>> > > >>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>> policies
>> > > >>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > model
>> > > >>> (e.g.
>> > > >>>>> for
>> > > >>>>>>>>> billing/usage
>> > > >>>>>>>>>>>>>>>>>>>> purposes,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
>> > Should
>> > > >>> there
>> > > >>>>> be a common
>> > > >>>>>>>>>>>>>>>>>> interface model
>> > > >>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > provisioning
>> > > >>> at
>> > > >>>>> runtime?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
>> > are
>> > > >>>>> conflicting
>> > > >>>>>>>>> provisioned
>> > > >>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > hypervisor
>> > > >>> and
>> > > >>>>> storage device
>> > > >>>>>>>>>>>>>>>>>> reconciled?
>> > > >>>>>>>>>>>>>>>>>>>> In
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
>> > to
>> > > >>>>> believe (and
>> > > >>>>>> billed)
>> > > >>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>> device
>> > > >>>>> has been
>> > > >>>>>>>>> configured
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > consistent
>> > > >>>>> configuration
>> > > >>>>>>>>> between a
>> > > >>>>>>>>>>>>>>>>>> VM and a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
>> > but a
>> > > >>>>> later
>> > > >>>>>> modification
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> inconsistency.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
>> > 2,
>> > > >>> 2013,
>> > > >>>>> at 8:38 PM,
>> > > >>>>>>>>> Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > believe
>> > > >>> Wei's
>> > > >>>>> feature deals
>> > > >>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>> controlling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>> hypervisor
>> > > >>>>> side.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> > feature
>> > > >>> is
>> > > >>>>> focused on
>> > > >>>>>>>>>>>>> controlling
>> > > >>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
>> > that
>> > > >>>>> helps. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
>> > Jun
>> > > >>> 2,
>> > > >>>>> 2013 at 6:35
>> > > >>>>>> PM,
>> > > >>>>>>>>>>>>> John
>> > > >>>>>>>>>>>>>>>>>>>> Burwell
>> > > >>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > jburwell@basho.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> > > >>> opinion is
>> > > >>>>> that no
>> > > >>>>>> features
>> > > >>>>>>>>>>>>>>>>>> should be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
>> > > >>> have
>> > > >>>>> been resolved
>> > > >>>>>> and
>> > > >>>>>>>>> it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>> ready
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > Ops
>> > > >>> vs
>> > > >>>>> discrete
>> > > >>>>>>>>> read/write
>> > > >>>>>>>>>>>>>>>>>> ops issue
>> > > >>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
>> > reviewed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > I
>> > > >>> don't
>> > > >>>>> think this
>> > > >>>>>>>>> criteria
>> > > >>>>>>>>>>>>>>>>>> has been
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
>> > how
>> > > >>>>> does this work
>> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>> > > >>>>>>>>>>>>>>>>>> )?
>> > > >>>>>>>>>>>>>>>>>>>> As I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>> > > >>> involves
>> > > >>>>> provisioned
>> > > >>>>>>>>> IOPS. I
>> > > >>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>> like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > scenario
>> > > >>>>> where provisioned
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> KVM and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > unnecessarily
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> incompatible.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > Jun 1,
>> > > >>> 2013,
>> > > >>>>> at 6:47 AM,
>> > > >>>>>>>>> Wei
>> > > >>>>>>>>>>>>>>>>>> ZHOU <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
>> > will
>> > > >>>>> change it next
>> > > >>>>>>>>> week.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/6/1
>> > > >>>>> Wido den Hollander
>> > > >>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 06/01/2013 08:24 AM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Exactly. I
>> > > >>> have
>> > > >>>>> pushed the
>> > > >>>>>>>>>>>>> features
>> > > >>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > anyone
>> > > >>>>> object thems for
>> > > >>>>>>>>>>>>> technical
>> > > >>>>>>>>>>>>>>>>>>>> reason
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>> > the
>> > > >>> sake
>> > > >>>>> of clarity I
>> > > >>>>>> just
>> > > >>>>>>>>>>>>> want
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> mention
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > total
>> > > >>> IOps
>> > > >>>>> to R/W IOps
>> > > >>>>>>>>> asap
>> > > >>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>> that we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>> > total
>> > > >>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
>> > laid
>> > > >>> the
>> > > >>>>> groundwork for
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>> throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > however
>> > > >>>>> prevent that we
>> > > >>>>>> create
>> > > >>>>>>>>>>>>>>>>>> legacy from
>> > > >>>>>>>>>>>>>>>>>>>>>> day
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/5/31
>> > > >>>>> Wido den
>> > > >>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/31/2013 03:59 PM, John
>> > > >>>>>>>>>>>>> Burwell
>> > > >>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
>> > this
>> > > >>>>> enhancement must
>> > > >>>>>> to
>> > > >>>>>>>>>>>>>>>>>> discretely
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > see
>> > > >>> how
>> > > >>>>> it could be
>> > > >>>>>>>>> fixed
>> > > >>>>>>>>>>>>>>>>>> later
>> > > >>>>>>>>>>>>>>>>>>>>>> because
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > correctly
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
>> > total
>> > > >>> IOPS
>> > > >>>>> into read
>> > > >>>>>> and
>> > > >>>>>>>>>>>>>>>>>> write.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>> unless/until we
>> > > >>>>>> decided
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> break
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
>> > Wei
>> > > >>>>> meant was merging
>> > > >>>>>> it
>> > > >>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>> master
>> > > >>>>>>>>>>>>>>>>>>>>>> now
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > branch
>> > > >>>>> and add Read /
>> > > >>>>>>>>> Write
>> > > >>>>>>>>>>>>> IOps
>> > > >>>>>>>>>>>>>>>>>>>> before
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
>> > > >>>>> released with Read
>> > > >>>>>> and
>> > > >>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>> instead
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
>> > to
>> > > >>>>> make the May 31st
>> > > >>>>>>>>>>>>> feature
>> > > >>>>>>>>>>>>>>>>>>>> freeze
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
>> > other
>> > > >>>>> threads) then it
>> > > >>>>>>>>> won't
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>> necessary
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
>> > > >>>>> completely agree that
>> > > >>>>>>>>>>> there
>> > > >>>>>>>>>>>>>>>>>> is no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > I/O.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > May 31,
>> > > >>>>> 2013, at 9:51 AM,
>> > > >>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>> den
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/31/2013 03:13 PM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>> > Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks.
>> > > >>> Good
>> > > >>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > thought
>> > > >>>>> about at the
>> > > >>>>>>>>>>> beginning.
>> > > >>>>>>>>>>>>>>>>>>>> Finally I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > difference
>> > > >>> of
>> > > >>>>> read and write
>> > > >>>>>>>>>>>>> mainly
>> > > >>>>>>>>>>>>>>>>>>>> because
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
>> > the
>> > > >>>>> difference of sent
>> > > >>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> received
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>> > > >>>>> reasoning seems odd.
>> > > >>>>>>>>>>>>>>>>>> Networking and
>> > > >>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>> > I/O is
>> > > >>>>> much more
>> > > >>>>>>>>> expensive
>> > > >>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> most
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > bandwith.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> Implementing
>> > > >>>>> it will be some
>> > > >>>>>>>>>>>>>>>>>> copy-paste
>> > > >>>>>>>>>>>>>>>>>>>>>> work.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> implemented
>> > > >>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
>> > days.
>> > > >>> For
>> > > >>>>> the deadline
>> > > >>>>>> of
>> > > >>>>>>>>>>>>>>>>>> feature
>> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
>> > if
>> > > >>>>> needed.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>> > think it's
>> > > >>> a
>> > > >>>>> feature we
>> > > >>>>>>>>> can't
>> > > >>>>>>>>>>>>>>>>>> miss. But
>> > > >>>>>>>>>>>>>>>>>>>>>> if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > window
>> > > >>> we
>> > > >>>>> have to make sure
>> > > >>>>>> we
>> > > >>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
>> > 4.3,
>> > > >>> that
>> > > >>>>> will confuse
>> > > >>>>>>>>>>>>> users.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/5/31
>> > > >>>>> Wido den
>> > > >>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/30/2013 06:03 PM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > would
>> > > >>> like to
>> > > >>>>> merge
>> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > nobody
>> > > >>>>> object, I will
>> > > >>>>>> merge
>> > > >>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>> master
>> > > >>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>> purpose
>> > > >>>>> is :
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
>> > > >>>>> machines are running
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> same
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > storage or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
>> > > >>> strage).
>> > > >>>>> Because of
>> > > >>>>>> the
>> > > >>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>>>> limitation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
>> > if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
>> > VM
>> > > >>> has
>> > > >>>>> large disk
>> > > >>>>>>>>> operation,
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>> affect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>> > VMs
>> > > >>>>> running on the
>> > > >>>>>> same
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>> device.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> > > >>> neccesary
>> > > >>>>> to set the
>> > > >>>>>>>>>>> maximum
>> > > >>>>>>>>>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Looking at
>> > > >>> the
>> > > >>>>> code I see
>> > > >>>>>> you
>> > > >>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>> no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
>> > and
>> > > >>>>> libvirt support
>> > > >>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>> both a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > > >>> which
>> > > >>>>> could benefit a
>> > > >>>>>>>>> lot of
>> > > >>>>>>>>>>>>>>>>>> users.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
>> > also
>> > > >>>>> strange, in the
>> > > >>>>>>>>> polling
>> > > >>>>>>>>>>>>>>>>>> side you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>> > but
>> > > >>> on
>> > > >>>>> the throttling
>> > > >>>>>>>>> side
>> > > >>>>>>>>>>>>>>>>>> you only
>> > > >>>>>>>>>>>>>>>>>>>> go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > IOps
>> > > >>> are
>> > > >>>>> usually much
>> > > >>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>> expensive
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
>> > valid
>> > > >>>>> use-case where
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > vs
>> > > >>> Read
>> > > >>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
>> > this
>> > > >>> only
>> > > >>>>> supports KVM
>> > > >>>>>>>>> at
>> > > >>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>> point I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>> > to at
>> > > >>>>> least have the
>> > > >>>>>>>>>>>>>>>>>> mechanism in
>> > > >>>>>>>>>>>>>>>>>>>>>> place
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> implementing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > later
>> > > >>>>> would be a lot of
>> > > >>>>>>>>>>> work.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
>> > > >>> hypervisor
>> > > >>>>> doesn't
>> > > >>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > you
>> > > >>> can
>> > > >>>>> always sum
>> > > >>>>>> both
>> > > >>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>> and set
>> > > >>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> > you
>> > > >>>>> explain why you
>> > > >>>>>>>>>>>>> implemented
>> > > >>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>> feature
>> > > >>>>> includes:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> > the
>> > > >>>>> maximum rate of
>> > > >>>>>>>>> VMs
>> > > >>>>>>>>>>>>> (in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> configuration)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> > change
>> > > >>> the
>> > > >>>>> maximum rate
>> > > >>>>>> of
>> > > >>>>>>>>>>> VMs
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> > limit
>> > > >>> the
>> > > >>>>> disk rate
>> > > >>>>>> (total
>> > > >>>>>>>>>>> bps
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> iops)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
>> > ticket:
>> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> issues.apache.org/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> jira/browse/CLOUDSTACK-1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1192>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>> 1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 1192
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 1192>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
>> > will
>> > > >>>>> update later) :
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>> VM+Disk+IO+Throttling<https://
>> > > >>>>>>>>>>>>> ****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cwiki.apache.org/confluence/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>
>> > > >>>
>> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>> > > >>>>> rottling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Merge
>> > > >>> check
>> > > >>>>> list :-
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
>> > you
>> > > >>>>> check the branch's
>> > > >>>>>>>>> RAT
>> > > >>>>>>>>>>>>>>>>>> execution
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
>> > > >>> there
>> > > >>>>> new dependencies
>> > > >>>>>>>>>>>>>>>>>> introduced?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> > What
>> > > >>>>> automated testing
>> > > >>>>>> (unit
>> > > >>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > feature?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
>> > tests
>> > > >>> are
>> > > >>>>> added.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> > What
>> > > >>>>> testing has been done
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> check for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> > the
>> > > >>>>> bytes rate and
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>> rate on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> > VM
>> > > >>>>> operations, including
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > deploy,
>> > > >>> stop,
>> > > >>>>> start, reboot,
>> > > >>>>>>>>>>>>>>>>>> destroy,
>> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> > > >>> Volume
>> > > >>>>> operations,
>> > > >>>>>>>>> including
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
>> > > >>>>> Detach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
>> > review
>> > > >>> the
>> > > >>>>> code, you can
>> > > >>>>>>>>> try
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
>> > > >>> regards,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>> VM+Disk+IO+Throttling<https://
>> > > >>>>>>>>>>>>> ****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cwiki.apache.org/confluence/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>
>> > > >>>
>> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>> > > >>>>> rottling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>> > > >>>>>>>>> refs/heads/disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>>>> 1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1301>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 1301
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 1301>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > issues.apache.org/****jira/**browse/CLOUDSTACK-
>> > > >>> 2071
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 2071
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 2071>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <h**ttps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 2071>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> (**CLOUDSTACK-1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> > VM
>> > > >>> Disk
>> > > >>>>> I/O
>> > > >>>>>> Throttling)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > > >>>>> Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>>>> CloudStack Developer,
>> > > >>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > > >>> 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Advancing
>> > > >>> the
>> > > >>>>> way the world
>> > > >>>>>>>>> uses
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > > >>> Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>> CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> > the
>> > > >>> way
>> > > >>>>> the world uses
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>> CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > > >>> way
>> > > >>>>> the world uses
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > way
>> > > >>> the
>> > > >>>>> world uses the
>> > > >>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > CloudStack
>> > > >>>>> Developer,
>> > > >>>>>> SolidFire
>> > > >>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > way
>> > > >>> the
>> > > >>>>> world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>>>> Developer, SolidFire
>> > > >>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> > the
>> > > >>>>> world uses the
>> > > >>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>> Developer,
>> > > >>>>> SolidFire
>> > > >>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> > the
>> > > >>> world
>> > > >>>>> uses the
>> > > >>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>> Developer,
>> > > >>>>> SolidFire
>> > > >>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > > >>> world
>> > > >>>>> uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > Developer,
>> > > >>>>> SolidFire
>> > > >>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > world
>> > > >>> uses
>> > > >>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > Developer,
>> > > >>>>> SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > world
>> > > >>> uses
>> > > >>>>> the cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>>>> SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> > uses
>> > > >>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> > uses
>> > > >>> the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > SolidFire
>> > > >>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > > >>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> --
>> > > >>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>> *(tm)*
>> > > >>>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> --
>> > > >>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>> o: 303.746.7302
>> > > >>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>> *(tm)*
>> > > >>>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>> --
>> > > >>>>>>> *Mike Tutkowski*
>> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>> o: 303.746.7302
>> > > >>>>>>> Advancing the way the world uses the
>> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>> *(tm)*
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> --
>> > > >>>>> *Mike Tutkowski*
>> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>> o: 303.746.7302
>> > > >>>>> Advancing the way the world uses the
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>> *(tm)*
>> > > >>>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> --
>> > > >>> *Mike Tutkowski*
>> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>> e: mike.tutkowski@solidfire.com
>> > > >>> o: 303.746.7302
>> > > >>> Advancing the way the world uses the
>> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>> *(tm)*
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Mike Tutkowski*
>> > > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > > e: mike.tutkowski@solidfire.com
>> > > > o: 303.746.7302
>> > > > Advancing the way the world uses the
>> > > > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > > *(tm)*
>> > >
>> > >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *(tm)*
>>
>>
>>
>>
>>
>>
>> --
>> Mike Tutkowski
>>
>> Senior CloudStack Developer, SolidFire Inc.
>>
>> e: mike.tutkowski@solidfire.com
>>
>> o: 303.746.7302
>>
>> Advancing the way the world uses the cloudâ„¢
>>
>>
>>
>>
>>
>>
>> --
>> Mike Tutkowski
>>
>> Senior CloudStack Developer, SolidFire Inc.
>>
>> e: mike.tutkowski@solidfire.com
>>
>> o: 303.746.7302
>>
>> Advancing the way the world uses the cloudâ„¢
>>
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloudâ„¢
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, Edison, that makes sense.
That is similar to how the code I put up for review works (the hypervisor
code checks a certain condition and creates the SR, if need be).
John preferred that the hypervisor code not know about that state and
instead offer up a new command to create the SR that the storage framework
could invoke if the driver says it's managed.
Up to you guys how we want to proceed.
On Mon, Jun 10, 2013 at 2:11 PM, Edison Su <Ed...@citrix.com> wrote:
> I haven’t review your patch yet, but I can show you what am doing on the
> object_store branch:****
>
> VolumeManagerImpl-> sendAttachVolumeCommand:****
>
> ** **
>
> DataTO volTO = volFactory.getVolume(volume.getId()).getTO();****
>
> 1557 DiskTO disk = new DiskTO(volTO, deviceId,
> volume.getVolumeType());****
>
> 1558 AttachCommand cmd = new AttachCommand(disk,
> vm.getInstanceName());****
>
> ** **
>
> ** **
>
> getTO is a method on each storage driver, so each storage driver can
> encode volume differently. In solidfire case, the volumeTO may contains the
> following:****
>
> ** **
>
> IQN of the volume, CHAP info, isManaged, and a dataStoreTO which represent
> your solidfire storage box.****
>
> DataStoreTO may contain the following information: the mgt ip of your
> storage box, credential of your storage box etc, etc.****
>
> ** **
>
> At the resource side, when execute(AttachCommand cmd):****
>
> DataTO vol = cmd.getdisk().getData();****
>
> If (vol.isManaged()) {****
>
> //need to create a sr based on vol.getpath(which is the IQN of the
> volume), then create a vdi on the SR, then****
>
> Vdi = a new vdi created from SR****
>
> } else {****
>
> //find out the vdi from vol.getPath(which is a uuid of the vdi)****
>
> Vid = VDI.findbyUUID(vol.getPath())****
>
> }****
>
> ** **
>
> //then attach the vdi to the VM.****
>
> ** **
>
> ** **
>
> Above code logic can be storage independent, as long as, the information
> given by DataTo is general enough for all the storage vendor. If the
> storage vendor has its own special way to access storage, then they need to
> add some functions in execute(AttachCommand cmd).****
>
> ** **
>
> Does it make sense for you?****
>
> ** **
>
> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> *Sent:* Monday, June 10, 2013 12:17 PM
> *To:* dev@cloudstack.apache.org; John Burwell; Edison Su
> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>
> ** **
>
> Well, that's sort of how I did it in the code that I originally submitted
> (except I labelled the storage as Dynamic and that was how the hypervisor
> knew what to do). :)****
>
> ** **
>
> On Mon, Jun 10, 2013 at 12:58 PM, Edison Su <Ed...@citrix.com> wrote:*
> ***
>
> I don't understand why we need to add a "managed" in the UI and in
> "storage_pool" table?
> AFAIK, what you guys trying to do is to enable attachVolume/DettachVolume
> if the volume is created on Solidfire, right?
> If the volume is created on Solidfire already, by createAsync, then why we
> can't just send out all the information about the volume(it's volumeTO
> which will be returned by each storage driver) to hypervisor, then at the
> hypervisor side, check the volumeTO, if the volumeTO is managed, then doing
> whatever necessary operations(create SR, create VDI etc), then attach the
> VDI to volume.
>
>
> > -----Original Message-----
> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > Sent: Monday, June 10, 2013 11:10 AM
> > To: John Burwell
> > Cc: Edison Su; dev@cloudstack.apache.org
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > I believe we're on the same page, John.
> >
> > I would add a new parameter to the createStoragePool API command (let's
> > call it "managed" and it can be true or false). If you pass in "true",
> but
> > the driver does not support being managed by CloudStack, an exception is
> > thrown by the driver. If you pass in "false", but the driver does not
> > support being managed outside of CloudStack, an exception is thrown by
> the
> > driver. It is possible one could write a driver that can be managed both
> > inside or outside of CloudStack (the SolidFire driver requires being
> > managed only by CloudStack).
> >
> > This managed value would end up in the storage_pool table in a new column
> > ("managed").
> >
> > An example usage: After the storage framework calls on the driver's
> > createAsync method, it will check to see if the driver is managed. If the
> > driver is managed, the storage framework will invoke the proper (new)
> > command of the hypervisor in use to set up, say, the SR for the SAN
> volume.
> >
> > There may be other cases when the isManaged() method is invoked on the
> > driver and - if true is returned - the storage framework sends a
> particular
> > command to the hypervisor in use.
> >
> > ********************
> >
> > As far as reworking the default storage plug-in (which sends commands to
> > hypervisors) for 4.2, I would check with Edison on his schedule.
> >
> > Thanks!
> >
> >
> > On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
> > wrote:
> >
> > > Mike,
> > >
> > > I want to make sure we are on the same page regarding isManaged. I see
> > > the addition of the following operations:
> > >
> > > * DataStoreDriver#supportsManagement() : boolean -- indicates whether
> > or
> > > not the driver support/implements the management functions
> > > * DataStore#isManaged() : boolean -- indicates whether or not a device
> is
> > > managed by CloudStack
> > >
> > > Only DataStores associated with DataStoreDrivers that support
> > management
> > > would be able to have the management flag enabled. This behavior
> allows
> > > operators to declare their intention to allow CloudStack to manage a
> > > device. This behavior would need to be exposed as flag on the HTTP
> API.
> > > Additionally, this flag would only be settable when a device is
> > > defined/created in the system (i.e. once a device is managed by
> CloudStack,
> > > you can't unmanaged it without dissociating it from CloudStack). In
> the
> > > future, we may consider relaxing that rule, but, in the short term, it
> > > greatly decreases the alternative flows.
> > >
> > > In terms of removing the dependencies from Storage to Hypervisor, I
> > would
> > > really like to see this accomplished in 4.2. It will be much more
> > > difficult to address this issue post 4.2 because we will have more
> plugins
> > > to address. I also see this issue as a critical architectural issues
> that
> > > for the management server. I recommend we identify the dependencies
> > that
> > > need to be inverted, and see if we can divide the work across a few
> more
> > > resources.
> > >
> > > Thanks,
> > > -John
> > >
> > > On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> > <mi...@solidfire.com>
> > > wrote:
> > >
> > > Sounds good, Edison :)
> > >
> > > I can implement John's isManaged() logic and resubmit my code for
> review.
> > >
> > > Perhaps in 4.3 I could address the direct-attach scenario. If it
> requires
> > > some refactoring of the storage framework, then I could do that.
> > >
> > > Also, the default storage plug-in does have hypervisor logic in it in
> 4.2.
> > > I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> > > with the above work I mentioned.
> > >
> > > Are you OK with that, guys?
> > >
> > >
> > > On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
> wrote:
> > >
> > >> Ok, I am Ok with the way you are talking about in 4.2.****
> > >>
> > >> ** **
> > >>
> > >> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> *Sent:* Sunday, June 09, 2013 8:59 PM
> > >> *To:* dev@cloudstack.apache.org
> > >> *Cc:* Edison Su; John Burwell
> > >> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> > >>
> > >> ** **
> > >>
> > >> Another thing we might want to consider for 4.2 is that the way the
> code
> > >> is currently implemented in master, I don't think it's even possible
> to
> > >> write a plug-in that connects storage directly to a VM because the
> > >> CloudStack storage framework will call the "attach" method of the
> > >> hypervisor in use and this method will fail because it assumes there
> is
> > >> (talking Xen here) an SR existent and in this situation there isn't
> one.*
> > >> ***
> > >>
> > >> ** **
> > >>
> > >> My thinking is that we should ignore the direct-attach-of-volume-to-VM
> > >> use case for 4.2. I think it will require too much of a rewrite at
> this
> > >> point in development (fewer than three weeks left until code
> > freeze).****
> > >>
> > >> ** **
> > >>
> > >> If the plug-in implements an isManaged() method (as John suggested),
> > the
> > >> storage framework can invoke the necessary commands on the
> > hypervisor in
> > >> use.****
> > >>
> > >> ** **
> > >>
> > >> For example, the createAsync method is called. The storage plug-in
> > >> creates a SAN volume and updates the CS DB. The storage framework
> > then asks
> > >> the driver if it's managed. If it is, the storage framework executes a
> > >> command against the hypervisor in use to have it create, say, an SR
> (with
> > a
> > >> single VDI that takes up the entire SR). Now, when the storage
> > framework
> > >> executes the "attach" command against the hypervisor, this command
> will
> > >> work, as well (because that logic assumes the prior existence of an
> SR and
> > >> it will, in fact, be present).****
> > >>
> > >> ** **
> > >>
> > >> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com> wrote:****
> > >>
> > >> Hi Edison and John,****
> > >>
> > >> ** **
> > >>
> > >> I wanted to point out something I thought was relevant to this
> > >> conversation.****
> > >>
> > >> ** **
> > >>
> > >> My plug-in supports both XenServer and ESX(i).****
> > >>
> > >> ** **
> > >>
> > >> These two hypervisors handle their respective hypervisor data
> structures
> > >> (XenServer SR or ESX(i) datastore) quite differently.****
> > >>
> > >> ** **
> > >>
> > >> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> > >> when you issue a remove-SR command to XenServer.****
> > >>
> > >> ** **
> > >>
> > >> When you later want to access the contents of that data, it is still
> > >> safely stored on the LUN and can be accessed from XenServer (from the
> > same
> > >> or different cluster (what XenServer calls a resource pool)) by
> creating a
> > >> new SR that is based on the same IQN and LUN.****
> > >>
> > >> ** **
> > >>
> > >> XenServer SRs exist at the cluster level.****
> > >>
> > >> ** **
> > >>
> > >> In ESX(i), the datastore that resides on the SAN LUN is actually
> > >> destroyed when you issue a remove-datastore command to ESX(i). In
> > other
> > >> words, the contents of the SAN LUN are destroyed.****
> > >>
> > >> ** **
> > >>
> > >> ESX(i) datastores exist at the datacenter level.****
> > >>
> > >> ** **
> > >>
> > >> The XenServer behavior (not destroying the contents of the LUN when
> > the
> > >> SR is removed) is desirable; the ESX(i) behavior (destroying the
> contents
> > >> of the LUN when the datastore is removed) is not desirable.****
> > >>
> > >> ** **
> > >>
> > >> I can get around the ESX(i) behavior by removing the referenced IQN
> > from
> > >> each host in the cluster when the CS storage framework notifies my
> plug-
> > in
> > >> of a detach-volume event. When the CS volume is later attached again
> > >> (either to the same or a different ESX(i) cluster), I can have each
> host in
> > >> the relevant ESX(i) cluster reference the IQN and the datastore will
> be
> > >> back again.****
> > >>
> > >> ** **
> > >>
> > >> This sequence of events is critical for my plug-in to support both
> > >> environments:****
> > >>
> > >> *XenServer flow (events for Driver class):*****
> > >>
> > >> createAsync (called right before first attach) = creates SAN
> volume****
> > >>
> > >> preAttachVolume = creates SR and creates VDI or just introduces SR****
> > >>
> > >> (CS volume gets attached and later detached)****
> > >>
> > >> postDetachVolume = removes SR (this does not delete data on SAN
> > volume)**
> > >> **
> > >>
> > >> deleteAsync (can be called when volume is not attached) = deletes the
> > SAN
> > >> volume****
> > >>
> > >> *VMware flow (events for Driver class):*****
> > >>
> > >> createAsync (called right before first attach) = creates SAN volume,
> > >> creates datastore, and creates VMDK file****
> > >>
> > >> preAttachVolume = adds iSCSI target to each host in cluster****
> > >>
> > >> (CS volume gets attached and later detached)****
> > >>
> > >> postDetachVolume = removes iSCSI target from each host in cluster****
> > >>
> > >> deleteAsync (can be called when volume is not attached) = deletes
> > >> datastore and deletes the SAN volume****
> > >>
> > >> ************************
> > >>
> > >> So, the way I have it written up here, the driver is talking to the
> > >> hypervisor. As John has pointed out, this may not be ideal.****
> > >>
> > >> ** **
> > >>
> > >> Whatever we decide on, it needs to be able to support this kind of
> flow.
> > >> :)****
> > >>
> > >> ** **
> > >>
> > >> We could go John's route, where the driver says if it's managed or
> not.**
> > >> **
> > >>
> > >> ** **
> > >>
> > >> If it is, then the storage framework could send the necessary
> messages to
> > >> the hypervisor in question at the right times (just migrate the
> hypervisor
> > >> logic above from the driver to the right parts of the storage
> framework).
> > >> ****
> > >>
> > >> ** **
> > >>
> > >> This is fine, except that it does not resolve Edison's use case of
> > >> desiring to have the storage LUN potentially connected directly to a
> VM.*
> > >> ***
> > >>
> > >> ** **
> > >>
> > >> I'm not sure we should design around that use case, though. Is it
> common
> > >> enough? Has anyone asked for it? Is it even desirable? How are usage
> > >> statistics gathered for such volumes? It's nice to offer the
> flexibility,
> > >> but is it a good idea in this case?****
> > >>
> > >> ** **
> > >>
> > >> Thanks!****
> > >>
> > >> ** **
> > >>
> > >> ** **
> > >>
> > >> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> wrote:**
> > >> **
> > >>
> > >>
> > >>
> > >> > -----Original Message-----
> > >> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> > Sent: Friday, June 07, 2013 4:08 PM
> > >> > To: dev@cloudstack.apache.org
> > >> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >> >
> > >> > Hi John,
> > >> >
> > >> > Yes, you are correct that Xen must layer an SR on top of the iSCSI
> > >> volume
> > >> > to use it. Same story for VMware: ESX(i) must layer a datastore on
> top
> > >> of
> > >> > the iSCSI volume to use it. I look at it like they are layering a
> > >> clustered
> > >> > file system on the SAN volume so the hypervisors can share access to
> > the
> > >> > contents of the volume.
> > >> >
> > >> > In 4.1, by the time the hypervisor-attach-volume logic is called,
> the
> > >> SR or
> > >> > datastore has already been created (usually manually by an admin).
> > >> >
> > >> > This pre-setup of the, say, SR is, of course, not acceptable in an
> > >> > environment where each CloudStack volume that a user creates is
> > mapped
> > >> > to a
> > >> > single SAN volume (via the SR).
> > >> >
> > >> > The question comes down to who should allocate the SR dynamically.
> > >> >
> > >> > We could have the storage framework ask the storage plug-in if it is
> > >> > managed. If it is, then the storage framework could send a message
> to
> > >> the
> > >> > hypervisor in question to create (let's talk Xen here) the SR ahead
> of
> > >> > time. Then, when the storage framework next sends the attach-volume
> > >> > command
> > >> > to the hypervisor, it should work without changes to that
> attach-volume
> > >> > logic (because - from the point of view of the attach logic - the
> SR is
> > >> > already existent, as expected).
> > >> >
> > >> > Now, as Edison has pointed out, this limits the power of the storage
> > >> > plug-in. A storage plug-in in this model cannot directly attach
> storage
> > >> to
> > >> > a VM (it must go through the hypervisor). Perhaps that is OK. We
> need
> > to
> > >> > make a call on that.
> > >>
> > >>
> > >> Yes, I think we need to address this usage case, and won't limit the
> > >> power of the storage plug-in.
> > >> John, what's your idea to fit this usage case into your abstraction?
> > >>
> > >>
> > >> >
> > >> > If we want to give more power to the storage vendor, we would have
> to
> > >> > have
> > >> > the storage framework call into the storage plug-in to send the
> > >> appropriate
> > >> > attach commands to the hypervisor.
> > >> >
> > >> > Let's discuss and come to a consensus (or at least agree on a path)
> as
> > >> soon
> > >> > as we can. :)
> > >> >
> > >> > Thanks!
> > >> >
> > >> >
> > >> >
> > >> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
> > >> wrote:
> > >> >
> > >> > > Mike,
> > >> > >
> > >> > > My understanding is the Xen expects an SR structure on all ISCSI
> > >> devices
> > >> > > -- at least that is how I read the code in your patch. Is my
> > >> understanding
> > >> > > correct? If so, the Xen plugin should be able to query the
> storage
> > >> device
> > >> > > to determine the presence of the SR structure and create it if it
> > >> does not
> > >> > > exist. Am I missing something in the implementation that makes
> that
> > >> type
> > >> > > of implementation impossible?
> > >> > >
> > >> > > Thanks,
> > >> > > -John
> > >> > >
> > >> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Yeah, if a storage vendor wanted to attach a volume directly to
> a
> > >> VM in
> > >> > > 4.2
> > >> > > > today, it would probably fail because the attach-volume logic
> > >> assumes
> > >> > the
> > >> > > > existence of the necessary hypervisor data structure (ex. SR on
> > >> Xen).
> > >> > > >
> > >> > > > If we wanted to enable such an attach, we could do it the way
> > Edison
> > >> > > > suggests.
> > >> > > >
> > >> > > >
> > >> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Edison.su@citrix.com
> >
> > >> wrote:
> > >> > > >
> > >> > > >>
> > >> > > >>
> > >> > > >>> -----Original Message-----
> > >> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> > > >>> Sent: Friday, June 07, 2013 2:37 PM
> > >> > > >>> To: dev@cloudstack.apache.org
> > >> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >> > > >>>
> > >> > > >>> As we only have three weeks until feature freeze, we should
> > come
> > >> to
> > >> > a
> > >> > > >>> consensus on this design point as soon as possible.
> > >> > > >>>
> > >> > > >>> Right now, if the storage framework asks my driver it is is
> > >> managed, it
> > >> > > >>> will say 'yes.' This means the framework will tell the driver
> to
> > >> > > perform
> > >> > > >>> its management activities. This then means the driver will
> call
> > >> into
> > >> > > the
> > >> > > >>> host (it doesn't know which hypervisor, by the way) to perform
> > the
> > >> > > >> activity
> > >> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> > >> > > >>>
> > >> > > >>> The driver doesn't know which hypervisor it's talking to, it
> just
> > >> > > sends a
> > >> > > >>> message to the host to perform the necessary pre-attach work.
> > >> > > >>
> > >> > > >> Could we just expose a method like
> > "attachVolume/dettachVolume" on
> > >> > the
> > >> > > >> PrimaryDataStoreDriver driver?
> > >> > > >> For most of cases, the implementation of each driver would just
> > >> send
> > >> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can
> > put
> > >> > the
> > >> > > >> implementation in a base class, so that can be shared by all of
> > >> this
> > >> > > >> drivers), and in each hypervisor resource code, which may just
> call
> > >> > > >> hypervisor's api to attach the volume to VM, while for certain
> > >> storage,
> > >> > > >> like SolidFire, may need to create a SR first, and create
> volume
> > >> on it,
> > >> > > >> then call hypervisor's API to attach volume to VM.
> > >> > > >> While for some other storage vendor, may want to bypass
> > hypervisor
> > >> > > during
> > >> > > >> attaching volume, so inside driver's attachvolume
> implementation,
> > >> the
> > >> > > >> driver can do some magic, such as, directly talk to an agent
> > >> inside VM
> > >> > > >> instance, then create a disk inside VM.
> > >> > > >>
> > >> > > >> How do you guys think?
> > >> > > >>
> > >> > > >>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <
> Edison.su@citrix.com>
> > >> > > wrote:
> > >> > > >>>
> > >> > > >>>>
> > >> > > >>>>
> > >> > > >>>>> -----Original Message-----
> > >> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> > >> > > >>>>> To: dev@cloudstack.apache.org
> > >> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >> > > >>>>>
> > >> > > >>>>> Hi John,
> > >> > > >>>>>
> > >> > > >>>>> How's about this:
> > >> > > >>>>>
> > >> > > >>>>> The driver can implement an isManaged() method. The
> > >> > > >>> VolumeManagerImpl
> > >> > > >>>>> can
> > >> > > >>>>> call into the driver to see if its managed. If it is, the
> > >> > > >>>> VolumeManagerImpl
> > >> > > >>>>> (which is responsible for calling into the hypervisor to
> attach
> > >> the
> > >> > > >> disk)
> > >> > > >>>>> can call into the hypervisor to create the necessary
> hypervisor
> > >> data
> > >> > > >>>>> structure (ex. for XenServer, a storage repository).
> > >> > > >>>>
> > >> > > >>>> The problem here is that storage vendor may work differently
> > with
> > >> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> > >> maybe
> > >> > > >> other
> > >> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> > >> > directly
> > >> > > >> to
> > >> > > >>>> VM instance, see the discuss(
> > >> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> > >> > > >>>
> > >> >
> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> > >> > > >>> %3E
> > >> > > >>>> ).
> > >> > > >>>> So I would let storage provider to implement attach disk to
> VM,
> > >> > > >> instead of
> > >> > > >>>> implemented by cloudstack itself.
> > >> > > >>>>
> > >> > > >>>>
> > >> > > >>>>>
> > >> > > >>>>> If that's what you're going for, that works for me. By the
> way,
> > >> > > >> Edison's
> > >> > > >>>>> default storage plug-in (which handles the default storage
> > >> behavior
> > >> > > >> in
> > >> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> > >> talks to
> > >> > > >>>>> hypervisors. You might want to contact him and inform him of
> > >> your
> > >> > > >>>> concerns
> > >> > > >>>>> or that logic (as is) will make it to production.
> > >> > > >>>>>
> > >> > > >>>>> Please let me know if what I wrote in above (for my
> solution)
> > >> is OK
> > >> > > >> with
> > >> > > >>>>> you. :)
> > >> > > >>>>>
> > >> > > >>>>> Thanks!
> > >> > > >>>>>
> > >> > > >>>>>
> > >> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> > >> jburwell@basho.com>
> > >> > > >>> wrote:
> > >> > > >>>>>
> > >> > > >>>>>> Mike,
> > >> > > >>>>>>
> > >> > > >>>>>> Please see my responses in-line below.
> > >> > > >>>>>>
> > >> > > >>>>>> Thanks,
> > >> > > >>>>>> -John
> > >> > > >>>>>>
> > >> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > >> > > >>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>> wrote:
> > >> > > >>>>>>
> > >> > > >>>>>>> Hey John,
> > >> > > >>>>>>>
> > >> > > >>>>>>> I still have a bit more testing I'd like to do before I
> build
> > >> up
> > >> > > >> a
> > >> > > >>>> patch
> > >> > > >>>>>>> file, but this is the gist of what I've done:
> > >> > > >>>>>>>
> > >> > > >>>>>>> * During a volume-attach operation, after
> > VolumeManagerImpl
> > >> > tells
> > >> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I
> > have
> > >> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> > driver if
> > >> it
> > >> > > >>>>> managed.
> > >> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> > >> > > >> whatever
> > >> > > >>>>>>> activity is required. In my case, this includes sending a
> > >> > > >> message to
> > >> > > >>>> the
> > >> > > >>>>>>> host where the VM is running to have, say XenServer, add a
> > >> > > >> storage
> > >> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of
> > the
> > >> > > >> SAN
> > >> > > >>>>>> volume,
> > >> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space
> it
> > >> can
> > >> > > >> on
> > >> > > >>>> the
> > >> > > >>>>>>> storage repository). After this, the normal attach-volume
> > >> > > >> message is
> > >> > > >>>> sent
> > >> > > >>>>>>> to the host by VolumeManagerImpl.
> > >> > > >>>>>>
> > >> > > >>>>>> There should be **no** code from a storage driver to a
> > >> hypervisor.
> > >> > > >> I
> > >> > > >>>>>> apologize for the repetition, but we simply can not have
> > >> hypervisor
> > >> > > >>>>>> specific code in the storage layer. The circular
> dependencies
> > >> > > >> between
> > >> > > >>>> the
> > >> > > >>>>>> two layers are not sustainable in the long term. Either
> the
> > >> > > >>>> VirtualManager
> > >> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> > >> > > >>> coordinate
> > >> > > >>>>>> volume creation and then populating the SR. Ideally, we
> can
> > >> > > >>>> generalize the
> > >> > > >>>>>> process flow for attaching volumes such that the Xen
> > hypervisor
> > >> > > >> plugin
> > >> > > >>>>>> would only implement callbacks to perform the attach action
> > and
> > >> > > >> create
> > >> > > >>>>> the
> > >> > > >>>>>> structure and SR. To my mind, the SolidFire driver should
> > >> only be
> > >> > > >>>>>> allocating space and providing information about contents
> > (e.g.
> > >> > > >> space
> > >> > > >>>>>> available, space consumed, streams to a URI, file handle
> for a
> > >> URI,
> > >> > > >>>> etc)
> > >> > > >>>>>> and capabilities.
> > >> > > >>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> * The reverse is performed for a detach-volume command.
> > >> > > >>>>>>>
> > >> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
> > >> driver.
> > >> > > >>>>>> Edison's
> > >> > > >>>>>>> default driver simply does a "return false;". We could
> add a
> > >> new
> > >> > > >>>>>> parameter
> > >> > > >>>>>>> to the createStoragePool API command, if we want, to
> > remove
> > >> > the
> > >> > > >>>>>> hard-coded
> > >> > > >>>>>>> return values in the drivers (although my driver will
> probably
> > >> > > >> just
> > >> > > >>>>>> ignore
> > >> > > >>>>>>> this parameter and always return true since it wouldn't
> make
> > >> > > >> sense
> > >> > > >>>> for it
> > >> > > >>>>>>> to ever return false). We'd need another column in the
> > >> > > >> storage_pool
> > >> > > >>>>> table
> > >> > > >>>>>>> to store this value.
> > >> > > >>>>>>
> > >> > > >>>>>> Yes, I think we should have a parameter added to the
> > >> > > >>> createStoragePool
> > >> > > >>>>>> surfaced to the HTTP API that allows DataStores to be
> > >> configured
> > >> > > >> for
> > >> > > >>>>>> management when their underlying drivers support it. To
> > >> simplify
> > >> > > >>>> things,
> > >> > > >>>>>> this flag should only be mutable when the DataStore is
> > >> created. It
> > >> > > >>>> would be
> > >> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged
> > after
> > >> > > >>>> creation.
> > >> > > >>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> Sound like I'm in sync with what you were thinking?
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > >> > > >>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>> wrote:
> > >> > > >>>>>>>
> > >> > > >>>>>>>> I agree, John. Just wanted to point out that I have a
> > working
> > >> > > >> GUI
> > >> > > >>>> for
> > >> > > >>>>>> you
> > >> > > >>>>>>>> to review (in that document), if you'd like to check it
> out.
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> Thanks!
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> > >> > > >> jburwell@basho.com>
> > >> > > >>>>>> wrote:
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>> Mike,
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> I would like the UIs of two features reviewed together
> to
> > >> > > >> ensure
> > >> > > >>>>>>>>> consistency across the concepts of hypervisor throttled
> > IOPs
> > >> > > >> and
> > >> > > >>>>>>>>> storage device provisioned IOPs. I see the potential
> for
> > >> > > >>>> confusion,
> > >> > > >>>>>>>>> and I think a side-by-side Ui review of these features
> will
> > >> > > >> help
> > >> > > >>>>>>>>> minimize any potential confusion.
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
> > >> itself
> > >> > > >> if
> > >> > > >>>> it
> > >> > > >>>>>>>>> is acceptable that a VM is only permitted utilize
> hypervisor
> > >> > > >>>> throttled
> > >> > > >>>>>>>>> IOPs or storage provisioned IOPs.
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> Thanks,
> > >> > > >>>>>>>>> -John
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > >> > > >>>>>>>>> <mi...@solidfire.com> wrote:
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
> > sent
> > >> > to
> > >> > > >>> you
> > >> > > >>>>> the
> > >> > > >>>>>>>>>> other day to see how the GUI looks for provisioned
> > storage
> > >> > > >> IOPS.
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail
> list
> > >> > > >> and we
> > >> > > >>>>>>>>> decided
> > >> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> > >> > Offering
> > >> > > >>>> dialog.
> > >> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> > >> > should
> > >> > > >>> be
> > >> > > >>>>> able
> > >> > > >>>>>> to
> > >> > > >>>>>>>>>> leverage this GUI going forward (even if they, say,
> only
> > >> use
> > >> > > >> Max
> > >> > > >>>>>> IOPS).
> > >> > > >>>>>>>>>> These fields are optional and can be ignored for
> storage
> > >> that
> > >> > > >>>> does not
> > >> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size
> field,
> > >> the
> > >> > > >>>> admin can
> > >> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> > Burst
> > >> > > >> IOPS.
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's
> feature
> > or
> > >> > > >> mine,
> > >> > > >>>> as
> > >> > > >>>>>> is
> > >> > > >>>>>>>>>> decided by the admin).
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> I'm not sure what we can do about these two features
> > >> > > >> expressing
> > >> > > >>>> the
> > >> > > >>>>>>>>> speed
> > >> > > >>>>>>>>>> in different terms. I've never seen a SAN express the
> > IOPS
> > >> for
> > >> > > >>>> QoS in
> > >> > > >>>>>>>>> any
> > >> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
> > >> read/write
> > >> > > >>>> IOPS).
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Thanks
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > >> > > >>> <jb...@basho.com>
> > >> > > >>>>>>>>> wrote:
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>> Wei,
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> > >> > > >>> Storage/Hypervisor
> > >> > > >>>>>> layer
> > >> > > >>>>>>>>>>> separation, but we still need to reconcile the
> behavior
> > of
> > >> > > >>>> hypervisor
> > >> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> > >> > > >> following
> > >> > > >>>>>> issues
> > >> > > >>>>>>>>>>> outstanding:
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > >> > > >>> read/write
> > >> > > >>>>>>>>> values
> > >> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as
> > total
> > >> > > >> IOPS.
> > >> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
> > to
> > >> > > >>> storage
> > >> > > >>>>>> volumes
> > >> > > >>>>>>>>>>> with provisioned IOPS?
> > >> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> > >> > > >>>> provisioned
> > >> > > >>>>>> IOPS
> > >> > > >>>>>>>>>>> that will permit providers to discriminate these
> > >> guaranteed
> > >> > > >>>>>> operations
> > >> > > >>>>>>>>> in
> > >> > > >>>>>>>>>>> the event they want to bill for it differently?
> > >> > > >>>>>>>>>>> 4. What is the user experience for throttled and
> > >> provisioned
> > >> > > >>> IOPS
> > >> > > >>>>>> that
> > >> > > >>>>>>>>>>> minimizes confusion of these concepts?
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> My thinking is that a VM can have either utilize
> > >> hypervisor
> > >> > > >>>> throttled
> > >> > > >>>>>>>>> IOPS
> > >> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> > >> solve
> > >> > > >>>> items 1
> > >> > > >>>>>>>>> and 2.
> > >> > > >>>>>>>>>>> Since the two facilities would not be permitted to
> > operate
> > >> > > >>>> together,
> > >> > > >>>>>>>>> they
> > >> > > >>>>>>>>>>> do not need to be semantically compatible. I think
> item
> > 3
> > >> > > >> can be
> > >> > > >>>>>>>>> resolved
> > >> > > >>>>>>>>>>> with an additional flag or two on the usage records.
> As
> > >> for
> > >> > > >>>> Item 4,
> > >> > > >>>>>> I
> > >> > > >>>>>>>>> am
> > >> > > >>>>>>>>>>> not familiar with how these two enhancements are
> > >> > depicted in
> > >> > > >>> the
> > >> > > >>>>> user
> > >> > > >>>>>>>>>>> interface. I think we need to review the UI
> > enhancements
> > >> > for
> > >> > > >>>> both
> > >> > > >>>>>>>>>>> enhancements and ensure they are consistent.
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> Do these solutions make sense?
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> > >> > <us...@gmail.com>
> > >> > > >>>>> wrote:
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>> John and Mike,
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> > >> > 2780/2729,
> > >> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > >> > > >>> CLOUDSTACK-
> > >> > > >>>>> 2875 ) this
> > >> > > >>>>>>>>> week.
> > >> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
> > and
> > >> > > >> ask
> > >> > > >>>> for
> > >> > > >>>>>>>>> second
> > >> > > >>>>>>>>>>>> merge request after testing.
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>> -Wei
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> > <mi...@solidfire.com>
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
> > >> John,
> > >> > > >> and
> > >> > > >>>>> have
> > >> > > >>>>>>>>> been
> > >> > > >>>>>>>>>>>>> re-working this section of code today.
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> Thanks for the comments,
> > >> > > >>>>>>>>>>>>> Mike
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > >> > > >>>>> <jb...@basho.com>
> > >> > > >>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> See my responses in-line below.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Thanks for the response.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> > >> > whether or
> > >> > > >>>>> not a
> > >> > > >>>>>>>>> Volume
> > >> > > >>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> > but an
> > >> > > >>> actual
> > >> > > >>>>>>>>> isManaged
> > >> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> > > >> resources
> > >> > > >>>> for
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>> if it is managed."
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> > >> > method
> > >> > > >>>>> added to
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns
> true,
> > >> > > >> then the
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever
> > name)
> > >> > > >>> method
> > >> > > >>>>> (which
> > >> > > >>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
> > well)
> > >> > and
> > >> > > >>> this
> > >> > > >>>>>> would
> > >> > > >>>>>>>>>>> call
> > >> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create,
> > say
> > >> > on
> > >> > > >>>>>>>>> XenServer,
> > >> > > >>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>> SR?
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> > >> > > >>>>> DataStoreDriver.
> > >> > > >>>>>>>>> Since
> > >> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> > storage,
> > >> > I
> > >> > > >>>> would
> > >> > > >>>>>> add
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> > >> > straw
> > >> > > >>> man
> > >> > > >>>>> term
> > >> > > >>>>>>>>> --
> > >> > > >>>>>>>>>>>>>> something other than volume) methods to
> > >> > > >>>>>>>>>>> allocate/create/deallocate/delete
> > >> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> > mutable
> > >> > > >>> property
> > >> > > >>>>> of
> > >> > > >>>>>>>>>>>>> DataStore
> > >> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> > >> > > >>> DataStoreDriver
> > >> > > >>>>> can be
> > >> > > >>>>>>>>>>>>> managed.
> > >> > > >>>>>>>>>>>>>> This approach allows operators to override
> > >> > manageability
> > >> > > >> of
> > >> > > >>>>>> devices.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
> > Xen
> > >> > > >> plugin
> > >> > > >>>>>> would
> > >> > > >>>>>>>>> be
> > >> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume
> > methods
> > >> > to
> > >> > > >>>>> create any
> > >> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There
> > >> should
> > >> > > >> be no
> > >> > > >>>>>>>>>>>>> dependencies
> > >> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I
> said
> > >> > > >> earlier,
> > >> > > >>>> such
> > >> > > >>>>>>>>>>>>> circular
> > >> > > >>>>>>>>>>>>>> dependencies will lead to a tangled,
> > unmaintainable
> > >> > mess.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with
> > you.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Thanks again, John
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > >> > > >>>>> <jb...@basho.com>
> > >> > > >>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
> > layer
> > >> > that
> > >> > > >>>>>> supports n
> > >> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> > >> > > >> hypervisors.
> > >> > > >>>>>> Such
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and
> > untestable
> > >> > > >>> beast.
> > >> > > >>>>>>>>>>>>> Therefore, my
> > >> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to
> > evolve
> > >> > the
> > >> > > >>>>> Storage
> > >> > > >>>>>> layer
> > >> > > >>>>>>>>>>>>> towards
> > >> > > >>>>>>>>>>>>>>> this separation.
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> > >> > whether
> > >> > > >>> or
> > >> > > >>>>> not a
> > >> > > >>>>>>>>> Volume
> > >> > > >>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> > but
> > >> > an
> > >> > > >>> actual
> > >> > > >>>>>>>>>>> isManaged
> > >> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> > > >> resources
> > >> > > >>>> for
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
> > >> needs
> > >> > > >> to
> > >> > > >>>>>> indicate
> > >> > > >>>>>>>>>>>>> whether
> > >> > > >>>>>>>>>>>>>>> or not it supports management operations.
> > Finally, I
> > >> > > >> think
> > >> > > >>>> we
> > >> > > >>>>>>>>> need to
> > >> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect
> to
> > >> have
> > >> > > >>>>>> something
> > >> > > >>>>>>>>>>>>> that is
> > >> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> > capable
> > >> > > >>> managing
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> > >> > unmanaged).
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be
> > done
> > >> > in
> > >> > > >>> the
> > >> > > >>>>> Xen
> > >> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> > >> > Storage
> > >> > > >>>>> layer to
> > >> > > >>>>>>>>>>> perform
> > >> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> > create a
> > >> > > >> file,
> > >> > > >>>> etc).
> > >> > > >>>>>>>>>>> This
> > >> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> > >> > > >>>> hypervisor, as
> > >> > > >>>>>>>>> such,
> > >> > > >>>>>>>>>>>>>>> should be confined to its implementation. From
> > my
> > >> > > >>>>> perspective,
> > >> > > >>>>>>>>>>> nothing
> > >> > > >>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with
> > content.
> > >> > From
> > >> > > >>> its
> > >> > > >>>>>>>>>>>>> perspective,
> > >> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the
> > means
> > >> > to
> > >> > > >>>> query
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>> support the interpretation of the content by
> > higher-
> > >> > level
> > >> > > >>>>> layers
> > >> > > >>>>>>>>> (e.g.
> > >> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> > >> > composition
> > >> > > >>> of
> > >> > > >>>>>>>>>>> operations
> > >> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> > >> > Volume
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>> protocol
> > >> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> > >> > implemented
> > >> > > >> in
> > >> > > >>>> my
> > >> > > >>>>>>>>> patch,
> > >> > > >>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>> could do the following:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> > >> > > >>>>>>>>> VolumeManagerImpl
> > >> > > >>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
> > the
> > >> > > >>>>> hypervisor.
> > >> > > >>>>>> Before
> > >> > > >>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> > >> > dealing
> > >> > > >>> with
> > >> > > >>>>>>>>> Dynamic
> > >> > > >>>>>>>>>>> (or
> > >> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> > >> send a
> > >> > > >>>> "Create
> > >> > > >>>>>>>>> SR"
> > >> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> > >> > would
> > >> > > >>> then
> > >> > > >>>>>>>>> proceed
> > >> > > >>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and
> > we just
> > >> > > >> add
> > >> > > >>>>>> another
> > >> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
> > >> > > >> particular
> > >> > > >>>>> type
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>> storage.
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> What do you think?
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> Hey John,
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
> > >> > outlines
> > >> > > >>> how
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably
> > be
> > >> of
> > >> > > >> use
> > >> > > >>>> to
> > >> > > >>>>>>>>> you, as
> > >> > > >>>>>>>>>>>>> well,
> > >> > > >>>>>>>>>>>>>>>> as you perform the code review.
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> I have shared this document with you (you
> > should
> > >> > have
> > >> > > >>>>> received
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> Talk to you later!
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > >> > > >>>>>> jburwell@basho.com
> > >> > > >>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> > >> > thoughts,
> > >> > > >>> but
> > >> > > >>>>>> want
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> > >> solidfire,
> > >> > > >>>>>>>>>>>>> disk_io_throttle,
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
> > in a
> > >> > > >>>>> coherent
> > >> > > >>>>>>>>> form
> > >> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> > introduces
> > >> > the
> > >> > > >>>> notion
> > >> > > >>>>>> of
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> > introduces
> > >> > > >> the
> > >> > > >>>>> notion
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the
> > plug-in
> > >> > was
> > >> > > >>>>>>>>> incomplete,
> > >> > > >>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>> I had
> > >> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> > >> > > >> device.
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> > spending on
> > >> > > >> this.
> > >> > > >>>> :)
> > >> > > >>>>> Do
> > >> > > >>>>>>>>> you
> > >> > > >>>>>>>>>>>>>>>>>> have a
> > >> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should
> > accomplish
> > >> > > >>> what
> > >> > > >>>>> you're
> > >> > > >>>>>>>>> looking
> > >> > > >>>>>>>>>>>>>>>>>> for?
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
> > <
> > >> > > >>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> > introduces
> > >> > the
> > >> > > >>>> notion
> > >> > > >>>>>> of
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> > management
> > >> > > >>>>> capabilities
> > >> > > >>>>>>>>> (i.e.
> > >> > > >>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
> > it
> > >> > > >>> managed
> > >> > > >>>>> by
> > >> > > >>>>>>>>> CloudStack)
> > >> > > >>>>>>>>>>>>>>>>>> from the
> > >> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
> > with
> > >> > a
> > >> > > >>>>> semantic
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>> will
> > >> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> > managed.
> > >> > I
> > >> > > >>> also
> > >> > > >>>>> want
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
> > from
> > >> > the
> > >> > > >>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> definitions.
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
> > <
> > >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> > Burwell <
> > >> > > >>>>>>>>>>> jburwell@basho.com
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> > >> > questions
> > >> > > >>> for
> > >> > > >>>>> me
> > >> > > >>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>> regards to
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> > LUNs
> > >> > and
> > >> > > >>> SAN
> > >> > > >>>>>> volumes?
> > >> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
> > LUN
> > >> > (LUN
> > >> > > >>> 0).
> > >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> > Volumes
> > >> > and
> > >> > > >>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> Volumes?
> > >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> > management
> > >> > > >>> server or
> > >> > > >>>>>>>>> externally
> > >> > > >>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>> operator?
> > >> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
> > SAN
> > >> > > >>> volume
> > >> > > >>>>> (same
> > >> > > >>>>>> as a
> > >> > > >>>>>>>>>>> SAN
> > >> > > >>>>>>>>>>>>>>>>>> LUN) is
> > >> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via
> > the
> > >> > plug-in)
> > >> > > >>>>> the
> > >> > > >>>>>> first
> > >> > > >>>>>>>>>>> time
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> > hypervisor.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
> > in,
> > >> > but
> > >> > > >>>> still
> > >> > > >>>>>>>>> want
> > >> > > >>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> use a
> > >> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
> > already
> > >> > > >>> today
> > >> > > >>>>>> (prior
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> 4.2). The
> > >> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
> > and -
> > >> > in
> > >> > > >>> this
> > >> > > >>>>> case -
> > >> > > >>>>>>>>>>>>>>>>>> multiple VMs
> > >> > > >>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> > While
> > >> > you
> > >> > > >>> can do
> > >> > > >>>>> this
> > >> > > >>>>>>>>>>>>> today,
> > >> > > >>>>>>>>>>>>>>>>>> it is
> > >> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
> > QoS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> > management
> > >> > server
> > >> > > >>> or
> > >> > > >>>>> externally
> > >> > > >>>>>>>>> by
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> operator?
> > >> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> > SAN
> > >> > > >>> volumes
> > >> > > >>>>> are
> > >> > > >>>>>>>>>>>>> completely
> > >> > > >>>>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
> > in).
> > >> > There
> > >> > > >>> is
> > >> > > >>>>> no
> > >> > > >>>>>> admin
> > >> > > >>>>>>>>>>>>>>>>>>>> interaction.
> > >> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
> > SAN
> > >> > > >>> volume
> > >> > > >>>>> and a
> > >> > > >>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
> > that
> > >> > > >>>>> supports
> > >> > > >>>>>> true
> > >> > > >>>>>>>>>>> QoS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
> > are
> > >> > > >> related
> > >> > > >>>>> and
> > >> > > >>>>>>>>>>>>> expected
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> operate.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> > Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> > >> > multiple
> > >> > > >>> VMs
> > >> > > >>>>> with
> > >> > > >>>>>>>>>>>>>>>>>> provisioned
> > >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> > storage."
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
> > map a
> > >> > > >>> single
> > >> > > >>>>> SAN
> > >> > > >>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>> (LUN) to a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
> > have
> > >> > > >>> multiple
> > >> > > >>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> > >> > guarantee
> > >> > > >>> QoS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> > >> > volume
> > >> > > >>> house
> > >> > > >>>>> more
> > >> > > >>>>>>>>> than
> > >> > > >>>>>>>>>>>>> one
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
> > today
> > >> > > >>> without
> > >> > > >>>>> any of
> > >> > > >>>>>> my
> > >> > > >>>>>>>>>>>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>>>> code.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> > >> > Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> > >> > volume
> > >> > > >>> for
> > >> > > >>>>>>>>>>> CloudStack's
> > >> > > >>>>>>>>>>>>>>>>>> use
> > >> > > >>>>>>>>>>>>>>>>>>>> onto
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> > created."
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> > >> > Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> > >> > concerns me
> > >> > > >>> --
> > >> > > >>>>>> concepts
> > >> > > >>>>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
> > layer.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> > >> > Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
> > is
> > >> > today
> > >> > > >>>> only
> > >> > > >>>>>> used
> > >> > > >>>>>>>>>>> (as
> > >> > > >>>>>>>>>>>>>>>>>> far as
> > >> > > >>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
> > have
> > >> > not
> > >> > > >>>>> PreSetup an
> > >> > > >>>>>>>>> SR).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> > volume
> > >> > from
> > >> > > >>>>> VMware, it
> > >> > > >>>>>>>>> uses
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> vmfs
> > >> > > >>>>>>>>>>>>>>>>>>>>>> type.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> > volume
> > >> > from
> > >> > > >>> KVM,
> > >> > > >>>>> it uses
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> > thinking
> > >> > > >>> here
> > >> > > >>>>> was to
> > >> > > >>>>>>>>> make a
> > >> > > >>>>>>>>>>>>>>>>>> new type
> > >> > > >>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
> > ability
> > >> > > >>> Edison
> > >> > > >>>>> added
> > >> > > >>>>>>>>> into
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> > more
> > >> > > >>> specificy,
> > >> > > >>>>> though:
> > >> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> > Mike
> > >> > > >>> Tutkowski
> > >> > > >>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> > shouldn't
> > >> > know
> > >> > > >>> or
> > >> > > >>>>> care
> > >> > > >>>>>> that
> > >> > > >>>>>>>>> it
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>> being
> > >> > > >>>>>>>>>>>>>>>>>>>>>> used
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
> > answer
> > >> > > >>>>> questions
> > >> > > >>>>>>>>> about
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
> > the
> > >> > > >>>>> SolidFire
> > >> > > >>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>> needs to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
> > the
> > >> > > >>> hypervisor
> > >> > > >>>>> logic
> > >> > > >>>>>>>>>>> knows
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>> treat
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> > >> > constraint
> > >> > > >>>>> and just
> > >> > > >>>>>>>>>>>>>>>>>> calling it
> > >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
> > way
> > >> > for
> > >> > > >>> the
> > >> > > >>>>> hypervisor
> > >> > > >>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> > >> > perform
> > >> > > >>> the
> > >> > > >>>>> necessary
> > >> > > >>>>>>>>>>>>>>>>>> activities.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> > John
> > >> > > >>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > >> > > >>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
> > be
> > >> > able to
> > >> > > >>>>> manage a
> > >> > > >>>>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > >> > > >>> CloudStack
> > >> > > >>>>> (e.g. the
> > >> > > >>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>> shared
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> > operator has
> > >> > > >>> chosen
> > >> > > >>>>> to
> > >> > > >>>>>>>>> dedicate
> > >> > > >>>>>>>>>>>>>>>>>> only a
> > >> > > >>>>>>>>>>>>>>>>>>>>>> portion
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> > reasoning
> > >> > > >>> make
> > >> > > >>>>> sense?"
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> > creating a
> > >> > SAN
> > >> > > >>>>> volume via
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> > hypervisor)
> > >> > would
> > >> > > >>>>> work unless
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
> > say,
> > >> > an
> > >> > > >>> SR.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
> > of
> > >> > > >>>>> CloudStack, an
> > >> > > >>>>>>>>>>>>> operator
> > >> > > >>>>>>>>>>>>>>>>>>>>>> allocates
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
> > and
> > >> > > >>> exposes it
> > >> > > >>>>>> through
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> LUN. They
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
> > over
> > >> > to
> > >> > > >>>>> CloudStack,
> > >> > > >>>>>>>>> but
> > >> > > >>>>>>>>>>>>>>>>>> not allow
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
> > LUNs.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
> > to my
> > >> > > >>>>> changes, we
> > >> > > >>>>>>>>> would
> > >> > > >>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>> passing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
> > the
> > >> > > >>>>> necessary
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>> (like
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> > maybe
> > >> > have
> > >> > > >>> the
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>>>> framework
> > >> > > >>>>>>>>>>>>>>>>>>>> itself
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> > >> > support
> > >> > > >>> from
> > >> > > >>>>> the
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> side and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> > specifically
> > >> > to
> > >> > > >>> create
> > >> > > >>>>> the
> > >> > > >>>>>>>>> SR
> > >> > > >>>>>>>>>>>>>>>>>> before the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> > management
> > >> > plugin
> > >> > > >>>>> should
> > >> > > >>>>>> have a
> > >> > > >>>>>>>>>>>>> rich
> > >> > > >>>>>>>>>>>>>>>>>> enough
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> > determine
> > >> > > >>> available
> > >> > > >>>>> for
> > >> > > >>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>> storage.
> > >> > > >>>>>>>>>>>>>>>>>>>> For
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > >> > > >>> interrogation of
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> > device
> > >> > itself
> > >> > > >>>>> shouldn't
> > >> > > >>>>>>>>> know
> > >> > > >>>>>>>>>>>>>>>>>> or care
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> > simply be
> > >> > able
> > >> > > >>> to
> > >> > > >>>>>> answer
> > >> > > >>>>>>>>>>>>>>>>>> questions
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> > SolidFire
> > >> > > >>> driver
> > >> > > >>>>> (which
> > >> > > >>>>>>>>> is a
> > >> > > >>>>>>>>>>>>>>>>>> type of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> > >> > done, a
> > >> > > >>> new
> > >> > > >>>>> Primary
> > >> > > >>>>>>>>>>>>>>>>>> Storage shows
> > >> > > >>>>>>>>>>>>>>>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> > Offering
> > >> > > >>> that
> > >> > > >>>>>> references
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>> tag
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> > Storage.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> > >> > CloudStack
> > >> > > >>>>> volume. This
> > >> > > >>>>>>>>> leads
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> a new
> > >> > > >>>>>>>>>>>>>>>>>>>>>> row
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> > >> > CloudStack
> > >> > > >>>>> volume to a
> > >> > > >>>>>>>>> VM
> > >> > > >>>>>>>>>>>>>>>>>> (attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>> disk).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> > framework
> > >> > calling
> > >> > > >>> the
> > >> > > >>>>> plug-in
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> create a
> > >> > > >>>>>>>>>>>>>>>>>>>> new
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
> > case, a
> > >> > > >>> SAN).
> > >> > > >>>>> The
> > >> > > >>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>> also
> > >> > > >>>>>>>>>>>>>>>>>>>>>> updates
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> > applicable
> > >> > data
> > >> > > >>>>> (like the
> > >> > > >>>>>>>>> IQN of
> > >> > > >>>>>>>>>>>>>>>>>> the SAN
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> > invoked if
> > >> > the
> > >> > > >>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> volume is in
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> > attach,
> > >> > the
> > >> > > >>>>> volume
> > >> > > >>>>>>>>> will be
> > >> > > >>>>>>>>>>>>>>>>>> in the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
> > and
> > >> > the
> > >> > > >>>>> plug-in
> > >> > > >>>>>>>>> code
> > >> > > >>>>>>>>>>>>>>>>>> will not
> > >> > > >>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> > volume.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
> > is run
> > >> > and
> > >> > > >>>>> detects
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
> > the
> > >> > form
> > >> > > >>> of a
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
> > PM,
> > >> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> > sync on
> > >> > > >>>>> terminology,
> > >> > > >>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>>>>>>> in these
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> > physical
> > >> > > >>>>> allocation on
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
> > try to
> > >> > > >>> mean
> > >> > > >>>>> 'SAN
> > >> > > >>>>>>>>> volume'.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
> > end
> > >> > user
> > >> > > >>> can
> > >> > > >>>>> make in
> > >> > > >>>>>>>>>>>>>>>>>> CloudStack, I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
> > PM,
> > >> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > >> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> > very
> > >> > well
> > >> > > >>> make
> > >> > > >>>>> sense, but
> > >> > > >>>>>>>>> I'm
> > >> > > >>>>>>>>>>>>>>>>>> having a
> > >> > > >>>>>>>>>>>>>>>>>>>>>> hard
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
> > Edison
> > >> > in on
> > >> > > >>>>> this
> > >> > > >>>>>>>>> conversation
> > >> > > >>>>>>>>>>>>>>>>>> as he
> > >> > > >>>>>>>>>>>>>>>>>>>> was
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > >> > > >>> approach I
> > >> > > >>>>> took.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
> > PM,
> > >> > > >>> John
> > >> > > >>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> > combining
> > >> > two
> > >> > > >>>>> distinct
> > >> > > >>>>>>>>> concepts
> > >> > > >>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> > >> > protocols.
> > >> > > >>> In
> > >> > > >>>>> both
> > >> > > >>>>>>>>> cases, we
> > >> > > >>>>>>>>>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> > >> > system to
> > >> > > >>>>>> create/delete
> > >> > > >>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> > requires
> > >> > the
> > >> > > >>>>> volume to
> > >> > > >>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>> volume to be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> > >> > context.
> > >> > > >>> To
> > >> > > >>>>> ensure
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>>>>> are in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
> > these
> > >> > > >>>>> definitions,
> > >> > > >>>>>>>>> refers to
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>> physical
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> > >> > correct?
> > >> > > >>>>> Minimally,
> > >> > > >>>>>> we
> > >> > > >>>>>>>>>>>>> must
> > >> > > >>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>> able
> > >> > > >>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> > storage
> > >> > device
> > >> > > >>> to
> > >> > > >>>>> move bits
> > >> > > >>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>> one place
> > >> > > >>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > >> > > >>> Optionally, a
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>> may
> > >> > > >>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> > >> > Therefore,
> > >> > > >>>>> we can have
> > >> > > >>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> unmanaged
> > >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
> > SR,
> > >> > and
> > >> > > >>> we
> > >> > > >>>>> can have a
> > >> > > >>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> > CloudStack is
> > >> > > >>> capable
> > >> > > >>>>> of
> > >> > > >>>>>>>>> allocating
> > >> > > >>>>>>>>>>>>>>>>>> LUNs and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> > >> > CloudStack
> > >> > > >>> may
> > >> > > >>>>> be able
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> manage a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> > leave it
> > >> > > >>>>> unmanaged by
> > >> > > >>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> (e.g.
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
> > and
> > >> > the
> > >> > > >>>>> operator has
> > >> > > >>>>>>>>> chosen
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> > CloudStack).
> > >> > Does
> > >> > > >>> my
> > >> > > >>>>>> reasoning
> > >> > > >>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>> sense?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> > above
> > >> > are
> > >> > > >>>>> reasonable, it
> > >> > > >>>>>>>>> seems
> > >> > > >>>>>>>>>>>>>>>>>>>> appropriate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> > >> > concerns
> > >> > > >>> from
> > >> > > >>>>>>>>> StoragePoolType,
> > >> > > >>>>>>>>>>>>>>>>>> add the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> > >> > attached
> > >> > > >>> driver
> > >> > > >>>>> that
> > >> > > >>>>>>>>>>> indicates
> > >> > > >>>>>>>>>>>>>>>>>> whether
> > >> > > >>>>>>>>>>>>>>>>>>>>>> or
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
> > and
> > >> > > >>> establish
> > >> > > >>>>> a
> > >> > > >>>>>>>>> abstraction
> > >> > > >>>>>>>>>>>>>>>>>>>>>> representing a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> > separate
> > >> > that
> > >> > > >>> is
> > >> > > >>>>>>>>> associated
> > >> > > >>>>>>>>>>>>>>>>>> with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>> volume.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> > >> > hypervisor
> > >> > > >>>>> drivers can
> > >> > > >>>>>>>>>>>>> declare
> > >> > > >>>>>>>>>>>>>>>>>> which
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> > >> > encounter
> > >> > > >>> a
> > >> > > >>>>> device
> > >> > > >>>>>> managed
> > >> > > >>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> > >> > exposed
> > >> > > >>> by
> > >> > > >>>>> the driver to
> > >> > > >>>>>>>>>>>>> automate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
> > make
> > >> > > >>> sense,
> > >> > > >>>>> then we can
> > >> > > >>>>>>>>> sit
> > >> > > >>>>>>>>>>>>>>>>>> down and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> > >> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > >> > > >>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> > >> > between
> > >> > > >>> the
> > >> > > >>>>> current
> > >> > > >>>>>> iSCSI
> > >> > > >>>>>>>>>>>>> type
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> > has to
> > >> > go in
> > >> > > >>> and
> > >> > > >>>>> create
> > >> > > >>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> Primary
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> > point in
> > >> > time,
> > >> > > >>>>> the
> > >> > > >>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>> volume must
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> > >> > allocated).
> > >> > > >>>>> Future
> > >> > > >>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
> > was
> > >> > created
> > >> > > >>>>> behind the
> > >> > > >>>>>>>>>>>>> scenes.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
> > has
> > >> > to
> > >> > > >>> go in
> > >> > > >>>>> and
> > >> > > >>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>> Primary
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
> > and
> > >> > delete
> > >> > > >>>>> volumes on
> > >> > > >>>>>>>>> its
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> > enabled via
> > >> > the
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> framework). When
> > >> > > >>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> > volume
> > >> > that
> > >> > > >>> was
> > >> > > >>>>> created,
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> framework
> > >> > > >>>>>>>>>>>>>>>>>>>>>> tells
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> > >> > volume.
> > >> > > >>> After
> > >> > > >>>>> this is
> > >> > > >>>>>>>>> done,
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> > question is
> > >> > called.
> > >> > > >>>>> No
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> > >> > volume
> > >> > > >>> was
> > >> > > >>>>> just
> > >> > > >>>>>>>>> created.
> > >> > > >>>>>>>>>>>>> The
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> > created.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 3:21
> > >> > PM,
> > >> > > >>> Mike
> > >> > > >>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
> > so I
> > >> > should
> > >> > > >>>>> probably
> > >> > > >>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>> defined them
> > >> > > >>>>>>>>>>>>>>>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
> > that is
> > >> > > >>> pre-
> > >> > > >>>>> allocated
> > >> > > >>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>> an admin
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
> > a
> > >> > > >>> hypervisor
> > >> > > >>>>> data
> > >> > > >>>>>>>>> structure
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> > XenServer
> > >> > SR),
> > >> > > >>> then
> > >> > > >>>>> that
> > >> > > >>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> > CloudStack.
> > >> > Disks
> > >> > > >>> (VDI)
> > >> > > >>>>> are later
> > >> > > >>>>>>>>>>>>> placed
> > >> > > >>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> > needed. In
> > >> > > >>> these
> > >> > > >>>>> cases, the
> > >> > > >>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> > structure is
> > >> > > >>> already
> > >> > > >>>>> in place
> > >> > > >>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> simply
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> > >> > structure
> > >> > > >>> to
> > >> > > >>>>> the VM in
> > >> > > >>>>>>>>>>>>>>>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> > Storage
> > >> > that is
> > >> > > >>> not
> > >> > > >>>>>>>>>>> pre-allocated.
> > >> > > >>>>>>>>>>>>>>>>>> Instead
> > >> > > >>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
> > this
> > >> > could
> > >> > > >>> be a
> > >> > > >>>>> SAN
> > >> > > >>>>>> (not
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> volume on
> > >> > > >>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> > hypervisor
> > >> > data
> > >> > > >>>>> structure
> > >> > > >>>>>>>>> must be
> > >> > > >>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> > >> > because
> > >> > > >>> these
> > >> > > >>>>> types of
> > >> > > >>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>> have not
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
> > data
> > >> > > >>>>> structure by an
> > >> > > >>>>>>>>>>> admin.
> > >> > > >>>>>>>>>>>>>>>>>> Once
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> > >> > XenServer
> > >> > > >>> for
> > >> > > >>>>> this
> > >> > > >>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>> attaches
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
> > to the
> > >> > VM
> > >> > > >>> in
> > >> > > >>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 3:13
> > >> > PM,
> > >> > > >>>>> John Burwell
> > >> > > >>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> > >> > implementation of
> > >> > > >>> the
> > >> > > >>>>> Dynamic
> > >> > > >>>>>> type
> > >> > > >>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>> behavior
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> > which
> > >> > why I
> > >> > > >>> ask
> > >> > > >>>>> about the
> > >> > > >>>>>>>>>>>>>>>>>> difference.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> > what is
> > >> > the
> > >> > > >>>>> definition
> > >> > > >>>>>> of
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> Dynamic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
> > PM,
> > >> > > >>> Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
> > iSCSI
> > >> > type
> > >> > > >>> is
> > >> > > >>>>>> uniquely
> > >> > > >>>>>>>>>>> used
> > >> > > >>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> > Primary
> > >> > > >>> Storage
> > >> > > >>>>> that is
> > >> > > >>>>>>>>> directly
> > >> > > >>>>>>>>>>>>>>>>>> based on
> > >> > > >>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> > skip the
> > >> > > >>> step of
> > >> > > >>>>> going
> > >> > > >>>>>> to
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
> > based
> > >> > on
> > >> > > >>> that
> > >> > > >>>>> iSCSI
> > >> > > >>>>>>>>> target as
> > >> > > >>>>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
> > only
> > >> > > >>>>> supported for
> > >> > > >>>>>>>>>>>>>>>>>> XenServer.
> > >> > > >>>>>>>>>>>>>>>>>>>> For
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> > must
> > >> > first go
> > >> > > >>> to
> > >> > > >>>>> the
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>> perform
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
> > what
> > >> > RBD is.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > >> > 2:13
> > >> > > >>> PM,
> > >> > > >>>>> John
> > >> > > >>>>>> Burwell
> > >> > > >>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
> > the
> > >> > code,
> > >> > > >>> what
> > >> > > >>>>> is the
> > >> > > >>>>>>>>>>> difference
> > >> > > >>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> > Why
> > >> > isn't
> > >> > > >>> RBD
> > >> > > >>>>> considered
> > >> > > >>>>>>>>>>>>> Dynamic?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> > 3:46
> > >> > PM,
> > >> > > >>> Mike
> > >> > > >>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> > >> > storage is
> > >> > > >>>>> defined in
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > >> > > >>>>> StoragePoolType {
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
> > //
> > >> > local
> > >> > > >>>>> directory
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > NetworkFilesystem(true),
> > >> > > >>> //
> > >> > > >>>>> NFS or CIFS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> > >> > shared
> > >> > > >>> LUN,
> > >> > > >>>>> with a
> > >> > > >>>>>>>>>>> clusterfs
> > >> > > >>>>>>>>>>>>>>>>>>>> overlay
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
> > e.g.,
> > >> > ZFS
> > >> > > >>>>> Comstar
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
> > iso
> > >> > image
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> > >> > XenServer
> > >> > > >>> local
> > >> > > >>>>> LVM SR
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > SharedMountPoint(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> > >> > VMware
> > >> > > >>> VMFS
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
> > for
> > >> > > >>>>> XenServer, Storage
> > >> > > >>>>>>>>> Pool
> > >> > > >>>>>>>>>>>>>>>>>> is set
> > >> > > >>>>>>>>>>>>>>>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> > >> > XenServer
> > >> > > >>> local
> > >> > > >>>>> EXT SR
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> > >> > dynamic,
> > >> > > >>>>> zone-wide
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > StoragePoolType(boolean
> > >> > > >>>>> shared) {
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> > shared;
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> > >> > isShared() {
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > >> > 1:41
> > >> > > >>> PM,
> > >> > > >>>>> Mike
> > >> > > >>>>>>>>> Tutkowski
> > >> > > >>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> > let's say
> > >> > > >>> another
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>> company
> > >> > > >>>>>>>>>>>>>>>>>> wants
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> > leverage its
> > >> > > >>> Quality
> > >> > > >>>>> of
> > >> > > >>>>>> Service
> > >> > > >>>>>>>>>>>>>>>>>> feature. It
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > storage, as
> > >> > well.
> > >> > > >>>>> They would
> > >> > > >>>>>>>>> need
> > >> > > >>>>>>>>>>>>> only
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> > made
> > >> > the
> > >> > > >>>>> necessary
> > >> > > >>>>>>>>> changes to
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
> > plug-
> > >> > in.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > >> > at
> > >> > > >>> 1:39
> > >> > > >>>>> PM, Mike
> > >> > > >>>>>>>>>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> > imply
> > >> > the
> > >> > > >>>>> XenServer code
> > >> > > >>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> > >> > attach
> > >> > > >>> logic is
> > >> > > >>>>> now aware
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>> dynamic,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
> > an
> > >> > > >>>>> implementation of
> > >> > > >>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>> kind of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
> > to 4.2
> > >> > > >>> with
> > >> > > >>>>> Edison's
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> framework
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
> > a
> > >> > new
> > >> > > >>>>> framework that
> > >> > > >>>>>>>>>>> supported
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> > >> > dynamically.
> > >> > > >>>>> However, when I
> > >> > > >>>>>>>>>>>>>>>>>> visited with
> > >> > > >>>>>>>>>>>>>>>>>>>>>> him
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> > realized
> > >> > that
> > >> > > >>> it
> > >> > > >>>>> was not
> > >> > > >>>>>>>>>>>>>>>>>> complete. We
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> > CloudStack
> > >> > could
> > >> > > >>> do
> > >> > > >>>>> with these
> > >> > > >>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>> unless
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> > >> > recognize
> > >> > > >>> this
> > >> > > >>>>> new type of
> > >> > > >>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> > >> > structure.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > >> > at
> > >> > > >>> 1:28
> > >> > > >>>>> PM, John
> > >> > > >>>>>>>>> Burwell
> > >> > > >>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
> > odd
> > >> > to me
> > >> > > >>>>> that any
> > >> > > >>>>>>>>>>> operation
> > >> > > >>>>>>>>>>>>>>>>>> in the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
> > care
> > >> > > >>> about
> > >> > > >>>>> details. I
> > >> > > >>>>>>>>> expect
> > >> > > >>>>>>>>>>>>>>>>>> to see
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
> > of
> > >> > > >>> operations
> > >> > > >>>>> that can be
> > >> > > >>>>>>>>>>>>>>>>>>>> composed/driven
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > implementations to
> > >> > > >>>>> allocate
> > >> > > >>>>>> space/create
> > >> > > >>>>>>>>>>>>>>>>>> structures
> > >> > > >>>>>>>>>>>>>>>>>>>>>> per
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
> > this
> > >> > > >>>>> dependency, we are
> > >> > > >>>>>>>>> going
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> end
> > >> > > >>>>>>>>>>>>>>>>>>>>>> with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
> > will
> > >> > make
> > >> > > >>> the
> > >> > > >>>>> system
> > >> > > >>>>>>>>>>>>> increasingly
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
> > I
> > >> > > >>>>> understanding that the
> > >> > > >>>>>>>>> Xen
> > >> > > >>>>>>>>>>>>>>>>>> specific
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > >> > > >>>>> CitrixResourceBase
> > >> > > >>>>>> class?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > >> > 2013 at
> > >> > > >>> 3:12
> > >> > > >>>>> PM, Mike
> > >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
> > this
> > >> > in a
> > >> > > >>> bit
> > >> > > >>>>> more
> > >> > > >>>>>>>>> detail:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
> > and
> > >> > aside
> > >> > > >>> from
> > >> > > >>>>> one setup
> > >> > > >>>>>>>>>>>>> method
> > >> > > >>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
> > a
> > >> > > >>> volume on
> > >> > > >>>>> the
> > >> > > >>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> system,
> > >> > > >>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>> go
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
> > data
> > >> > > >>> structure
> > >> > > >>>>> to make
> > >> > > >>>>>> use
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > >> > > >>> XenServer
> > >> > > >>>>> or a
> > >> > > >>>>>> datastore
> > >> > > >>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>> ESX(i)).
> > >> > > >>>>>>>>>>>>>>>>>>>>>> VMs
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> > >> > storage
> > >> > > >>>>> system's volume.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
> > new
> > >> > > >>> storage
> > >> > > >>>>> framework,
> > >> > > >>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> need
> > >> > > >>>>>>>>>>>>>>>>>>>> no
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
> > easily
> > >> > > >>> create
> > >> > > >>>>> a 1:1
> > >> > > >>>>>>>>>>>>> relationship
> > >> > > >>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
> > the
> > >> > VM's
> > >> > > >>> data
> > >> > > >>>>> disk
> > >> > > >>>>>>>>> (necessary
> > >> > > >>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
> > write
> > >> > a
> > >> > > >>> plug-
> > >> > > >>>>> in that is
> > >> > > >>>>>>>>> called
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> > >> > needed.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
> > that
> > >> > the
> > >> > > >>>>> storage
> > >> > > >>>>>> framework
> > >> > > >>>>>>>>> did
> > >> > > >>>>>>>>>>>>>>>>>> not
> > >> > > >>>>>>>>>>>>>>>>>>>>>> address
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > >> > > >>> hypervisor-
> > >> > > >>>>> specific data
> > >> > > >>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>> when
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > attach/detach.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
> > the
> > >> > case,
> > >> > > >>> I've
> > >> > > >>>>> been
> > >> > > >>>>>>>>> enhancing
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> to do
> > >> > > >>>>>>>>>>>>>>>>>>>> so.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
> > and
> > >> > > >>>>> submitted. I've got
> > >> > > >>>>>>>>> ESX(i)
> > >> > > >>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> my
> > >> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
> > the
> > >> > 4.2
> > >> > > >>>>> freeze date.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> > help a
> > >> > bit? :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> > 3,
> > >> > 2013 at
> > >> > > >>>>> 1:03 PM, Mike
> > >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> > >> > plug-in -
> > >> > > >>> by
> > >> > > >>>>> itself - is
> > >> > > >>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
> > with
> > >> > the
> > >> > > >>>>> volume-attach
> > >> > > >>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>> (in the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> > calls
> > >> > into
> > >> > > >>> the
> > >> > > >>>>> plug-in to
> > >> > > >>>>>>>>> have
> > >> > > >>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>> create a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
> > time
> > >> > > >>> comes to
> > >> > > >>>>> attach the
> > >> > > >>>>>>>>> volume
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
> > smart
> > >> > > >>> enough
> > >> > > >>>>> to recognize
> > >> > > >>>>>>>>> it's
> > >> > > >>>>>>>>>>>>>>>>>> being
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> > >> > volume
> > >> > > >>> has
> > >> > > >>>>> just been
> > >> > > >>>>>>>>> created)
> > >> > > >>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>> create,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> > (for
> > >> > > >>>>> XenServer) or a
> > >> > > >>>>>>>>> datastore
> > >> > > >>>>>>>>>>>>>>>>>> (for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
> > was
> > >> > just
> > >> > > >>>>> created.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> > >> > spending
> > >> > > >>> most
> > >> > > >>>>> of my time
> > >> > > >>>>>>>>>>>>> recently
> > >> > > >>>>>>>>>>>>>>>>>> making
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> > >> > code.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> > clear it
> > >> > up?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> > Jun 3,
> > >> > 2013
> > >> > > >>> at
> > >> > > >>>>> 12:48 PM,
> > >> > > >>>>>> John
> > >> > > >>>>>>>>>>>>>>>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> > explain
> > >> > > >>> why
> > >> > > >>>>> the the storage
> > >> > > >>>>>>>>>>>>> driver
> > >> > > >>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> > 2013,
> > >> > at
> > >> > > >>> 1:21
> > >> > > >>>>> PM, Mike
> > >> > > >>>>>>>>>>>>> Tutkowski
> > >> > > >>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> > >> > ultimately I
> > >> > > >>>>> would like to
> > >> > > >>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>> all
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
> > I
> > >> > think
> > >> > > >>> I'm
> > >> > > >>>>> just out of
> > >> > > >>>>>>>>> time
> > >> > > >>>>>>>>>>>>>>>>>> for 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> > now this
> > >> > > >>> plug-in
> > >> > > >>>>> supports
> > >> > > >>>>>>>>>>>>> XenServer.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> > to 4.2
> > >> > > >>> feature
> > >> > > >>>>> freeze, I
> > >> > > >>>>>>>>> have
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> working
> > >> > > >>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
> > to
> > >> > be
> > >> > > >>> clear,
> > >> > > >>>>> this is
> > >> > > >>>>>> all
> > >> > > >>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> regards
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > Compute
> > >> > > >>>>> Offerings post
> > >> > > >>>>>> 4.2.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> > Jun 3,
> > >> > > >>> 2013
> > >> > > >>>>> at 11:14 AM,
> > >> > > >>>>>>>>> Kelcey
> > >> > > >>>>>>>>>>>>>>>>>> Jamison
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > kelcey@bbits.ca
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
> > any
> > >> > plan
> > >> > > >>> on
> > >> > > >>>>> supporting
> > >> > > >>>>>>>>> KVM in
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> patch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> > Original
> > >> > > >>>>> Message -----
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
> > "Mike
> > >> > > >>>>> Tutkowski" <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > >> > > >>>>> dev@cloudstack.apache.org
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> > >> > Monday,
> > >> > > >>> June
> > >> > > >>>>> 3, 2013
> > >> > > >>>>>>>>> 10:12:32 AM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
> > Re:
> > >> > > >>> [MERGE]
> > >> > > >>>>>>>>>>> disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>> MASTER
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
> > on
> > >> > > >>> merging
> > >> > > >>>>> Wei's feature
> > >> > > >>>>>>>>>>> first,
> > >> > > >>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mine.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
> > feature
> > >> > is
> > >> > > >>> for
> > >> > > >>>>> KVM only,
> > >> > > >>>>>>>>> then
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> is a
> > >> > > >>>>>>>>>>>>>>>>>>>> non
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
> > 4.2.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> > Jun
> > >> > 3,
> > >> > > >>> 2013
> > >> > > >>>>> at 8:55 AM,
> > >> > > >>>>>> Wei
> > >> > > >>>>>>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > ustcweizhou@gmail.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> > the
> > >> > billing,
> > >> > > >>> as
> > >> > > >>>>> no one works
> > >> > > >>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>> billing
> > >> > > >>>>>>>>>>>>>>>>>>>> now,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > billing
> > >> > by
> > >> > > >>>>> themselves. They
> > >> > > >>>>>>>>> can
> > >> > > >>>>>>>>>>>>> get
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > disk_offering of
> > >> > > >>> a
> > >> > > >>>>> VMs and
> > >> > > >>>>>> volumes
> > >> > > >>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
> > user
> > >> > the
> > >> > > >>>>> exact
> > >> > > >>>>>> limitation
> > >> > > >>>>>>>>>>>>> value
> > >> > > >>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > >> > limitation
> > >> > > >>> for
> > >> > > >>>>> nics as
> > >> > > >>>>>> well.
> > >> > > >>>>>>>>> I
> > >> > > >>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>> work
> > >> > > >>>>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> > part of
> > >> > I/O
> > >> > > >>>>> throttling?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
> > my
> > >> > > >>>>> misunstand the second
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
> > with
> > >> > > >>> what
> > >> > > >>>>> you said about
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>> two
> > >> > > >>>>>>>>>>>>>>>>>>>> features.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/6/3
> > >> > John
> > >> > > >>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Jun 3,
> > >> > 2013,
> > >> > > >>> at
> > >> > > >>>>> 2:13 AM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> > John,
> > >> > Mike
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > hope
> > >> > Mike's
> > >> > > >>>>> aswer helps you.
> > >> > > >>>>>>>>> I am
> > >> > > >>>>>>>>>>>>>>>>>> trying
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> > >> > think
> > >> > > >>> billing
> > >> > > >>>>> should
> > >> > > >>>>>>>>> depend
> > >> > > >>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>> IO
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > limitation.
> > >> > > >>> Please
> > >> > > >>>>> review
> > >> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
> > >> > > >>>>>>>>>>>>>>>>>>>> you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_io_stat
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> > the
> > >> > IO
> > >> > > >>>>> statistics
> > >> > > >>>>>> including
> > >> > > >>>>>>>>>>>>>>>>>> bytes/iops
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > individual
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > virtual
> > >> > > >>> machine.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Going by
> > >> > the
> > >> > > >>> AWS
> > >> > > >>>>> model,
> > >> > > >>>>>> customers
> > >> > > >>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>> billed
> > >> > > >>>>>>>>>>>>>>>>>>>>>> more
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > provisioned
> > >> > > >>> IOPS,
> > >> > > >>>>> as well as,
> > >> > > >>>>>> for
> > >> > > >>>>>>>>>>>>> those
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://aws.amazon.com/ebs/).
> > >> > > >>>>>> I
> > >> > > >>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>> imagine
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > option to
> > >> > > >>> employ
> > >> > > >>>>> similar cost
> > >> > > >>>>>>>>>>> models.
> > >> > > >>>>>>>>>>>>>>>>>> Could
> > >> > > >>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > billing
> > >> > model
> > >> > > >>> in
> > >> > > >>>>> the current
> > >> > > >>>>>>>>> patch?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > Do
> > >> > you
> > >> > > >>> mean
> > >> > > >>>>> IOPS runtime
> > >> > > >>>>>>>>>>> change?
> > >> > > >>>>>>>>>>>>>>>>>> KVM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > limitation for
> > >> > > >>> a
> > >> > > >>>>> running
> > >> > > >>>>>> virtual
> > >> > > >>>>>>>>>>>>>>>>>> machine
> > >> > > >>>>>>>>>>>>>>>>>>>>>> through
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > However,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > CloudStack
> > >> > > >>> does
> > >> > > >>>>> not support
> > >> > > >>>>>>>>>>> changing
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > offering
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > (computer
> > >> > > >>>>> offering or disk
> > >> > > >>>>>>>>>>>>> offering).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > meant at
> > >> > the
> > >> > > >>>>> Java interface
> > >> > > >>>>>>>>> level.
> > >> > > >>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>> apologize
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > more
> > >> > > >>>>> generalize allocation
> > >> > > >>>>>>>>>>>>>>>>>> algorithms
> > >> > > >>>>>>>>>>>>>>>>>>>> with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > describe
> > >> > the
> > >> > > >>>>> service guarantees
> > >> > > >>>>>>>>>>>>>>>>>> provided by a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > It is a
> > >> > > >>> good
> > >> > > >>>>> question.
> > >> > > >>>>>>>>> Maybe it
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>> better
> > >> > > >>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> disk_io_throttling as Mike
> > >> > > >>>>>>>>> needs to
> > >> > > >>>>>>>>>>>>>>>>>> consider
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > type, I
> > >> > think.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> > >> > expand
> > >> > > >>> on
> > >> > > >>>>> my thoughts
> > >> > > >>>>>> in a
> > >> > > >>>>>>>>>>>>> later
> > >> > > >>>>>>>>>>>>>>>>>>>>>> response
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > touch
> > >> > points
> > >> > > >>>>> between these two
> > >> > > >>>>>>>>>>>>>>>>>> features. I
> > >> > > >>>>>>>>>>>>>>>>>>>>>> think
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > need
> > >> > to be
> > >> > > >>>>> merged before
> > >> > > >>>>>>>>>>>>>>>>>> SolidFire, but
> > >> > > >>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > coordination
> > >> > > >>>>> between the
> > >> > > >>>>>> branches
> > >> > > >>>>>>>>>>>>>>>>>> (possibly
> > >> > > >>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> disk_io_throttling)
> > >> > > >>>>> to
> > >> > > >>>>>>>>> coordinate on
> > >> > > >>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>> issue.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/6/3
> > >> > > >>> John
> > >> > > >>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > The
> > >> > things I
> > >> > > >>>>> want to
> > >> > > >>>>>> understand
> > >> > > >>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
> > Is
> > >> > there
> > >> > > >>>>> value in
> > >> > > >>>>>> capturing
> > >> > > >>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>> policies
> > >> > > >>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > common
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > data
> > >> > model
> > >> > > >>> (e.g.
> > >> > > >>>>> for
> > >> > > >>>>>>>>> billing/usage
> > >> > > >>>>>>>>>>>>>>>>>>>> purposes,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> > >> > Should
> > >> > > >>> there
> > >> > > >>>>> be a common
> > >> > > >>>>>>>>>>>>>>>>>> interface model
> > >> > > >>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > provisioning
> > >> > > >>> at
> > >> > > >>>>> runtime?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
> > How
> > >> > are
> > >> > > >>>>> conflicting
> > >> > > >>>>>>>>> provisioned
> > >> > > >>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > hypervisor
> > >> > > >>> and
> > >> > > >>>>> storage device
> > >> > > >>>>>>>>>>>>>>>>>> reconciled?
> > >> > > >>>>>>>>>>>>>>>>>>>> In
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > where
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > lead
> > >> > to
> > >> > > >>>>> believe (and
> > >> > > >>>>>> billed)
> > >> > > >>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>> more
> > >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > storage
> > >> > > >>> device
> > >> > > >>>>> has been
> > >> > > >>>>>>>>> configured
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> > a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > consistent
> > >> > > >>>>> configuration
> > >> > > >>>>>>>>> between a
> > >> > > >>>>>>>>>>>>>>>>>> VM and a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > time,
> > >> > but a
> > >> > > >>>>> later
> > >> > > >>>>>> modification
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> inconsistency.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Jun
> > >> > 2,
> > >> > > >>> 2013,
> > >> > > >>>>> at 8:38 PM,
> > >> > > >>>>>>>>> Mike
> > >> > > >>>>>>>>>>>>>>>>>> Tutkowski
> > >> > > >>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> > John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > believe
> > >> > > >>> Wei's
> > >> > > >>>>> feature deals
> > >> > > >>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>> controlling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > the
> > >> > > >>> hypervisor
> > >> > > >>>>> side.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > My
> > >> > feature
> > >> > > >>> is
> > >> > > >>>>> focused on
> > >> > > >>>>>>>>>>>>> controlling
> > >> > > >>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > hope
> > >> > that
> > >> > > >>>>> helps. :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Sun,
> > >> > Jun
> > >> > > >>> 2,
> > >> > > >>>>> 2013 at 6:35
> > >> > > >>>>>> PM,
> > >> > > >>>>>>>>>>>>> John
> > >> > > >>>>>>>>>>>>>>>>>>>> Burwell
> > >> > > >>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > jburwell@basho.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > My
> > >> > > >>> opinion is
> > >> > > >>>>> that no
> > >> > > >>>>>> features
> > >> > > >>>>>>>>>>>>>>>>>> should be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > functional
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > issues
> > >> > > >>> have
> > >> > > >>>>> been resolved
> > >> > > >>>>>> and
> > >> > > >>>>>>>>> it
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>> ready
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > total
> > >> > Ops
> > >> > > >>> vs
> > >> > > >>>>> discrete
> > >> > > >>>>>>>>> read/write
> > >> > > >>>>>>>>>>>>>>>>>> ops issue
> > >> > > >>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> > >> > reviewed
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > I
> > >> > > >>> don't
> > >> > > >>>>> think this
> > >> > > >>>>>>>>> criteria
> > >> > > >>>>>>>>>>>>>>>>>> has been
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Also,
> > >> > how
> > >> > > >>>>> does this work
> > >> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> > >> > > >>>>>>>>>>>>>>>>>> )?
> > >> > > >>>>>>>>>>>>>>>>>>>> As I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > also
> > >> > > >>> involves
> > >> > > >>>>> provisioned
> > >> > > >>>>>>>>> IOPS. I
> > >> > > >>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>> like
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > scenario
> > >> > > >>>>> where provisioned
> > >> > > >>>>>>>>> IOPS
> > >> > > >>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> KVM and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > unnecessarily
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> incompatible.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > Jun 1,
> > >> > > >>> 2013,
> > >> > > >>>>> at 6:47 AM,
> > >> > > >>>>>>>>> Wei
> > >> > > >>>>>>>>>>>>>>>>>> ZHOU <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > ustcweizhou@gmail.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Sure. I
> > >> > will
> > >> > > >>>>> change it next
> > >> > > >>>>>>>>> week.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > 2013/6/1
> > >> > > >>>>> Wido den Hollander
> > >> > > >>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 06/01/2013 08:24 AM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Exactly. I
> > >> > > >>> have
> > >> > > >>>>> pushed the
> > >> > > >>>>>>>>>>>>> features
> > >> > > >>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > anyone
> > >> > > >>>>> object thems for
> > >> > > >>>>>>>>>>>>> technical
> > >> > > >>>>>>>>>>>>>>>>>>>> reason
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > them.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > For
> > >> > the
> > >> > > >>> sake
> > >> > > >>>>> of clarity I
> > >> > > >>>>>> just
> > >> > > >>>>>>>>>>>>> want
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mention
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > change
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > the
> > >> > total
> > >> > > >>> IOps
> > >> > > >>>>> to R/W IOps
> > >> > > >>>>>>>>> asap
> > >> > > >>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>> that we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > only
> > >> > total
> > >> > > >>> IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > You
> > >> > laid
> > >> > > >>> the
> > >> > > >>>>> groundwork for
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>> I/O
> > >> > > >>>>>>>>>>>>>>>>>>>>>> throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > should
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > however
> > >> > > >>>>> prevent that we
> > >> > > >>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>> legacy from
> > >> > > >>>>>>>>>>>>>>>>>>>>>> day
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > 2013/5/31
> > >> > > >>>>> Wido den
> > >> > > >>>>>> Hollander <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 05/31/2013 03:59 PM, John
> > >> > > >>>>>>>>>>>>> Burwell
> > >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > +1 --
> > >> > this
> > >> > > >>>>> enhancement must
> > >> > > >>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> discretely
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > don't
> > >> > see
> > >> > > >>> how
> > >> > > >>>>> it could be
> > >> > > >>>>>>>>> fixed
> > >> > > >>>>>>>>>>>>>>>>>> later
> > >> > > >>>>>>>>>>>>>>>>>>>>>> because
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > correctly
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > split
> > >> > total
> > >> > > >>> IOPS
> > >> > > >>>>> into read
> > >> > > >>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> write.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > total
> > >> > > >>>>> unless/until we
> > >> > > >>>>>> decided
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> break
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > What
> > >> > Wei
> > >> > > >>>>> meant was merging
> > >> > > >>>>>> it
> > >> > > >>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>> master
> > >> > > >>>>>>>>>>>>>>>>>>>>>> now
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 4.2
> > >> > branch
> > >> > > >>>>> and add Read /
> > >> > > >>>>>>>>> Write
> > >> > > >>>>>>>>>>>>> IOps
> > >> > > >>>>>>>>>>>>>>>>>>>> before
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > will be
> > >> > > >>>>> released with Read
> > >> > > >>>>>> and
> > >> > > >>>>>>>>>>>>> Write
> > >> > > >>>>>>>>>>>>>>>>>>>> instead
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > This is
> > >> > to
> > >> > > >>>>> make the May 31st
> > >> > > >>>>>>>>>>>>> feature
> > >> > > >>>>>>>>>>>>>>>>>>>> freeze
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > moves
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (see
> > >> > other
> > >> > > >>>>> threads) then it
> > >> > > >>>>>>>>> won't
> > >> > > >>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>> necessary
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > also
> > >> > > >>>>> completely agree that
> > >> > > >>>>>>>>>>> there
> > >> > > >>>>>>>>>>>>>>>>>> is no
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk
> > >> > I/O.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > May 31,
> > >> > > >>>>> 2013, at 9:51 AM,
> > >> > > >>>>>>>>> Wido
> > >> > > >>>>>>>>>>>>>>>>>> den
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wido@widodh.nl
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 05/31/2013 03:13 PM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi
> > >> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Thanks.
> > >> > > >>> Good
> > >> > > >>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > thought
> > >> > > >>>>> about at the
> > >> > > >>>>>>>>>>> beginning.
> > >> > > >>>>>>>>>>>>>>>>>>>> Finally I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > difference
> > >> > > >>> of
> > >> > > >>>>> read and write
> > >> > > >>>>>>>>>>>>> mainly
> > >> > > >>>>>>>>>>>>>>>>>>>> because
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > not
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > care
> > >> > the
> > >> > > >>>>> difference of sent
> > >> > > >>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> received
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > That
> > >> > > >>>>> reasoning seems odd.
> > >> > > >>>>>>>>>>>>>>>>>> Networking and
> > >> > > >>>>>>>>>>>>>>>>>>>>>> disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > different.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Disk
> > >> > I/O is
> > >> > > >>>>> much more
> > >> > > >>>>>>>>> expensive
> > >> > > >>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> most
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > bandwith.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> Implementing
> > >> > > >>>>> it will be some
> > >> > > >>>>>>>>>>>>>>>>>> copy-paste
> > >> > > >>>>>>>>>>>>>>>>>>>>>> work.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> implemented
> > >> > > >>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > few
> > >> > days.
> > >> > > >>> For
> > >> > > >>>>> the deadline
> > >> > > >>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>> feature
> > >> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > after
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > that ,
> > >> > if
> > >> > > >>>>> needed.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> > >> > think it's
> > >> > > >>> a
> > >> > > >>>>> feature we
> > >> > > >>>>>>>>> can't
> > >> > > >>>>>>>>>>>>>>>>>> miss. But
> > >> > > >>>>>>>>>>>>>>>>>>>>>> if
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > window
> > >> > > >>> we
> > >> > > >>>>> have to make sure
> > >> > > >>>>>> we
> > >> > > >>>>>>>>>>>>> don't
> > >> > > >>>>>>>>>>>>>>>>>>>> release
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > fix
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > in
> > >> > 4.3,
> > >> > > >>> that
> > >> > > >>>>> will confuse
> > >> > > >>>>>>>>>>>>> users.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > 2013/5/31
> > >> > > >>>>> Wido den
> > >> > > >>>>>> Hollander <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 05/30/2013 06:03 PM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > would
> > >> > > >>> like to
> > >> > > >>>>> merge
> > >> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > nobody
> > >> > > >>>>> object, I will
> > >> > > >>>>>> merge
> > >> > > >>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>> master
> > >> > > >>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > The
> > >> > > >>> purpose
> > >> > > >>>>> is :
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Virtual
> > >> > > >>>>> machines are running
> > >> > > >>>>>>>>> on
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> same
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > storage or
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > share
> > >> > > >>> strage).
> > >> > > >>>>> Because of
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>> rate
> > >> > > >>>>>>>>>>>>>>>>>>>>>> limitation
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > iops),
> > >> > if
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > one
> > >> > VM
> > >> > > >>> has
> > >> > > >>>>> large disk
> > >> > > >>>>>>>>> operation,
> > >> > > >>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> may
> > >> > > >>>>>>>>>>>>>>>>>>>>>> affect
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > performance
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > other
> > >> > VMs
> > >> > > >>>>> running on the
> > >> > > >>>>>> same
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>> device.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> > is
> > >> > > >>> neccesary
> > >> > > >>>>> to set the
> > >> > > >>>>>>>>>>> maximum
> > >> > > >>>>>>>>>>>>>>>>>> rate
> > >> > > >>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > VMs.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Looking at
> > >> > > >>> the
> > >> > > >>>>> code I see
> > >> > > >>>>>> you
> > >> > > >>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>> no
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Qemu
> > >> > and
> > >> > > >>>>> libvirt support
> > >> > > >>>>>>>>> setting
> > >> > > >>>>>>>>>>>>>>>>>> both a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps
> > >> > > >>> which
> > >> > > >>>>> could benefit a
> > >> > > >>>>>>>>> lot of
> > >> > > >>>>>>>>>>>>>>>>>> users.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > It's
> > >> > also
> > >> > > >>>>> strange, in the
> > >> > > >>>>>>>>> polling
> > >> > > >>>>>>>>>>>>>>>>>> side you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps,
> > >> > but
> > >> > > >>> on
> > >> > > >>>>> the throttling
> > >> > > >>>>>>>>> side
> > >> > > >>>>>>>>>>>>>>>>>> you only
> > >> > > >>>>>>>>>>>>>>>>>>>> go
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > IOps
> > >> > > >>> are
> > >> > > >>>>> usually much
> > >> > > >>>>>>>>> more
> > >> > > >>>>>>>>>>>>>>>>>> expensive
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > seems
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > like a
> > >> > valid
> > >> > > >>>>> use-case where
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>> admin
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps
> > >> > vs
> > >> > > >>> Read
> > >> > > >>>>> IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Since
> > >> > this
> > >> > > >>> only
> > >> > > >>>>> supports KVM
> > >> > > >>>>>>>>> at
> > >> > > >>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>> point I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > great
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > value
> > >> > to at
> > >> > > >>>>> least have the
> > >> > > >>>>>>>>>>>>>>>>>> mechanism in
> > >> > > >>>>>>>>>>>>>>>>>>>>>> place
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> implementing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > this
> > >> > later
> > >> > > >>>>> would be a lot of
> > >> > > >>>>>>>>>>> work.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > a
> > >> > > >>> hypervisor
> > >> > > >>>>> doesn't
> > >> > > >>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>> setting
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > write
> > >> > you
> > >> > > >>> can
> > >> > > >>>>> always sum
> > >> > > >>>>>> both
> > >> > > >>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>> and set
> > >> > > >>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Can
> > >> > you
> > >> > > >>>>> explain why you
> > >> > > >>>>>>>>>>>>> implemented
> > >> > > >>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > The
> > >> > > >>> feature
> > >> > > >>>>> includes:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (1) set
> > >> > the
> > >> > > >>>>> maximum rate of
> > >> > > >>>>>>>>> VMs
> > >> > > >>>>>>>>>>>>> (in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> configuration)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (2)
> > >> > change
> > >> > > >>> the
> > >> > > >>>>> maximum rate
> > >> > > >>>>>> of
> > >> > > >>>>>>>>>>> VMs
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (3)
> > >> > limit
> > >> > > >>> the
> > >> > > >>>>> disk rate
> > >> > > >>>>>> (total
> > >> > > >>>>>>>>>>> bps
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>> iops)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > JIRA
> > >> > ticket:
> > >> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> issues.apache.org/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> jira/browse/CLOUDSTACK-1192<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1192>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>> 1192<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > >> > 1192
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1192<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > https://issues.apache.org/jira/browse/CLOUDSTACK-
> > >> > 1192>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > FS (I
> > >> > will
> > >> > > >>>>> update later) :
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>
> > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>> VM+Disk+IO+Throttling<https://
> <https://%0b>> >> <https://%0b/>> > >>>>>>>>>>>>> ****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cwiki.apache.org/confluence/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >> > > >>>>> rottling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Merge
> > >> > > >>> check
> > >> > > >>>>> list :-
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > Did
> > >> > you
> > >> > > >>>>> check the branch's
> > >> > > >>>>>>>>> RAT
> > >> > > >>>>>>>>>>>>>>>>>> execution
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Yes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > Are
> > >> > > >>> there
> > >> > > >>>>> new dependencies
> > >> > > >>>>>>>>>>>>>>>>>> introduced?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > No
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > >> > What
> > >> > > >>>>> automated testing
> > >> > > >>>>>> (unit
> > >> > > >>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > new
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > feature?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Unit
> > >> > tests
> > >> > > >>> are
> > >> > > >>>>> added.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > >> > What
> > >> > > >>>>> testing has been done
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> check for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (1) set
> > >> > the
> > >> > > >>>>> bytes rate and
> > >> > > >>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>> rate on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (2)
> > >> > VM
> > >> > > >>>>> operations, including
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > deploy,
> > >> > > >>> stop,
> > >> > > >>>>> start, reboot,
> > >> > > >>>>>>>>>>>>>>>>>> destroy,
> > >> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (3)
> > >> > > >>> Volume
> > >> > > >>>>> operations,
> > >> > > >>>>>>>>> including
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Attach,
> > >> > > >>>>> Detach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > To
> > >> > review
> > >> > > >>> the
> > >> > > >>>>> code, you can
> > >> > > >>>>>>>>> try
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > git diff
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Best
> > >> > > >>> regards,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > [1]
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>
> > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>> VM+Disk+IO+Throttling<https://
> <https://%0b>> >> <https://%0b/>> > >>>>>>>>>>>>> ****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cwiki.apache.org/confluence/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >> > > >>>>> rottling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > [2]
> > >> > > >>>>>>>>> refs/heads/disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > [3]
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> > 1301
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1301
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/****jira/browse/CLOUDSTACK-
> > >> > > >>> 1301<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1301>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>> 1301<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > >> > 1301
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1301<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > https://issues.apache.org/jira/browse/CLOUDSTACK-
> > >> > 1301>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > issues.apache.org/****jira/**browse/CLOUDSTACK-
> > >> > > >>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>>>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > **<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>>>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > >> > 2071
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >> > > >>>>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 2071>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <h**ttps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > https://issues.apache.org/jira/browse/CLOUDSTACK-
> > >> > 2071>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> (**CLOUDSTACK-1301
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > >> > VM
> > >> > > >>> Disk
> > >> > > >>>>> I/O
> > >> > > >>>>>> Throttling)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > *Mike
> > >> > > >>>>> Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > *Senior
> > >> > > >>>>> CloudStack Developer,
> > >> > > >>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >> > > >>> 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Advancing
> > >> > > >>> the
> > >> > > >>>>> way the world
> > >> > > >>>>>>>>> uses
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >> > > >>> Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > > >>> CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Advancing
> > >> > the
> > >> > > >>> way
> > >> > > >>>>> the world uses
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > > >>> CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Advancing the
> > >> > > >>> way
> > >> > > >>>>> the world uses
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > >> > way
> > >> > > >>> the
> > >> > > >>>>> world uses the
> > >> > > >>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>> SolidFire
> > >> > > >>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > >> > way
> > >> > > >>> the
> > >> > > >>>>> world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > >> > > >>>>> Developer, SolidFire
> > >> > > >>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > >> > the
> > >> > > >>>>> world uses the
> > >> > > >>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > >> > > >>> Developer,
> > >> > > >>>>> SolidFire
> > >> > > >>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > >> > the
> > >> > > >>> world
> > >> > > >>>>> uses the
> > >> > > >>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > >> > > >>> Developer,
> > >> > > >>>>> SolidFire
> > >> > > >>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way the
> > >> > > >>> world
> > >> > > >>>>> uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >> > Developer,
> > >> > > >>>>> SolidFire
> > >> > > >>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > >> > world
> > >> > > >>> uses
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >> > Developer,
> > >> > > >>>>> SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > >> > world
> > >> > > >>> uses
> > >> > > >>>>> the cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > >> > > >>>>> SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > >> > uses
> > >> > > >>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > >> > > >>> SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > >> > uses
> > >> > > >>> the
> > >> > > >>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > >> > > >>> SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > >> > the
> > >> > > >>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >> > > >>> SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > >> > the
> > >> > > >>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >> > SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > >> > the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >> > SolidFire
> > >> > > >>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > >> > > >>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > >> > > >>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > >> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > >> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> --
> > >> > > >>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> --
> > >> > > >>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>> *(tm)*
> > >> > > >>>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> --
> > >> > > >>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>> o: 303.746.7302
> > >> > > >>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>> *(tm)*
> > >> > > >>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>>>
> > >> > > >>>>> --
> > >> > > >>>>> *Mike Tutkowski*
> > >> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>> o: 303.746.7302
> > >> > > >>>>> Advancing the way the world uses the
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>> *(tm)*
> > >> > > >>>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> --
> > >> > > >>> *Mike Tutkowski*
> > >> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>> e: mike.tutkowski@solidfire.com
> > >> > > >>> o: 303.746.7302
> > >> > > >>> Advancing the way the world uses the
> > >> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>> *(tm)*
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > *Mike Tutkowski*
> > >> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > > e: mike.tutkowski@solidfire.com
> > >> > > > o: 303.746.7302
> > >> > > > Advancing the way the world uses the
> > >> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > > *(tm)*
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > *Mike Tutkowski*
> > >> > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > e: mike.tutkowski@solidfire.com
> > >> > o: 303.746.7302
> > >> > Advancing the way the world uses the
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > *(tm)*****
> > >>
> > >>
> > >>
> > >> ****
> > >>
> > >> ** **
> > >>
> > >> --
> > >> *Mike Tutkowski*****
> > >>
> > >> *Senior CloudStack Developer, SolidFire Inc.*****
> > >>
> > >> e: mike.tutkowski@solidfire.com****
> > >>
> > >> o: 303.746.7302****
> > >>
> > >> Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> *(tm)*****
> > >>
> > >>
> > >>
> > >> ****
> > >>
> > >> ** **
> > >>
> > >> --
> > >> *Mike Tutkowski*****
> > >>
> > >> *Senior CloudStack Developer, SolidFire Inc.*****
> > >>
> > >> e: mike.tutkowski@solidfire.com****
> > >>
> > >> o: 303.746.7302****
> > >>
> > >> Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> *(tm)*****
> > >>
> > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *(tm)*
> > >
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*****
>
>
>
> ****
>
> ** **
>
> --
> *Mike Tutkowski*****
>
> *Senior CloudStack Developer, SolidFire Inc.*****
>
> e: mike.tutkowski@solidfire.com****
>
> o: 303.746.7302****
>
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*****
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
I haven't review your patch yet, but I can show you what am doing on the object_store branch:
VolumeManagerImpl-> sendAttachVolumeCommand:
DataTO volTO = volFactory.getVolume(volume.getId()).getTO();
1557 DiskTO disk = new DiskTO(volTO, deviceId, volume.getVolumeType());
1558 AttachCommand cmd = new AttachCommand(disk, vm.getInstanceName());
getTO is a method on each storage driver, so each storage driver can encode volume differently. In solidfire case, the volumeTO may contains the following:
IQN of the volume, CHAP info, isManaged, and a dataStoreTO which represent your solidfire storage box.
DataStoreTO may contain the following information: the mgt ip of your storage box, credential of your storage box etc, etc.
At the resource side, when execute(AttachCommand cmd):
DataTO vol = cmd.getdisk().getData();
If (vol.isManaged()) {
//need to create a sr based on vol.getpath(which is the IQN of the volume), then create a vdi on the SR, then
Vdi = a new vdi created from SR
} else {
//find out the vdi from vol.getPath(which is a uuid of the vdi)
Vid = VDI.findbyUUID(vol.getPath())
}
//then attach the vdi to the VM.
Above code logic can be storage independent, as long as, the information given by DataTo is general enough for all the storage vendor. If the storage vendor has its own special way to access storage, then they need to add some functions in execute(AttachCommand cmd).
Does it make sense for you?
From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
Sent: Monday, June 10, 2013 12:17 PM
To: dev@cloudstack.apache.org; John Burwell; Edison Su
Subject: Re: [MERGE] disk_io_throttling to MASTER
Well, that's sort of how I did it in the code that I originally submitted (except I labelled the storage as Dynamic and that was how the hypervisor knew what to do). :)
On Mon, Jun 10, 2013 at 12:58 PM, Edison Su <Ed...@citrix.com>> wrote:
I don't understand why we need to add a "managed" in the UI and in "storage_pool" table?
AFAIK, what you guys trying to do is to enable attachVolume/DettachVolume if the volume is created on Solidfire, right?
If the volume is created on Solidfire already, by createAsync, then why we can't just send out all the information about the volume(it's volumeTO which will be returned by each storage driver) to hypervisor, then at the hypervisor side, check the volumeTO, if the volumeTO is managed, then doing whatever necessary operations(create SR, create VDI etc), then attach the VDI to volume.
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> Sent: Monday, June 10, 2013 11:10 AM
> To: John Burwell
> Cc: Edison Su; dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> I believe we're on the same page, John.
>
> I would add a new parameter to the createStoragePool API command (let's
> call it "managed" and it can be true or false). If you pass in "true", but
> the driver does not support being managed by CloudStack, an exception is
> thrown by the driver. If you pass in "false", but the driver does not
> support being managed outside of CloudStack, an exception is thrown by the
> driver. It is possible one could write a driver that can be managed both
> inside or outside of CloudStack (the SolidFire driver requires being
> managed only by CloudStack).
>
> This managed value would end up in the storage_pool table in a new column
> ("managed").
>
> An example usage: After the storage framework calls on the driver's
> createAsync method, it will check to see if the driver is managed. If the
> driver is managed, the storage framework will invoke the proper (new)
> command of the hypervisor in use to set up, say, the SR for the SAN volume.
>
> There may be other cases when the isManaged() method is invoked on the
> driver and - if true is returned - the storage framework sends a particular
> command to the hypervisor in use.
>
> ********************
>
> As far as reworking the default storage plug-in (which sends commands to
> hypervisors) for 4.2, I would check with Edison on his schedule.
>
> Thanks!
>
>
> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>>
> wrote:
>
> > Mike,
> >
> > I want to make sure we are on the same page regarding isManaged. I see
> > the addition of the following operations:
> >
> > * DataStoreDriver#supportsManagement() : boolean -- indicates whether
> or
> > not the driver support/implements the management functions
> > * DataStore#isManaged() : boolean -- indicates whether or not a device is
> > managed by CloudStack
> >
> > Only DataStores associated with DataStoreDrivers that support
> management
> > would be able to have the management flag enabled. This behavior allows
> > operators to declare their intention to allow CloudStack to manage a
> > device. This behavior would need to be exposed as flag on the HTTP API.
> > Additionally, this flag would only be settable when a device is
> > defined/created in the system (i.e. once a device is managed by CloudStack,
> > you can't unmanaged it without dissociating it from CloudStack). In the
> > future, we may consider relaxing that rule, but, in the short term, it
> > greatly decreases the alternative flows.
> >
> > In terms of removing the dependencies from Storage to Hypervisor, I
> would
> > really like to see this accomplished in 4.2. It will be much more
> > difficult to address this issue post 4.2 because we will have more plugins
> > to address. I also see this issue as a critical architectural issues that
> > for the management server. I recommend we identify the dependencies
> that
> > need to be inverted, and see if we can divide the work across a few more
> > resources.
> >
> > Thanks,
> > -John
> >
> > On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> <mi...@solidfire.com>>
> > wrote:
> >
> > Sounds good, Edison :)
> >
> > I can implement John's isManaged() logic and resubmit my code for review.
> >
> > Perhaps in 4.3 I could address the direct-attach scenario. If it requires
> > some refactoring of the storage framework, then I could do that.
> >
> > Also, the default storage plug-in does have hypervisor logic in it in 4.2.
> > I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> > with the above work I mentioned.
> >
> > Are you OK with that, guys?
> >
> >
> > On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>> wrote:
> >
> >> Ok, I am Ok with the way you are talking about in 4.2.****
> >>
> >> ** **
> >>
> >> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> >> *Sent:* Sunday, June 09, 2013 8:59 PM
> >> *To:* dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> >> *Cc:* Edison Su; John Burwell
> >> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>
> >> ** **
> >>
> >> Another thing we might want to consider for 4.2 is that the way the code
> >> is currently implemented in master, I don't think it's even possible to
> >> write a plug-in that connects storage directly to a VM because the
> >> CloudStack storage framework will call the "attach" method of the
> >> hypervisor in use and this method will fail because it assumes there is
> >> (talking Xen here) an SR existent and in this situation there isn't one.*
> >> ***
> >>
> >> ** **
> >>
> >> My thinking is that we should ignore the direct-attach-of-volume-to-VM
> >> use case for 4.2. I think it will require too much of a rewrite at this
> >> point in development (fewer than three weeks left until code
> freeze).****
> >>
> >> ** **
> >>
> >> If the plug-in implements an isManaged() method (as John suggested),
> the
> >> storage framework can invoke the necessary commands on the
> hypervisor in
> >> use.****
> >>
> >> ** **
> >>
> >> For example, the createAsync method is called. The storage plug-in
> >> creates a SAN volume and updates the CS DB. The storage framework
> then asks
> >> the driver if it's managed. If it is, the storage framework executes a
> >> command against the hypervisor in use to have it create, say, an SR (with
> a
> >> single VDI that takes up the entire SR). Now, when the storage
> framework
> >> executes the "attach" command against the hypervisor, this command will
> >> work, as well (because that logic assumes the prior existence of an SR and
> >> it will, in fact, be present).****
> >>
> >> ** **
> >>
> >> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:****
> >>
> >> Hi Edison and John,****
> >>
> >> ** **
> >>
> >> I wanted to point out something I thought was relevant to this
> >> conversation.****
> >>
> >> ** **
> >>
> >> My plug-in supports both XenServer and ESX(i).****
> >>
> >> ** **
> >>
> >> These two hypervisors handle their respective hypervisor data structures
> >> (XenServer SR or ESX(i) datastore) quite differently.****
> >>
> >> ** **
> >>
> >> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> >> when you issue a remove-SR command to XenServer.****
> >>
> >> ** **
> >>
> >> When you later want to access the contents of that data, it is still
> >> safely stored on the LUN and can be accessed from XenServer (from the
> same
> >> or different cluster (what XenServer calls a resource pool)) by creating a
> >> new SR that is based on the same IQN and LUN.****
> >>
> >> ** **
> >>
> >> XenServer SRs exist at the cluster level.****
> >>
> >> ** **
> >>
> >> In ESX(i), the datastore that resides on the SAN LUN is actually
> >> destroyed when you issue a remove-datastore command to ESX(i). In
> other
> >> words, the contents of the SAN LUN are destroyed.****
> >>
> >> ** **
> >>
> >> ESX(i) datastores exist at the datacenter level.****
> >>
> >> ** **
> >>
> >> The XenServer behavior (not destroying the contents of the LUN when
> the
> >> SR is removed) is desirable; the ESX(i) behavior (destroying the contents
> >> of the LUN when the datastore is removed) is not desirable.****
> >>
> >> ** **
> >>
> >> I can get around the ESX(i) behavior by removing the referenced IQN
> from
> >> each host in the cluster when the CS storage framework notifies my plug-
> in
> >> of a detach-volume event. When the CS volume is later attached again
> >> (either to the same or a different ESX(i) cluster), I can have each host in
> >> the relevant ESX(i) cluster reference the IQN and the datastore will be
> >> back again.****
> >>
> >> ** **
> >>
> >> This sequence of events is critical for my plug-in to support both
> >> environments:****
> >>
> >> *XenServer flow (events for Driver class):*****
> >>
> >> createAsync (called right before first attach) = creates SAN volume****
> >>
> >> preAttachVolume = creates SR and creates VDI or just introduces SR****
> >>
> >> (CS volume gets attached and later detached)****
> >>
> >> postDetachVolume = removes SR (this does not delete data on SAN
> volume)**
> >> **
> >>
> >> deleteAsync (can be called when volume is not attached) = deletes the
> SAN
> >> volume****
> >>
> >> *VMware flow (events for Driver class):*****
> >>
> >> createAsync (called right before first attach) = creates SAN volume,
> >> creates datastore, and creates VMDK file****
> >>
> >> preAttachVolume = adds iSCSI target to each host in cluster****
> >>
> >> (CS volume gets attached and later detached)****
> >>
> >> postDetachVolume = removes iSCSI target from each host in cluster****
> >>
> >> deleteAsync (can be called when volume is not attached) = deletes
> >> datastore and deletes the SAN volume****
> >>
> >> ************************
> >>
> >> So, the way I have it written up here, the driver is talking to the
> >> hypervisor. As John has pointed out, this may not be ideal.****
> >>
> >> ** **
> >>
> >> Whatever we decide on, it needs to be able to support this kind of flow.
> >> :)****
> >>
> >> ** **
> >>
> >> We could go John's route, where the driver says if it's managed or not.**
> >> **
> >>
> >> ** **
> >>
> >> If it is, then the storage framework could send the necessary messages to
> >> the hypervisor in question at the right times (just migrate the hypervisor
> >> logic above from the driver to the right parts of the storage framework).
> >> ****
> >>
> >> ** **
> >>
> >> This is fine, except that it does not resolve Edison's use case of
> >> desiring to have the storage LUN potentially connected directly to a VM.*
> >> ***
> >>
> >> ** **
> >>
> >> I'm not sure we should design around that use case, though. Is it common
> >> enough? Has anyone asked for it? Is it even desirable? How are usage
> >> statistics gathered for such volumes? It's nice to offer the flexibility,
> >> but is it a good idea in this case?****
> >>
> >> ** **
> >>
> >> Thanks!****
> >>
> >> ** **
> >>
> >> ** **
> >>
> >> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>> wrote:**
> >> **
> >>
> >>
> >>
> >> > -----Original Message-----
> >> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> >> > Sent: Friday, June 07, 2013 4:08 PM
> >> > To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> >> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >> >
> >> > Hi John,
> >> >
> >> > Yes, you are correct that Xen must layer an SR on top of the iSCSI
> >> volume
> >> > to use it. Same story for VMware: ESX(i) must layer a datastore on top
> >> of
> >> > the iSCSI volume to use it. I look at it like they are layering a
> >> clustered
> >> > file system on the SAN volume so the hypervisors can share access to
> the
> >> > contents of the volume.
> >> >
> >> > In 4.1, by the time the hypervisor-attach-volume logic is called, the
> >> SR or
> >> > datastore has already been created (usually manually by an admin).
> >> >
> >> > This pre-setup of the, say, SR is, of course, not acceptable in an
> >> > environment where each CloudStack volume that a user creates is
> mapped
> >> > to a
> >> > single SAN volume (via the SR).
> >> >
> >> > The question comes down to who should allocate the SR dynamically.
> >> >
> >> > We could have the storage framework ask the storage plug-in if it is
> >> > managed. If it is, then the storage framework could send a message to
> >> the
> >> > hypervisor in question to create (let's talk Xen here) the SR ahead of
> >> > time. Then, when the storage framework next sends the attach-volume
> >> > command
> >> > to the hypervisor, it should work without changes to that attach-volume
> >> > logic (because - from the point of view of the attach logic - the SR is
> >> > already existent, as expected).
> >> >
> >> > Now, as Edison has pointed out, this limits the power of the storage
> >> > plug-in. A storage plug-in in this model cannot directly attach storage
> >> to
> >> > a VM (it must go through the hypervisor). Perhaps that is OK. We need
> to
> >> > make a call on that.
> >>
> >>
> >> Yes, I think we need to address this usage case, and won't limit the
> >> power of the storage plug-in.
> >> John, what's your idea to fit this usage case into your abstraction?
> >>
> >>
> >> >
> >> > If we want to give more power to the storage vendor, we would have to
> >> > have
> >> > the storage framework call into the storage plug-in to send the
> >> appropriate
> >> > attach commands to the hypervisor.
> >> >
> >> > Let's discuss and come to a consensus (or at least agree on a path) as
> >> soon
> >> > as we can. :)
> >> >
> >> > Thanks!
> >> >
> >> >
> >> >
> >> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>>
> >> wrote:
> >> >
> >> > > Mike,
> >> > >
> >> > > My understanding is the Xen expects an SR structure on all ISCSI
> >> devices
> >> > > -- at least that is how I read the code in your patch. Is my
> >> understanding
> >> > > correct? If so, the Xen plugin should be able to query the storage
> >> device
> >> > > to determine the presence of the SR structure and create it if it
> >> does not
> >> > > exist. Am I missing something in the implementation that makes that
> >> type
> >> > > of implementation impossible?
> >> > >
> >> > > Thanks,
> >> > > -John
> >> > >
> >> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > wrote:
> >> > >
> >> > > > Yeah, if a storage vendor wanted to attach a volume directly to a
> >> VM in
> >> > > 4.2
> >> > > > today, it would probably fail because the attach-volume logic
> >> assumes
> >> > the
> >> > > > existence of the necessary hypervisor data structure (ex. SR on
> >> Xen).
> >> > > >
> >> > > > If we wanted to enable such an attach, we could do it the way
> Edison
> >> > > > suggests.
> >> > > >
> >> > > >
> >> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>>
> >> wrote:
> >> > > >
> >> > > >>
> >> > > >>
> >> > > >>> -----Original Message-----
> >> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> >> > > >>> Sent: Friday, June 07, 2013 2:37 PM
> >> > > >>> To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> >> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >> > > >>>
> >> > > >>> As we only have three weeks until feature freeze, we should
> come
> >> to
> >> > a
> >> > > >>> consensus on this design point as soon as possible.
> >> > > >>>
> >> > > >>> Right now, if the storage framework asks my driver it is is
> >> managed, it
> >> > > >>> will say 'yes.' This means the framework will tell the driver to
> >> > > perform
> >> > > >>> its management activities. This then means the driver will call
> >> into
> >> > > the
> >> > > >>> host (it doesn't know which hypervisor, by the way) to perform
> the
> >> > > >> activity
> >> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> >> > > >>>
> >> > > >>> The driver doesn't know which hypervisor it's talking to, it just
> >> > > sends a
> >> > > >>> message to the host to perform the necessary pre-attach work.
> >> > > >>
> >> > > >> Could we just expose a method like
> "attachVolume/dettachVolume" on
> >> > the
> >> > > >> PrimaryDataStoreDriver driver?
> >> > > >> For most of cases, the implementation of each driver would just
> >> send
> >> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can
> put
> >> > the
> >> > > >> implementation in a base class, so that can be shared by all of
> >> this
> >> > > >> drivers), and in each hypervisor resource code, which may just call
> >> > > >> hypervisor's api to attach the volume to VM, while for certain
> >> storage,
> >> > > >> like SolidFire, may need to create a SR first, and create volume
> >> on it,
> >> > > >> then call hypervisor's API to attach volume to VM.
> >> > > >> While for some other storage vendor, may want to bypass
> hypervisor
> >> > > during
> >> > > >> attaching volume, so inside driver's attachvolume implementation,
> >> the
> >> > > >> driver can do some magic, such as, directly talk to an agent
> >> inside VM
> >> > > >> instance, then create a disk inside VM.
> >> > > >>
> >> > > >> How do you guys think?
> >> > > >>
> >> > > >>
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>>
> >> > > wrote:
> >> > > >>>
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>> -----Original Message-----
> >> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> >> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> >> > > >>>>> To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> >> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >> > > >>>>>
> >> > > >>>>> Hi John,
> >> > > >>>>>
> >> > > >>>>> How's about this:
> >> > > >>>>>
> >> > > >>>>> The driver can implement an isManaged() method. The
> >> > > >>> VolumeManagerImpl
> >> > > >>>>> can
> >> > > >>>>> call into the driver to see if its managed. If it is, the
> >> > > >>>> VolumeManagerImpl
> >> > > >>>>> (which is responsible for calling into the hypervisor to attach
> >> the
> >> > > >> disk)
> >> > > >>>>> can call into the hypervisor to create the necessary hypervisor
> >> data
> >> > > >>>>> structure (ex. for XenServer, a storage repository).
> >> > > >>>>
> >> > > >>>> The problem here is that storage vendor may work differently
> with
> >> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> >> maybe
> >> > > >> other
> >> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> >> > directly
> >> > > >> to
> >> > > >>>> VM instance, see the discuss(
> >> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >> > > >>>
> >> >
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com<ma...@mail.gmail.com>
> >> > > >>> %3E
> >> > > >>>> ).
> >> > > >>>> So I would let storage provider to implement attach disk to VM,
> >> > > >> instead of
> >> > > >>>> implemented by cloudstack itself.
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>>
> >> > > >>>>> If that's what you're going for, that works for me. By the way,
> >> > > >> Edison's
> >> > > >>>>> default storage plug-in (which handles the default storage
> >> behavior
> >> > > >> in
> >> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> >> talks to
> >> > > >>>>> hypervisors. You might want to contact him and inform him of
> >> your
> >> > > >>>> concerns
> >> > > >>>>> or that logic (as is) will make it to production.
> >> > > >>>>>
> >> > > >>>>> Please let me know if what I wrote in above (for my solution)
> >> is OK
> >> > > >> with
> >> > > >>>>> you. :)
> >> > > >>>>>
> >> > > >>>>> Thanks!
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >> jburwell@basho.com<ma...@basho.com>>
> >> > > >>> wrote:
> >> > > >>>>>
> >> > > >>>>>> Mike,
> >> > > >>>>>>
> >> > > >>>>>> Please see my responses in-line below.
> >> > > >>>>>>
> >> > > >>>>>> Thanks,
> >> > > >>>>>> -John
> >> > > >>>>>>
> >> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >> > > >>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>> wrote:
> >> > > >>>>>>
> >> > > >>>>>>> Hey John,
> >> > > >>>>>>>
> >> > > >>>>>>> I still have a bit more testing I'd like to do before I build
> >> up
> >> > > >> a
> >> > > >>>> patch
> >> > > >>>>>>> file, but this is the gist of what I've done:
> >> > > >>>>>>>
> >> > > >>>>>>> * During a volume-attach operation, after
> VolumeManagerImpl
> >> > tells
> >> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I
> have
> >> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> driver if
> >> it
> >> > > >>>>> managed.
> >> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> >> > > >> whatever
> >> > > >>>>>>> activity is required. In my case, this includes sending a
> >> > > >> message to
> >> > > >>>> the
> >> > > >>>>>>> host where the VM is running to have, say XenServer, add a
> >> > > >> storage
> >> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of
> the
> >> > > >> SAN
> >> > > >>>>>> volume,
> >> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
> >> can
> >> > > >> on
> >> > > >>>> the
> >> > > >>>>>>> storage repository). After this, the normal attach-volume
> >> > > >> message is
> >> > > >>>> sent
> >> > > >>>>>>> to the host by VolumeManagerImpl.
> >> > > >>>>>>
> >> > > >>>>>> There should be **no** code from a storage driver to a
> >> hypervisor.
> >> > > >> I
> >> > > >>>>>> apologize for the repetition, but we simply can not have
> >> hypervisor
> >> > > >>>>>> specific code in the storage layer. The circular dependencies
> >> > > >> between
> >> > > >>>> the
> >> > > >>>>>> two layers are not sustainable in the long term. Either the
> >> > > >>>> VirtualManager
> >> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >> > > >>> coordinate
> >> > > >>>>>> volume creation and then populating the SR. Ideally, we can
> >> > > >>>> generalize the
> >> > > >>>>>> process flow for attaching volumes such that the Xen
> hypervisor
> >> > > >> plugin
> >> > > >>>>>> would only implement callbacks to perform the attach action
> and
> >> > > >> create
> >> > > >>>>> the
> >> > > >>>>>> structure and SR. To my mind, the SolidFire driver should
> >> only be
> >> > > >>>>>> allocating space and providing information about contents
> (e.g.
> >> > > >> space
> >> > > >>>>>> available, space consumed, streams to a URI, file handle for a
> >> URI,
> >> > > >>>> etc)
> >> > > >>>>>> and capabilities.
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> * The reverse is performed for a detach-volume command.
> >> > > >>>>>>>
> >> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
> >> driver.
> >> > > >>>>>> Edison's
> >> > > >>>>>>> default driver simply does a "return false;". We could add a
> >> new
> >> > > >>>>>> parameter
> >> > > >>>>>>> to the createStoragePool API command, if we want, to
> remove
> >> > the
> >> > > >>>>>> hard-coded
> >> > > >>>>>>> return values in the drivers (although my driver will probably
> >> > > >> just
> >> > > >>>>>> ignore
> >> > > >>>>>>> this parameter and always return true since it wouldn't make
> >> > > >> sense
> >> > > >>>> for it
> >> > > >>>>>>> to ever return false). We'd need another column in the
> >> > > >> storage_pool
> >> > > >>>>> table
> >> > > >>>>>>> to store this value.
> >> > > >>>>>>
> >> > > >>>>>> Yes, I think we should have a parameter added to the
> >> > > >>> createStoragePool
> >> > > >>>>>> surfaced to the HTTP API that allows DataStores to be
> >> configured
> >> > > >> for
> >> > > >>>>>> management when their underlying drivers support it. To
> >> simplify
> >> > > >>>> things,
> >> > > >>>>>> this flag should only be mutable when the DataStore is
> >> created. It
> >> > > >>>> would be
> >> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged
> after
> >> > > >>>> creation.
> >> > > >>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> Sound like I'm in sync with what you were thinking?
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >> > > >>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>> wrote:
> >> > > >>>>>>>
> >> > > >>>>>>>> I agree, John. Just wanted to point out that I have a
> working
> >> > > >> GUI
> >> > > >>>> for
> >> > > >>>>>> you
> >> > > >>>>>>>> to review (in that document), if you'd like to check it out.
> >> > > >>>>>>>>
> >> > > >>>>>>>> Thanks!
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >> > > >> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>> wrote:
> >> > > >>>>>>>>
> >> > > >>>>>>>>> Mike,
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> I would like the UIs of two features reviewed together to
> >> > > >> ensure
> >> > > >>>>>>>>> consistency across the concepts of hypervisor throttled
> IOPs
> >> > > >> and
> >> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
> >> > > >>>> confusion,
> >> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
> >> > > >> help
> >> > > >>>>>>>>> minimize any potential confusion.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
> >> itself
> >> > > >> if
> >> > > >>>> it
> >> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> >> > > >>>> throttled
> >> > > >>>>>>>>> IOPs or storage provisioned IOPs.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> Thanks,
> >> > > >>>>>>>>> -John
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >> > > >>>>>>>>> <mi...@solidfire.com>> wrote:
> >> > > >>>>>>>>>
> >> > > >>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
> sent
> >> > to
> >> > > >>> you
> >> > > >>>>> the
> >> > > >>>>>>>>>> other day to see how the GUI looks for provisioned
> storage
> >> > > >> IOPS.
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> >> > > >> and we
> >> > > >>>>>>>>> decided
> >> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> >> > Offering
> >> > > >>>> dialog.
> >> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> >> > should
> >> > > >>> be
> >> > > >>>>> able
> >> > > >>>>>> to
> >> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only
> >> use
> >> > > >> Max
> >> > > >>>>>> IOPS).
> >> > > >>>>>>>>>> These fields are optional and can be ignored for storage
> >> that
> >> > > >>>> does not
> >> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
> >> the
> >> > > >>>> admin can
> >> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> Burst
> >> > > >> IOPS.
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature
> or
> >> > > >> mine,
> >> > > >>>> as
> >> > > >>>>>> is
> >> > > >>>>>>>>>> decided by the admin).
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> I'm not sure what we can do about these two features
> >> > > >> expressing
> >> > > >>>> the
> >> > > >>>>>>>>> speed
> >> > > >>>>>>>>>> in different terms. I've never seen a SAN express the
> IOPS
> >> for
> >> > > >>>> QoS in
> >> > > >>>>>>>>> any
> >> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
> >> read/write
> >> > > >>>> IOPS).
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> Thanks
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >> > > >>> <jb...@basho.com>>
> >> > > >>>>>>>>> wrote:
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>> Wei,
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> >> > > >>> Storage/Hypervisor
> >> > > >>>>>> layer
> >> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior
> of
> >> > > >>>> hypervisor
> >> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> >> > > >> following
> >> > > >>>>>> issues
> >> > > >>>>>>>>>>> outstanding:
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >> > > >>> read/write
> >> > > >>>>>>>>> values
> >> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as
> total
> >> > > >> IOPS.
> >> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
> to
> >> > > >>> storage
> >> > > >>>>>> volumes
> >> > > >>>>>>>>>>> with provisioned IOPS?
> >> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> >> > > >>>> provisioned
> >> > > >>>>>> IOPS
> >> > > >>>>>>>>>>> that will permit providers to discriminate these
> >> guaranteed
> >> > > >>>>>> operations
> >> > > >>>>>>>>> in
> >> > > >>>>>>>>>>> the event they want to bill for it differently?
> >> > > >>>>>>>>>>> 4. What is the user experience for throttled and
> >> provisioned
> >> > > >>> IOPS
> >> > > >>>>>> that
> >> > > >>>>>>>>>>> minimizes confusion of these concepts?
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> My thinking is that a VM can have either utilize
> >> hypervisor
> >> > > >>>> throttled
> >> > > >>>>>>>>> IOPS
> >> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> >> solve
> >> > > >>>> items 1
> >> > > >>>>>>>>> and 2.
> >> > > >>>>>>>>>>> Since the two facilities would not be permitted to
> operate
> >> > > >>>> together,
> >> > > >>>>>>>>> they
> >> > > >>>>>>>>>>> do not need to be semantically compatible. I think item
> 3
> >> > > >> can be
> >> > > >>>>>>>>> resolved
> >> > > >>>>>>>>>>> with an additional flag or two on the usage records. As
> >> for
> >> > > >>>> Item 4,
> >> > > >>>>>> I
> >> > > >>>>>>>>> am
> >> > > >>>>>>>>>>> not familiar with how these two enhancements are
> >> > depicted in
> >> > > >>> the
> >> > > >>>>> user
> >> > > >>>>>>>>>>> interface. I think we need to review the UI
> enhancements
> >> > for
> >> > > >>>> both
> >> > > >>>>>>>>>>> enhancements and ensure they are consistent.
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> Do these solutions make sense?
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>> -John
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >> > <us...@gmail.com>>
> >> > > >>>>> wrote:
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>>> John and Mike,
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> >> > 2780/2729,
> >> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >> > > >>> CLOUDSTACK-
> >> > > >>>>> 2875 ) this
> >> > > >>>>>>>>> week.
> >> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
> and
> >> > > >> ask
> >> > > >>>> for
> >> > > >>>>>>>>> second
> >> > > >>>>>>>>>>>> merge request after testing.
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>> -Wei
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> <mi...@solidfire.com>>
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
> >> John,
> >> > > >> and
> >> > > >>>>> have
> >> > > >>>>>>>>> been
> >> > > >>>>>>>>>>>>> re-working this section of code today.
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> Thanks for the comments,
> >> > > >>>>>>>>>>>>> Mike
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >> > > >>>>> <jb...@basho.com>>
> >> > > >>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> See my responses in-line below.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Thanks for the response.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> >> > whether or
> >> > > >>>>> not a
> >> > > >>>>>>>>> Volume
> >> > > >>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> but an
> >> > > >>> actual
> >> > > >>>>>>>>> isManaged
> >> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> >> > > >> resources
> >> > > >>>> for
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>> if it is managed."
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> >> > method
> >> > > >>>>> added to
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> >> > > >> then the
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever
> name)
> >> > > >>> method
> >> > > >>>>> (which
> >> > > >>>>>>>>>>> would
> >> > > >>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
> well)
> >> > and
> >> > > >>> this
> >> > > >>>>>> would
> >> > > >>>>>>>>>>> call
> >> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create,
> say
> >> > on
> >> > > >>>>>>>>> XenServer,
> >> > > >>>>>>>>>>> an
> >> > > >>>>>>>>>>>>> SR?
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >> > > >>>>> DataStoreDriver.
> >> > > >>>>>>>>> Since
> >> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> storage,
> >> > I
> >> > > >>>> would
> >> > > >>>>>> add
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> >> > straw
> >> > > >>> man
> >> > > >>>>> term
> >> > > >>>>>>>>> --
> >> > > >>>>>>>>>>>>>> something other than volume) methods to
> >> > > >>>>>>>>>>> allocate/create/deallocate/delete
> >> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> mutable
> >> > > >>> property
> >> > > >>>>> of
> >> > > >>>>>>>>>>>>> DataStore
> >> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> >> > > >>> DataStoreDriver
> >> > > >>>>> can be
> >> > > >>>>>>>>>>>>> managed.
> >> > > >>>>>>>>>>>>>> This approach allows operators to override
> >> > manageability
> >> > > >> of
> >> > > >>>>>> devices.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
> Xen
> >> > > >> plugin
> >> > > >>>>>> would
> >> > > >>>>>>>>> be
> >> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume
> methods
> >> > to
> >> > > >>>>> create any
> >> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There
> >> should
> >> > > >> be no
> >> > > >>>>>>>>>>>>> dependencies
> >> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> >> > > >> earlier,
> >> > > >>>> such
> >> > > >>>>>>>>>>>>> circular
> >> > > >>>>>>>>>>>>>> dependencies will lead to a tangled,
> unmaintainable
> >> > mess.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with
> you.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Thanks again, John
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >> > > >>>>> <jb...@basho.com>>
> >> > > >>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
> layer
> >> > that
> >> > > >>>>>> supports n
> >> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> >> > > >> hypervisors.
> >> > > >>>>>> Such
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and
> untestable
> >> > > >>> beast.
> >> > > >>>>>>>>>>>>> Therefore, my
> >> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to
> evolve
> >> > the
> >> > > >>>>> Storage
> >> > > >>>>>> layer
> >> > > >>>>>>>>>>>>> towards
> >> > > >>>>>>>>>>>>>>> this separation.
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> >> > whether
> >> > > >>> or
> >> > > >>>>> not a
> >> > > >>>>>>>>> Volume
> >> > > >>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> but
> >> > an
> >> > > >>> actual
> >> > > >>>>>>>>>>> isManaged
> >> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >> > > >> resources
> >> > > >>>> for
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
> >> needs
> >> > > >> to
> >> > > >>>>>> indicate
> >> > > >>>>>>>>>>>>> whether
> >> > > >>>>>>>>>>>>>>> or not it supports management operations.
> Finally, I
> >> > > >> think
> >> > > >>>> we
> >> > > >>>>>>>>> need to
> >> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to
> >> have
> >> > > >>>>>> something
> >> > > >>>>>>>>>>>>> that is
> >> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> capable
> >> > > >>> managing
> >> > > >>>>> the
> >> > > >>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> >> > unmanaged).
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be
> done
> >> > in
> >> > > >>> the
> >> > > >>>>> Xen
> >> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> >> > Storage
> >> > > >>>>> layer to
> >> > > >>>>>>>>>>> perform
> >> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> create a
> >> > > >> file,
> >> > > >>>> etc).
> >> > > >>>>>>>>>>> This
> >> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >> > > >>>> hypervisor, as
> >> > > >>>>>>>>> such,
> >> > > >>>>>>>>>>>>>>> should be confined to its implementation. From
> my
> >> > > >>>>> perspective,
> >> > > >>>>>>>>>>> nothing
> >> > > >>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with
> content.
> >> > From
> >> > > >>> its
> >> > > >>>>>>>>>>>>> perspective,
> >> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the
> means
> >> > to
> >> > > >>>> query
> >> > > >>>>>> the
> >> > > >>>>>>>>>>> data
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>> support the interpretation of the content by
> higher-
> >> > level
> >> > > >>>>> layers
> >> > > >>>>>>>>> (e.g.
> >> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> >> > composition
> >> > > >>> of
> >> > > >>>>>>>>>>> operations
> >> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> >> > Volume
> >> > > >>>>> storage
> >> > > >>>>>>>>>>> protocol
> >> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> >> > implemented
> >> > > >> in
> >> > > >>>> my
> >> > > >>>>>>>>> patch,
> >> > > >>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>> could do the following:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >> > > >>>>>>>>> VolumeManagerImpl
> >> > > >>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
> the
> >> > > >>>>> hypervisor.
> >> > > >>>>>> Before
> >> > > >>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> >> > dealing
> >> > > >>> with
> >> > > >>>>>>>>> Dynamic
> >> > > >>>>>>>>>>> (or
> >> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> >> send a
> >> > > >>>> "Create
> >> > > >>>>>>>>> SR"
> >> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> >> > would
> >> > > >>> then
> >> > > >>>>>>>>> proceed
> >> > > >>>>>>>>>>> to
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and
> we just
> >> > > >> add
> >> > > >>>>>> another
> >> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
> >> > > >> particular
> >> > > >>>>> type
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>>>> storage.
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> What do you think?
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> Hey John,
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
> >> > outlines
> >> > > >>> how
> >> > > >>>>> the
> >> > > >>>>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably
> be
> >> of
> >> > > >> use
> >> > > >>>> to
> >> > > >>>>>>>>> you, as
> >> > > >>>>>>>>>>>>> well,
> >> > > >>>>>>>>>>>>>>>> as you perform the code review.
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> I have shared this document with you (you
> should
> >> > have
> >> > > >>>>> received
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> Talk to you later!
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >> > > >>>>>> jburwell@basho.com<ma...@basho.com>
> >> > > >>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> >> > thoughts,
> >> > > >>> but
> >> > > >>>>>> want
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> >> solidfire,
> >> > > >>>>>>>>>>>>> disk_io_throttle,
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
> in a
> >> > > >>>>> coherent
> >> > > >>>>>>>>> form
> >> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> introduces
> >> > the
> >> > > >>>> notion
> >> > > >>>>>> of
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> introduces
> >> > > >> the
> >> > > >>>>> notion
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the
> plug-in
> >> > was
> >> > > >>>>>>>>> incomplete,
> >> > > >>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>> I had
> >> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> >> > > >> device.
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> spending on
> >> > > >> this.
> >> > > >>>> :)
> >> > > >>>>> Do
> >> > > >>>>>>>>> you
> >> > > >>>>>>>>>>>>>>>>>> have a
> >> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should
> accomplish
> >> > > >>> what
> >> > > >>>>> you're
> >> > > >>>>>>>>> looking
> >> > > >>>>>>>>>>>>>>>>>> for?
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
> <
> >> > > >>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> introduces
> >> > the
> >> > > >>>> notion
> >> > > >>>>>> of
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> management
> >> > > >>>>> capabilities
> >> > > >>>>>>>>> (i.e.
> >> > > >>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
> it
> >> > > >>> managed
> >> > > >>>>> by
> >> > > >>>>>>>>> CloudStack)
> >> > > >>>>>>>>>>>>>>>>>> from the
> >> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
> with
> >> > a
> >> > > >>>>> semantic
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>> will
> >> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> managed.
> >> > I
> >> > > >>> also
> >> > > >>>>> want
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
> from
> >> > the
> >> > > >>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> definitions.
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
> <
> >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> Burwell <
> >> > > >>>>>>>>>>> jburwell@basho.com<ma...@basho.com>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> >> > questions
> >> > > >>> for
> >> > > >>>>> me
> >> > > >>>>>> with
> >> > > >>>>>>>>>>>>>>>>>> regards to
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> LUNs
> >> > and
> >> > > >>> SAN
> >> > > >>>>>> volumes?
> >> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
> LUN
> >> > (LUN
> >> > > >>> 0).
> >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> Volumes
> >> > and
> >> > > >>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> Volumes?
> >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> management
> >> > > >>> server or
> >> > > >>>>>>>>> externally
> >> > > >>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>> operator?
> >> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
> SAN
> >> > > >>> volume
> >> > > >>>>> (same
> >> > > >>>>>> as a
> >> > > >>>>>>>>>>> SAN
> >> > > >>>>>>>>>>>>>>>>>> LUN) is
> >> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via
> the
> >> > plug-in)
> >> > > >>>>> the
> >> > > >>>>>> first
> >> > > >>>>>>>>>>> time
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> hypervisor.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
> in,
> >> > but
> >> > > >>>> still
> >> > > >>>>>>>>> want
> >> > > >>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> use a
> >> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
> already
> >> > > >>> today
> >> > > >>>>>> (prior
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> 4.2). The
> >> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
> and -
> >> > in
> >> > > >>> this
> >> > > >>>>> case -
> >> > > >>>>>>>>>>>>>>>>>> multiple VMs
> >> > > >>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> While
> >> > you
> >> > > >>> can do
> >> > > >>>>> this
> >> > > >>>>>>>>>>>>> today,
> >> > > >>>>>>>>>>>>>>>>>> it is
> >> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
> QoS.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> management
> >> > server
> >> > > >>> or
> >> > > >>>>> externally
> >> > > >>>>>>>>> by
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> operator?
> >> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> SAN
> >> > > >>> volumes
> >> > > >>>>> are
> >> > > >>>>>>>>>>>>> completely
> >> > > >>>>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
> in).
> >> > There
> >> > > >>> is
> >> > > >>>>> no
> >> > > >>>>>> admin
> >> > > >>>>>>>>>>>>>>>>>>>> interaction.
> >> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
> SAN
> >> > > >>> volume
> >> > > >>>>> and a
> >> > > >>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>> volume,
> >> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
> that
> >> > > >>>>> supports
> >> > > >>>>>> true
> >> > > >>>>>>>>>>> QoS.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
> are
> >> > > >> related
> >> > > >>>>> and
> >> > > >>>>>>>>>>>>> expected
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> operate.
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> >> > multiple
> >> > > >>> VMs
> >> > > >>>>> with
> >> > > >>>>>>>>>>>>>>>>>> provisioned
> >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> storage."
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
> map a
> >> > > >>> single
> >> > > >>>>> SAN
> >> > > >>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>> (LUN) to a
> >> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
> have
> >> > > >>> multiple
> >> > > >>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> >> > guarantee
> >> > > >>> QoS.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> >> > volume
> >> > > >>> house
> >> > > >>>>> more
> >> > > >>>>>>>>> than
> >> > > >>>>>>>>>>>>> one
> >> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
> today
> >> > > >>> without
> >> > > >>>>> any of
> >> > > >>>>>> my
> >> > > >>>>>>>>>>>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>>>> code.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> >> > Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> >> > volume
> >> > > >>> for
> >> > > >>>>>>>>>>> CloudStack's
> >> > > >>>>>>>>>>>>>>>>>> use
> >> > > >>>>>>>>>>>>>>>>>>>> onto
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> created."
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> >> > Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> >> > concerns me
> >> > > >>> --
> >> > > >>>>>> concepts
> >> > > >>>>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
> layer.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> >> > Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
> is
> >> > today
> >> > > >>>> only
> >> > > >>>>>> used
> >> > > >>>>>>>>>>> (as
> >> > > >>>>>>>>>>>>>>>>>> far as
> >> > > >>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
> have
> >> > not
> >> > > >>>>> PreSetup an
> >> > > >>>>>>>>> SR).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> volume
> >> > from
> >> > > >>>>> VMware, it
> >> > > >>>>>>>>> uses
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> vmfs
> >> > > >>>>>>>>>>>>>>>>>>>>>> type.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> volume
> >> > from
> >> > > >>> KVM,
> >> > > >>>>> it uses
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> thinking
> >> > > >>> here
> >> > > >>>>> was to
> >> > > >>>>>>>>> make a
> >> > > >>>>>>>>>>>>>>>>>> new type
> >> > > >>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
> ability
> >> > > >>> Edison
> >> > > >>>>> added
> >> > > >>>>>>>>> into
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> more
> >> > > >>> specificy,
> >> > > >>>>> though:
> >> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> Mike
> >> > > >>> Tutkowski
> >> > > >>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> shouldn't
> >> > know
> >> > > >>> or
> >> > > >>>>> care
> >> > > >>>>>> that
> >> > > >>>>>>>>> it
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>> being
> >> > > >>>>>>>>>>>>>>>>>>>>>> used
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
> answer
> >> > > >>>>> questions
> >> > > >>>>>>>>> about
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
> the
> >> > > >>>>> SolidFire
> >> > > >>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>> needs to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
> the
> >> > > >>> hypervisor
> >> > > >>>>> logic
> >> > > >>>>>>>>>>> knows
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>> treat
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> >> > constraint
> >> > > >>>>> and just
> >> > > >>>>>>>>>>>>>>>>>> calling it
> >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
> way
> >> > for
> >> > > >>> the
> >> > > >>>>> hypervisor
> >> > > >>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> >> > perform
> >> > > >>> the
> >> > > >>>>> necessary
> >> > > >>>>>>>>>>>>>>>>>> activities.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> John
> >> > > >>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> >> > > >>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
> be
> >> > able to
> >> > > >>>>> manage a
> >> > > >>>>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> >> > > >>> CloudStack
> >> > > >>>>> (e.g. the
> >> > > >>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>> shared
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> operator has
> >> > > >>> chosen
> >> > > >>>>> to
> >> > > >>>>>>>>> dedicate
> >> > > >>>>>>>>>>>>>>>>>> only a
> >> > > >>>>>>>>>>>>>>>>>>>>>> portion
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> reasoning
> >> > > >>> make
> >> > > >>>>> sense?"
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> creating a
> >> > SAN
> >> > > >>>>> volume via
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> hypervisor)
> >> > would
> >> > > >>>>> work unless
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
> say,
> >> > an
> >> > > >>> SR.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
> of
> >> > > >>>>> CloudStack, an
> >> > > >>>>>>>>>>>>> operator
> >> > > >>>>>>>>>>>>>>>>>>>>>> allocates
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
> and
> >> > > >>> exposes it
> >> > > >>>>>> through
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> LUN. They
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
> over
> >> > to
> >> > > >>>>> CloudStack,
> >> > > >>>>>>>>> but
> >> > > >>>>>>>>>>>>>>>>>> not allow
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
> LUNs.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
> to my
> >> > > >>>>> changes, we
> >> > > >>>>>>>>> would
> >> > > >>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>> passing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
> the
> >> > > >>>>> necessary
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>> (like
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> maybe
> >> > have
> >> > > >>> the
> >> > > >>>>> storage
> >> > > >>>>>>>>>>>>> framework
> >> > > >>>>>>>>>>>>>>>>>>>> itself
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> >> > support
> >> > > >>> from
> >> > > >>>>> the
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> side and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> specifically
> >> > to
> >> > > >>> create
> >> > > >>>>> the
> >> > > >>>>>>>>> SR
> >> > > >>>>>>>>>>>>>>>>>> before the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> management
> >> > plugin
> >> > > >>>>> should
> >> > > >>>>>> have a
> >> > > >>>>>>>>>>>>> rich
> >> > > >>>>>>>>>>>>>>>>>> enough
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> determine
> >> > > >>> available
> >> > > >>>>> for
> >> > > >>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>> storage.
> >> > > >>>>>>>>>>>>>>>>>>>> For
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >> > > >>> interrogation of
> >> > > >>>>> the
> >> > > >>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> device
> >> > itself
> >> > > >>>>> shouldn't
> >> > > >>>>>>>>> know
> >> > > >>>>>>>>>>>>>>>>>> or care
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> simply be
> >> > able
> >> > > >>> to
> >> > > >>>>>> answer
> >> > > >>>>>>>>>>>>>>>>>> questions
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
> Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> SolidFire
> >> > > >>> driver
> >> > > >>>>> (which
> >> > > >>>>>>>>> is a
> >> > > >>>>>>>>>>>>>>>>>> type of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> >> > done, a
> >> > > >>> new
> >> > > >>>>> Primary
> >> > > >>>>>>>>>>>>>>>>>> Storage shows
> >> > > >>>>>>>>>>>>>>>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> Offering
> >> > > >>> that
> >> > > >>>>>> references
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>> tag
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> Storage.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> >> > CloudStack
> >> > > >>>>> volume. This
> >> > > >>>>>>>>> leads
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> a new
> >> > > >>>>>>>>>>>>>>>>>>>>>> row
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> >> > CloudStack
> >> > > >>>>> volume to a
> >> > > >>>>>>>>> VM
> >> > > >>>>>>>>>>>>>>>>>> (attach
> >> > > >>>>>>>>>>>>>>>>>>>>>> disk).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> framework
> >> > calling
> >> > > >>> the
> >> > > >>>>> plug-in
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> create a
> >> > > >>>>>>>>>>>>>>>>>>>> new
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
> case, a
> >> > > >>> SAN).
> >> > > >>>>> The
> >> > > >>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>> also
> >> > > >>>>>>>>>>>>>>>>>>>>>> updates
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> applicable
> >> > data
> >> > > >>>>> (like the
> >> > > >>>>>>>>> IQN of
> >> > > >>>>>>>>>>>>>>>>>> the SAN
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> invoked if
> >> > the
> >> > > >>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> volume is in
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> attach,
> >> > the
> >> > > >>>>> volume
> >> > > >>>>>>>>> will be
> >> > > >>>>>>>>>>>>>>>>>> in the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
> and
> >> > the
> >> > > >>>>> plug-in
> >> > > >>>>>>>>> code
> >> > > >>>>>>>>>>>>>>>>>> will not
> >> > > >>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> volume.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
> is run
> >> > and
> >> > > >>>>> detects
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
> the
> >> > form
> >> > > >>> of a
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
> PM,
> >> > Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> sync on
> >> > > >>>>> terminology,
> >> > > >>>>>>>>> volume,
> >> > > >>>>>>>>>>>>>>>>>> in these
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> physical
> >> > > >>>>> allocation on
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
> try to
> >> > > >>> mean
> >> > > >>>>> 'SAN
> >> > > >>>>>>>>> volume'.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
> end
> >> > user
> >> > > >>> can
> >> > > >>>>> make in
> >> > > >>>>>>>>>>>>>>>>>> CloudStack, I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
> PM,
> >> > Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> very
> >> > well
> >> > > >>> make
> >> > > >>>>> sense, but
> >> > > >>>>>>>>> I'm
> >> > > >>>>>>>>>>>>>>>>>> having a
> >> > > >>>>>>>>>>>>>>>>>>>>>> hard
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
> Edison
> >> > in on
> >> > > >>>>> this
> >> > > >>>>>>>>> conversation
> >> > > >>>>>>>>>>>>>>>>>> as he
> >> > > >>>>>>>>>>>>>>>>>>>> was
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> >> > > >>> approach I
> >> > > >>>>> took.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
> PM,
> >> > > >>> John
> >> > > >>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> combining
> >> > two
> >> > > >>>>> distinct
> >> > > >>>>>>>>> concepts
> >> > > >>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> >> > protocols.
> >> > > >>> In
> >> > > >>>>> both
> >> > > >>>>>>>>> cases, we
> >> > > >>>>>>>>>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> >> > system to
> >> > > >>>>>> create/delete
> >> > > >>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> requires
> >> > the
> >> > > >>>>> volume to
> >> > > >>>>>> be
> >> > > >>>>>>>>>>>>>>>>>> volume to be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> >> > context.
> >> > > >>> To
> >> > > >>>>> ensure
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>>>>> are in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
> these
> >> > > >>>>> definitions,
> >> > > >>>>>>>>> refers to
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>> physical
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> >> > correct?
> >> > > >>>>> Minimally,
> >> > > >>>>>> we
> >> > > >>>>>>>>>>>>> must
> >> > > >>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>> able
> >> > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> storage
> >> > device
> >> > > >>> to
> >> > > >>>>> move bits
> >> > > >>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>> one place
> >> > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >> > > >>> Optionally, a
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>> may
> >> > > >>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> >> > Therefore,
> >> > > >>>>> we can have
> >> > > >>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> unmanaged
> >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
> SR,
> >> > and
> >> > > >>> we
> >> > > >>>>> can have a
> >> > > >>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> CloudStack is
> >> > > >>> capable
> >> > > >>>>> of
> >> > > >>>>>>>>> allocating
> >> > > >>>>>>>>>>>>>>>>>> LUNs and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> >> > CloudStack
> >> > > >>> may
> >> > > >>>>> be able
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> manage a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> leave it
> >> > > >>>>> unmanaged by
> >> > > >>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> (e.g.
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
> and
> >> > the
> >> > > >>>>> operator has
> >> > > >>>>>>>>> chosen
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> CloudStack).
> >> > Does
> >> > > >>> my
> >> > > >>>>>> reasoning
> >> > > >>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>> sense?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> above
> >> > are
> >> > > >>>>> reasonable, it
> >> > > >>>>>>>>> seems
> >> > > >>>>>>>>>>>>>>>>>>>> appropriate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> >> > concerns
> >> > > >>> from
> >> > > >>>>>>>>> StoragePoolType,
> >> > > >>>>>>>>>>>>>>>>>> add the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> >> > attached
> >> > > >>> driver
> >> > > >>>>> that
> >> > > >>>>>>>>>>> indicates
> >> > > >>>>>>>>>>>>>>>>>> whether
> >> > > >>>>>>>>>>>>>>>>>>>>>> or
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
> and
> >> > > >>> establish
> >> > > >>>>> a
> >> > > >>>>>>>>> abstraction
> >> > > >>>>>>>>>>>>>>>>>>>>>> representing a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> separate
> >> > that
> >> > > >>> is
> >> > > >>>>>>>>> associated
> >> > > >>>>>>>>>>>>>>>>>> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>> volume.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> >> > hypervisor
> >> > > >>>>> drivers can
> >> > > >>>>>>>>>>>>> declare
> >> > > >>>>>>>>>>>>>>>>>> which
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> >> > encounter
> >> > > >>> a
> >> > > >>>>> device
> >> > > >>>>>> managed
> >> > > >>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> >> > exposed
> >> > > >>> by
> >> > > >>>>> the driver to
> >> > > >>>>>>>>>>>>> automate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
> make
> >> > > >>> sense,
> >> > > >>>>> then we can
> >> > > >>>>>>>>> sit
> >> > > >>>>>>>>>>>>>>>>>> down and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> >> > Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> >> > between
> >> > > >>> the
> >> > > >>>>> current
> >> > > >>>>>> iSCSI
> >> > > >>>>>>>>>>>>> type
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> has to
> >> > go in
> >> > > >>> and
> >> > > >>>>> create
> >> > > >>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> Primary
> >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> point in
> >> > time,
> >> > > >>>>> the
> >> > > >>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>> volume must
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> >> > allocated).
> >> > > >>>>> Future
> >> > > >>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
> was
> >> > created
> >> > > >>>>> behind the
> >> > > >>>>>>>>>>>>> scenes.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
> has
> >> > to
> >> > > >>> go in
> >> > > >>>>> and
> >> > > >>>>>> create
> >> > > >>>>>>>>>>>>>>>>>> Primary
> >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
> and
> >> > delete
> >> > > >>>>> volumes on
> >> > > >>>>>>>>> its
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> enabled via
> >> > the
> >> > > >>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> framework). When
> >> > > >>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> volume
> >> > that
> >> > > >>> was
> >> > > >>>>> created,
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> framework
> >> > > >>>>>>>>>>>>>>>>>>>>>> tells
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> >> > volume.
> >> > > >>> After
> >> > > >>>>> this is
> >> > > >>>>>>>>> done,
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> question is
> >> > called.
> >> > > >>>>> No
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> >> > volume
> >> > > >>> was
> >> > > >>>>> just
> >> > > >>>>>>>>> created.
> >> > > >>>>>>>>>>>>> The
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> created.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 3:21
> >> > PM,
> >> > > >>> Mike
> >> > > >>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
> so I
> >> > should
> >> > > >>>>> probably
> >> > > >>>>>> have
> >> > > >>>>>>>>>>>>>>>>>> defined them
> >> > > >>>>>>>>>>>>>>>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
> that is
> >> > > >>> pre-
> >> > > >>>>> allocated
> >> > > >>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>> an admin
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
> a
> >> > > >>> hypervisor
> >> > > >>>>> data
> >> > > >>>>>>>>> structure
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> XenServer
> >> > SR),
> >> > > >>> then
> >> > > >>>>> that
> >> > > >>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> CloudStack.
> >> > Disks
> >> > > >>> (VDI)
> >> > > >>>>> are later
> >> > > >>>>>>>>>>>>> placed
> >> > > >>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> needed. In
> >> > > >>> these
> >> > > >>>>> cases, the
> >> > > >>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> structure is
> >> > > >>> already
> >> > > >>>>> in place
> >> > > >>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> simply
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> >> > structure
> >> > > >>> to
> >> > > >>>>> the VM in
> >> > > >>>>>>>>>>>>>>>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> Storage
> >> > that is
> >> > > >>> not
> >> > > >>>>>>>>>>> pre-allocated.
> >> > > >>>>>>>>>>>>>>>>>> Instead
> >> > > >>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
> this
> >> > could
> >> > > >>> be a
> >> > > >>>>> SAN
> >> > > >>>>>> (not
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> volume on
> >> > > >>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> hypervisor
> >> > data
> >> > > >>>>> structure
> >> > > >>>>>>>>> must be
> >> > > >>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> >> > because
> >> > > >>> these
> >> > > >>>>> types of
> >> > > >>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>> have not
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
> data
> >> > > >>>>> structure by an
> >> > > >>>>>>>>>>> admin.
> >> > > >>>>>>>>>>>>>>>>>> Once
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> >> > XenServer
> >> > > >>> for
> >> > > >>>>> this
> >> > > >>>>>>>>> volume,
> >> > > >>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>> attaches
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
> to the
> >> > VM
> >> > > >>> in
> >> > > >>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 3:13
> >> > PM,
> >> > > >>>>> John Burwell
> >> > > >>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >> > implementation of
> >> > > >>> the
> >> > > >>>>> Dynamic
> >> > > >>>>>> type
> >> > > >>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>> behavior
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> which
> >> > why I
> >> > > >>> ask
> >> > > >>>>> about the
> >> > > >>>>>>>>>>>>>>>>>> difference.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> what is
> >> > the
> >> > > >>>>> definition
> >> > > >>>>>> of
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> Dynamic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
> PM,
> >> > > >>> Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
> iSCSI
> >> > type
> >> > > >>> is
> >> > > >>>>>> uniquely
> >> > > >>>>>>>>>>> used
> >> > > >>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> Primary
> >> > > >>> Storage
> >> > > >>>>> that is
> >> > > >>>>>>>>> directly
> >> > > >>>>>>>>>>>>>>>>>> based on
> >> > > >>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> skip the
> >> > > >>> step of
> >> > > >>>>> going
> >> > > >>>>>> to
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
> based
> >> > on
> >> > > >>> that
> >> > > >>>>> iSCSI
> >> > > >>>>>>>>> target as
> >> > > >>>>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
> only
> >> > > >>>>> supported for
> >> > > >>>>>>>>>>>>>>>>>> XenServer.
> >> > > >>>>>>>>>>>>>>>>>>>> For
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> must
> >> > first go
> >> > > >>> to
> >> > > >>>>> the
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>> perform
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
> what
> >> > RBD is.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> >> > 2:13
> >> > > >>> PM,
> >> > > >>>>> John
> >> > > >>>>>> Burwell
> >> > > >>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
> the
> >> > code,
> >> > > >>> what
> >> > > >>>>> is the
> >> > > >>>>>>>>>>> difference
> >> > > >>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> Why
> >> > isn't
> >> > > >>> RBD
> >> > > >>>>> considered
> >> > > >>>>>>>>>>>>> Dynamic?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> 3:46
> >> > PM,
> >> > > >>> Mike
> >> > > >>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> >> > storage is
> >> > > >>>>> defined in
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> >> > > >>>>> StoragePoolType {
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
> //
> >> > local
> >> > > >>>>> directory
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > NetworkFilesystem(true),
> >> > > >>> //
> >> > > >>>>> NFS or CIFS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> >> > shared
> >> > > >>> LUN,
> >> > > >>>>> with a
> >> > > >>>>>>>>>>> clusterfs
> >> > > >>>>>>>>>>>>>>>>>>>> overlay
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
> e.g.,
> >> > ZFS
> >> > > >>>>> Comstar
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
> iso
> >> > image
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> >> > XenServer
> >> > > >>> local
> >> > > >>>>> LVM SR
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > SharedMountPoint(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> >> > VMware
> >> > > >>> VMFS
> >> > > >>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
> for
> >> > > >>>>> XenServer, Storage
> >> > > >>>>>>>>> Pool
> >> > > >>>>>>>>>>>>>>>>>> is set
> >> > > >>>>>>>>>>>>>>>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> >> > XenServer
> >> > > >>> local
> >> > > >>>>> EXT SR
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> >> > dynamic,
> >> > > >>>>> zone-wide
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > StoragePoolType(boolean
> >> > > >>>>> shared) {
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> shared;
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> >> > isShared() {
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013 at
> >> > 1:41
> >> > > >>> PM,
> >> > > >>>>> Mike
> >> > > >>>>>>>>> Tutkowski
> >> > > >>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> let's say
> >> > > >>> another
> >> > > >>>>> storage
> >> > > >>>>>>>>>>> company
> >> > > >>>>>>>>>>>>>>>>>> wants
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> leverage its
> >> > > >>> Quality
> >> > > >>>>> of
> >> > > >>>>>> Service
> >> > > >>>>>>>>>>>>>>>>>> feature. It
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> storage, as
> >> > well.
> >> > > >>>>> They would
> >> > > >>>>>>>>> need
> >> > > >>>>>>>>>>>>> only
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> made
> >> > the
> >> > > >>>>> necessary
> >> > > >>>>>>>>> changes to
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
> plug-
> >> > in.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> >> > at
> >> > > >>> 1:39
> >> > > >>>>> PM, Mike
> >> > > >>>>>>>>>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> imply
> >> > the
> >> > > >>>>> XenServer code
> >> > > >>>>>> is
> >> > > >>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> >> > attach
> >> > > >>> logic is
> >> > > >>>>> now aware
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>> dynamic,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
> an
> >> > > >>>>> implementation of
> >> > > >>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>> kind of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
> to 4.2
> >> > > >>> with
> >> > > >>>>> Edison's
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> framework
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
> a
> >> > new
> >> > > >>>>> framework that
> >> > > >>>>>>>>>>> supported
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> >> > dynamically.
> >> > > >>>>> However, when I
> >> > > >>>>>>>>>>>>>>>>>> visited with
> >> > > >>>>>>>>>>>>>>>>>>>>>> him
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> realized
> >> > that
> >> > > >>> it
> >> > > >>>>> was not
> >> > > >>>>>>>>>>>>>>>>>> complete. We
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> CloudStack
> >> > could
> >> > > >>> do
> >> > > >>>>> with these
> >> > > >>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>> unless
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> >> > recognize
> >> > > >>> this
> >> > > >>>>> new type of
> >> > > >>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>> create
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >> > structure.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> >> > at
> >> > > >>> 1:28
> >> > > >>>>> PM, John
> >> > > >>>>>>>>> Burwell
> >> > > >>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
> odd
> >> > to me
> >> > > >>>>> that any
> >> > > >>>>>>>>>>> operation
> >> > > >>>>>>>>>>>>>>>>>> in the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
> care
> >> > > >>> about
> >> > > >>>>> details. I
> >> > > >>>>>>>>> expect
> >> > > >>>>>>>>>>>>>>>>>> to see
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
> of
> >> > > >>> operations
> >> > > >>>>> that can be
> >> > > >>>>>>>>>>>>>>>>>>>> composed/driven
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> implementations to
> >> > > >>>>> allocate
> >> > > >>>>>> space/create
> >> > > >>>>>>>>>>>>>>>>>> structures
> >> > > >>>>>>>>>>>>>>>>>>>>>> per
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
> this
> >> > > >>>>> dependency, we are
> >> > > >>>>>>>>> going
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> end
> >> > > >>>>>>>>>>>>>>>>>>>>>> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
> will
> >> > make
> >> > > >>> the
> >> > > >>>>> system
> >> > > >>>>>>>>>>>>> increasingly
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
> I
> >> > > >>>>> understanding that the
> >> > > >>>>>>>>> Xen
> >> > > >>>>>>>>>>>>>>>>>> specific
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >> > > >>>>> CitrixResourceBase
> >> > > >>>>>> class?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> > 2013 at
> >> > > >>> 3:12
> >> > > >>>>> PM, Mike
> >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
> this
> >> > in a
> >> > > >>> bit
> >> > > >>>>> more
> >> > > >>>>>>>>> detail:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
> and
> >> > aside
> >> > > >>> from
> >> > > >>>>> one setup
> >> > > >>>>>>>>>>>>> method
> >> > > >>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
> a
> >> > > >>> volume on
> >> > > >>>>> the
> >> > > >>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> system,
> >> > > >>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>> go
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
> data
> >> > > >>> structure
> >> > > >>>>> to make
> >> > > >>>>>> use
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> >> > > >>> XenServer
> >> > > >>>>> or a
> >> > > >>>>>> datastore
> >> > > >>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>> ESX(i)).
> >> > > >>>>>>>>>>>>>>>>>>>>>> VMs
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> >> > storage
> >> > > >>>>> system's volume.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
> new
> >> > > >>> storage
> >> > > >>>>> framework,
> >> > > >>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> need
> >> > > >>>>>>>>>>>>>>>>>>>> no
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
> easily
> >> > > >>> create
> >> > > >>>>> a 1:1
> >> > > >>>>>>>>>>>>> relationship
> >> > > >>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
> the
> >> > VM's
> >> > > >>> data
> >> > > >>>>> disk
> >> > > >>>>>>>>> (necessary
> >> > > >>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
> write
> >> > a
> >> > > >>> plug-
> >> > > >>>>> in that is
> >> > > >>>>>>>>> called
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> >> > needed.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
> that
> >> > the
> >> > > >>>>> storage
> >> > > >>>>>> framework
> >> > > >>>>>>>>> did
> >> > > >>>>>>>>>>>>>>>>>> not
> >> > > >>>>>>>>>>>>>>>>>>>>>> address
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >> > > >>> hypervisor-
> >> > > >>>>> specific data
> >> > > >>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>> when
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> attach/detach.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
> the
> >> > case,
> >> > > >>> I've
> >> > > >>>>> been
> >> > > >>>>>>>>> enhancing
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> to do
> >> > > >>>>>>>>>>>>>>>>>>>> so.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
> and
> >> > > >>>>> submitted. I've got
> >> > > >>>>>>>>> ESX(i)
> >> > > >>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> my
> >> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
> the
> >> > 4.2
> >> > > >>>>> freeze date.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> help a
> >> > bit? :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> 3,
> >> > 2013 at
> >> > > >>>>> 1:03 PM, Mike
> >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> >> > plug-in -
> >> > > >>> by
> >> > > >>>>> itself - is
> >> > > >>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
> with
> >> > the
> >> > > >>>>> volume-attach
> >> > > >>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>> (in the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> calls
> >> > into
> >> > > >>> the
> >> > > >>>>> plug-in to
> >> > > >>>>>>>>> have
> >> > > >>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>> create a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
> time
> >> > > >>> comes to
> >> > > >>>>> attach the
> >> > > >>>>>>>>> volume
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
> smart
> >> > > >>> enough
> >> > > >>>>> to recognize
> >> > > >>>>>>>>> it's
> >> > > >>>>>>>>>>>>>>>>>> being
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> >> > volume
> >> > > >>> has
> >> > > >>>>> just been
> >> > > >>>>>>>>> created)
> >> > > >>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>> create,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> (for
> >> > > >>>>> XenServer) or a
> >> > > >>>>>>>>> datastore
> >> > > >>>>>>>>>>>>>>>>>> (for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
> was
> >> > just
> >> > > >>>>> created.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> >> > spending
> >> > > >>> most
> >> > > >>>>> of my time
> >> > > >>>>>>>>>>>>> recently
> >> > > >>>>>>>>>>>>>>>>>> making
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> >> > code.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> clear it
> >> > up?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >> > 2013
> >> > > >>> at
> >> > > >>>>> 12:48 PM,
> >> > > >>>>>> John
> >> > > >>>>>>>>>>>>>>>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> explain
> >> > > >>> why
> >> > > >>>>> the the storage
> >> > > >>>>>>>>>>>>> driver
> >> > > >>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> 2013,
> >> > at
> >> > > >>> 1:21
> >> > > >>>>> PM, Mike
> >> > > >>>>>>>>>>>>> Tutkowski
> >> > > >>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> >> > ultimately I
> >> > > >>>>> would like to
> >> > > >>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>> all
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
> I
> >> > think
> >> > > >>> I'm
> >> > > >>>>> just out of
> >> > > >>>>>>>>> time
> >> > > >>>>>>>>>>>>>>>>>> for 4.2
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> now this
> >> > > >>> plug-in
> >> > > >>>>> supports
> >> > > >>>>>>>>>>>>> XenServer.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> to 4.2
> >> > > >>> feature
> >> > > >>>>> freeze, I
> >> > > >>>>>>>>> have
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> working
> >> > > >>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
> to
> >> > be
> >> > > >>> clear,
> >> > > >>>>> this is
> >> > > >>>>>> all
> >> > > >>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> regards
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > Compute
> >> > > >>>>> Offerings post
> >> > > >>>>>> 4.2.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >> > > >>> 2013
> >> > > >>>>> at 11:14 AM,
> >> > > >>>>>>>>> Kelcey
> >> > > >>>>>>>>>>>>>>>>>> Jamison
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> kelcey@bbits.ca<ma...@bbits.ca>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
> any
> >> > plan
> >> > > >>> on
> >> > > >>>>> supporting
> >> > > >>>>>>>>> KVM in
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> patch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> Original
> >> > > >>>>> Message -----
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
> "Mike
> >> > > >>>>> Tutkowski" <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >> > > >>>>> dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> >> > Monday,
> >> > > >>> June
> >> > > >>>>> 3, 2013
> >> > > >>>>>>>>> 10:12:32 AM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
> Re:
> >> > > >>> [MERGE]
> >> > > >>>>>>>>>>> disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>> MASTER
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
> on
> >> > > >>> merging
> >> > > >>>>> Wei's feature
> >> > > >>>>>>>>>>> first,
> >> > > >>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>> mine.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
> feature
> >> > is
> >> > > >>> for
> >> > > >>>>> KVM only,
> >> > > >>>>>>>>> then
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> is a
> >> > > >>>>>>>>>>>>>>>>>>>> non
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
> 4.2.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun
> >> > 3,
> >> > > >>> 2013
> >> > > >>>>> at 8:55 AM,
> >> > > >>>>>> Wei
> >> > > >>>>>>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> ustcweizhou@gmail.com<ma...@gmail.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> the
> >> > billing,
> >> > > >>> as
> >> > > >>>>> no one works
> >> > > >>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>> billing
> >> > > >>>>>>>>>>>>>>>>>>>> now,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> billing
> >> > by
> >> > > >>>>> themselves. They
> >> > > >>>>>>>>> can
> >> > > >>>>>>>>>>>>> get
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > disk_offering of
> >> > > >>> a
> >> > > >>>>> VMs and
> >> > > >>>>>> volumes
> >> > > >>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
> user
> >> > the
> >> > > >>>>> exact
> >> > > >>>>>> limitation
> >> > > >>>>>>>>>>>>> value
> >> > > >>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >> > limitation
> >> > > >>> for
> >> > > >>>>> nics as
> >> > > >>>>>> well.
> >> > > >>>>>>>>> I
> >> > > >>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>> work
> >> > > >>>>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> part of
> >> > I/O
> >> > > >>>>> throttling?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
> my
> >> > > >>>>> misunstand the second
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
> with
> >> > > >>> what
> >> > > >>>>> you said about
> >> > > >>>>>> the
> >> > > >>>>>>>>>>> two
> >> > > >>>>>>>>>>>>>>>>>>>> features.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/6/3
> >> > John
> >> > > >>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Jun 3,
> >> > 2013,
> >> > > >>> at
> >> > > >>>>> 2:13 AM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com<ma...@gmail.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> John,
> >> > Mike
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> hope
> >> > Mike's
> >> > > >>>>> aswer helps you.
> >> > > >>>>>>>>> I am
> >> > > >>>>>>>>>>>>>>>>>> trying
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> >> > think
> >> > > >>> billing
> >> > > >>>>> should
> >> > > >>>>>>>>> depend
> >> > > >>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>> IO
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > limitation.
> >> > > >>> Please
> >> > > >>>>> review
> >> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
> >> > > >>>>>>>>>>>>>>>>>>>> you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk_io_stat
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> the
> >> > IO
> >> > > >>>>> statistics
> >> > > >>>>>> including
> >> > > >>>>>>>>>>>>>>>>>> bytes/iops
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> individual
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> virtual
> >> > > >>> machine.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Going by
> >> > the
> >> > > >>> AWS
> >> > > >>>>> model,
> >> > > >>>>>> customers
> >> > > >>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>> billed
> >> > > >>>>>>>>>>>>>>>>>>>>>> more
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > provisioned
> >> > > >>> IOPS,
> >> > > >>>>> as well as,
> >> > > >>>>>> for
> >> > > >>>>>>>>>>>>> those
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://aws.amazon.com/ebs/).
> >> > > >>>>>> I
> >> > > >>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>> imagine
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> option to
> >> > > >>> employ
> >> > > >>>>> similar cost
> >> > > >>>>>>>>>>> models.
> >> > > >>>>>>>>>>>>>>>>>> Could
> >> > > >>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> billing
> >> > model
> >> > > >>> in
> >> > > >>>>> the current
> >> > > >>>>>>>>> patch?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> Do
> >> > you
> >> > > >>> mean
> >> > > >>>>> IOPS runtime
> >> > > >>>>>>>>>>> change?
> >> > > >>>>>>>>>>>>>>>>>> KVM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > limitation for
> >> > > >>> a
> >> > > >>>>> running
> >> > > >>>>>> virtual
> >> > > >>>>>>>>>>>>>>>>>> machine
> >> > > >>>>>>>>>>>>>>>>>>>>>> through
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> However,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > CloudStack
> >> > > >>> does
> >> > > >>>>> not support
> >> > > >>>>>>>>>>> changing
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> offering
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > (computer
> >> > > >>>>> offering or disk
> >> > > >>>>>>>>>>>>> offering).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> meant at
> >> > the
> >> > > >>>>> Java interface
> >> > > >>>>>>>>> level.
> >> > > >>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>> apologize
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> more
> >> > > >>>>> generalize allocation
> >> > > >>>>>>>>>>>>>>>>>> algorithms
> >> > > >>>>>>>>>>>>>>>>>>>> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> describe
> >> > the
> >> > > >>>>> service guarantees
> >> > > >>>>>>>>>>>>>>>>>> provided by a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> It is a
> >> > > >>> good
> >> > > >>>>> question.
> >> > > >>>>>>>>> Maybe it
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>> better
> >> > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> disk_io_throttling as Mike
> >> > > >>>>>>>>> needs to
> >> > > >>>>>>>>>>>>>>>>>> consider
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> type, I
> >> > think.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> >> > expand
> >> > > >>> on
> >> > > >>>>> my thoughts
> >> > > >>>>>> in a
> >> > > >>>>>>>>>>>>> later
> >> > > >>>>>>>>>>>>>>>>>>>>>> response
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> touch
> >> > points
> >> > > >>>>> between these two
> >> > > >>>>>>>>>>>>>>>>>> features. I
> >> > > >>>>>>>>>>>>>>>>>>>>>> think
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> need
> >> > to be
> >> > > >>>>> merged before
> >> > > >>>>>>>>>>>>>>>>>> SolidFire, but
> >> > > >>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > coordination
> >> > > >>>>> between the
> >> > > >>>>>> branches
> >> > > >>>>>>>>>>>>>>>>>> (possibly
> >> > > >>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> disk_io_throttling)
> >> > > >>>>> to
> >> > > >>>>>>>>> coordinate on
> >> > > >>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>> issue.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/6/3
> >> > > >>> John
> >> > > >>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >> > things I
> >> > > >>>>> want to
> >> > > >>>>>> understand
> >> > > >>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
> Is
> >> > there
> >> > > >>>>> value in
> >> > > >>>>>> capturing
> >> > > >>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>> policies
> >> > > >>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> common
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> data
> >> > model
> >> > > >>> (e.g.
> >> > > >>>>> for
> >> > > >>>>>>>>> billing/usage
> >> > > >>>>>>>>>>>>>>>>>>>> purposes,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> >> > Should
> >> > > >>> there
> >> > > >>>>> be a common
> >> > > >>>>>>>>>>>>>>>>>> interface model
> >> > > >>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > provisioning
> >> > > >>> at
> >> > > >>>>> runtime?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
> How
> >> > are
> >> > > >>>>> conflicting
> >> > > >>>>>>>>> provisioned
> >> > > >>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > hypervisor
> >> > > >>> and
> >> > > >>>>> storage device
> >> > > >>>>>>>>>>>>>>>>>> reconciled?
> >> > > >>>>>>>>>>>>>>>>>>>> In
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> where
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> lead
> >> > to
> >> > > >>>>> believe (and
> >> > > >>>>>> billed)
> >> > > >>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>> more
> >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> storage
> >> > > >>> device
> >> > > >>>>> has been
> >> > > >>>>>>>>> configured
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > consistent
> >> > > >>>>> configuration
> >> > > >>>>>>>>> between a
> >> > > >>>>>>>>>>>>>>>>>> VM and a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> time,
> >> > but a
> >> > > >>>>> later
> >> > > >>>>>> modification
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> inconsistency.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Jun
> >> > 2,
> >> > > >>> 2013,
> >> > > >>>>> at 8:38 PM,
> >> > > >>>>>>>>> Mike
> >> > > >>>>>>>>>>>>>>>>>> Tutkowski
> >> > > >>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > believe
> >> > > >>> Wei's
> >> > > >>>>> feature deals
> >> > > >>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>> controlling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >> > > >>> hypervisor
> >> > > >>>>> side.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> My
> >> > feature
> >> > > >>> is
> >> > > >>>>> focused on
> >> > > >>>>>>>>>>>>> controlling
> >> > > >>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> hope
> >> > that
> >> > > >>>>> helps. :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Sun,
> >> > Jun
> >> > > >>> 2,
> >> > > >>>>> 2013 at 6:35
> >> > > >>>>>> PM,
> >> > > >>>>>>>>>>>>> John
> >> > > >>>>>>>>>>>>>>>>>>>> Burwell
> >> > > >>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > jburwell@basho.com<ma...@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> My
> >> > > >>> opinion is
> >> > > >>>>> that no
> >> > > >>>>>> features
> >> > > >>>>>>>>>>>>>>>>>> should be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> functional
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> issues
> >> > > >>> have
> >> > > >>>>> been resolved
> >> > > >>>>>> and
> >> > > >>>>>>>>> it
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>> ready
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> total
> >> > Ops
> >> > > >>> vs
> >> > > >>>>> discrete
> >> > > >>>>>>>>> read/write
> >> > > >>>>>>>>>>>>>>>>>> ops issue
> >> > > >>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> >> > reviewed
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > I
> >> > > >>> don't
> >> > > >>>>> think this
> >> > > >>>>>>>>> criteria
> >> > > >>>>>>>>>>>>>>>>>> has been
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Also,
> >> > how
> >> > > >>>>> does this work
> >> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >> > > >>>>>>>>>>>>>>>>>> )?
> >> > > >>>>>>>>>>>>>>>>>>>> As I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> also
> >> > > >>> involves
> >> > > >>>>> provisioned
> >> > > >>>>>>>>> IOPS. I
> >> > > >>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>> like
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > scenario
> >> > > >>>>> where provisioned
> >> > > >>>>>>>>> IOPS
> >> > > >>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> KVM and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > unnecessarily
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> incompatible.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > Jun 1,
> >> > > >>> 2013,
> >> > > >>>>> at 6:47 AM,
> >> > > >>>>>>>>> Wei
> >> > > >>>>>>>>>>>>>>>>>> ZHOU <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> ustcweizhou@gmail.com<ma...@gmail.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Sure. I
> >> > will
> >> > > >>>>> change it next
> >> > > >>>>>>>>> week.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > 2013/6/1
> >> > > >>>>> Wido den Hollander
> >> > > >>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl<ma...@widodh.nl>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 06/01/2013 08:24 AM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Exactly. I
> >> > > >>> have
> >> > > >>>>> pushed the
> >> > > >>>>>>>>>>>>> features
> >> > > >>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > anyone
> >> > > >>>>> object thems for
> >> > > >>>>>>>>>>>>> technical
> >> > > >>>>>>>>>>>>>>>>>>>> reason
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> them.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> For
> >> > the
> >> > > >>> sake
> >> > > >>>>> of clarity I
> >> > > >>>>>> just
> >> > > >>>>>>>>>>>>> want
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> mention
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> change
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >> > total
> >> > > >>> IOps
> >> > > >>>>> to R/W IOps
> >> > > >>>>>>>>> asap
> >> > > >>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>> that we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> only
> >> > total
> >> > > >>> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> You
> >> > laid
> >> > > >>> the
> >> > > >>>>> groundwork for
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>> I/O
> >> > > >>>>>>>>>>>>>>>>>>>>>> throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> should
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > however
> >> > > >>>>> prevent that we
> >> > > >>>>>> create
> >> > > >>>>>>>>>>>>>>>>>> legacy from
> >> > > >>>>>>>>>>>>>>>>>>>>>> day
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > 2013/5/31
> >> > > >>>>> Wido den
> >> > > >>>>>> Hollander <
> >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl<ma...@widodh.nl>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 05/31/2013 03:59 PM, John
> >> > > >>>>>>>>>>>>> Burwell
> >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> +1 --
> >> > this
> >> > > >>>>> enhancement must
> >> > > >>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> discretely
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> don't
> >> > see
> >> > > >>> how
> >> > > >>>>> it could be
> >> > > >>>>>>>>> fixed
> >> > > >>>>>>>>>>>>>>>>>> later
> >> > > >>>>>>>>>>>>>>>>>>>>>> because
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > correctly
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> split
> >> > total
> >> > > >>> IOPS
> >> > > >>>>> into read
> >> > > >>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> write.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> total
> >> > > >>>>> unless/until we
> >> > > >>>>>> decided
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> break
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> What
> >> > Wei
> >> > > >>>>> meant was merging
> >> > > >>>>>> it
> >> > > >>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>> master
> >> > > >>>>>>>>>>>>>>>>>>>>>> now
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 4.2
> >> > branch
> >> > > >>>>> and add Read /
> >> > > >>>>>>>>> Write
> >> > > >>>>>>>>>>>>> IOps
> >> > > >>>>>>>>>>>>>>>>>>>> before
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> will be
> >> > > >>>>> released with Read
> >> > > >>>>>> and
> >> > > >>>>>>>>>>>>> Write
> >> > > >>>>>>>>>>>>>>>>>>>> instead
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> This is
> >> > to
> >> > > >>>>> make the May 31st
> >> > > >>>>>>>>>>>>> feature
> >> > > >>>>>>>>>>>>>>>>>>>> freeze
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> moves
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (see
> >> > other
> >> > > >>>>> threads) then it
> >> > > >>>>>>>>> won't
> >> > > >>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>> necessary
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> also
> >> > > >>>>> completely agree that
> >> > > >>>>>>>>>>> there
> >> > > >>>>>>>>>>>>>>>>>> is no
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk
> >> > I/O.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > May 31,
> >> > > >>>>> 2013, at 9:51 AM,
> >> > > >>>>>>>>> Wido
> >> > > >>>>>>>>>>>>>>>>>> den
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wido@widodh.nl<ma...@widodh.nl>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 05/31/2013 03:13 PM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi
> >> > Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Thanks.
> >> > > >>> Good
> >> > > >>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > thought
> >> > > >>>>> about at the
> >> > > >>>>>>>>>>> beginning.
> >> > > >>>>>>>>>>>>>>>>>>>> Finally I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > difference
> >> > > >>> of
> >> > > >>>>> read and write
> >> > > >>>>>>>>>>>>> mainly
> >> > > >>>>>>>>>>>>>>>>>>>> because
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> not
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> care
> >> > the
> >> > > >>>>> difference of sent
> >> > > >>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> received
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> That
> >> > > >>>>> reasoning seems odd.
> >> > > >>>>>>>>>>>>>>>>>> Networking and
> >> > > >>>>>>>>>>>>>>>>>>>>>> disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> different.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Disk
> >> > I/O is
> >> > > >>>>> much more
> >> > > >>>>>>>>> expensive
> >> > > >>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> most
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > bandwith.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> Implementing
> >> > > >>>>> it will be some
> >> > > >>>>>>>>>>>>>>>>>> copy-paste
> >> > > >>>>>>>>>>>>>>>>>>>>>> work.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> implemented
> >> > > >>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> few
> >> > days.
> >> > > >>> For
> >> > > >>>>> the deadline
> >> > > >>>>>> of
> >> > > >>>>>>>>>>>>>>>>>> feature
> >> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> after
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> that ,
> >> > if
> >> > > >>>>> needed.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >> > think it's
> >> > > >>> a
> >> > > >>>>> feature we
> >> > > >>>>>>>>> can't
> >> > > >>>>>>>>>>>>>>>>>> miss. But
> >> > > >>>>>>>>>>>>>>>>>>>>>> if
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > window
> >> > > >>> we
> >> > > >>>>> have to make sure
> >> > > >>>>>> we
> >> > > >>>>>>>>>>>>> don't
> >> > > >>>>>>>>>>>>>>>>>>>> release
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> fix
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> in
> >> > 4.3,
> >> > > >>> that
> >> > > >>>>> will confuse
> >> > > >>>>>>>>>>>>> users.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > 2013/5/31
> >> > > >>>>> Wido den
> >> > > >>>>>> Hollander <
> >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl<ma...@widodh.nl>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 05/30/2013 06:03 PM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > would
> >> > > >>> like to
> >> > > >>>>> merge
> >> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > nobody
> >> > > >>>>> object, I will
> >> > > >>>>>> merge
> >> > > >>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>> master
> >> > > >>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >> > > >>> purpose
> >> > > >>>>> is :
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Virtual
> >> > > >>>>> machines are running
> >> > > >>>>>>>>> on
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> same
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > storage or
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> share
> >> > > >>> strage).
> >> > > >>>>> Because of
> >> > > >>>>>> the
> >> > > >>>>>>>>>>> rate
> >> > > >>>>>>>>>>>>>>>>>>>>>> limitation
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> iops),
> >> > if
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> one
> >> > VM
> >> > > >>> has
> >> > > >>>>> large disk
> >> > > >>>>>>>>> operation,
> >> > > >>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> may
> >> > > >>>>>>>>>>>>>>>>>>>>>> affect
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> performance
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> other
> >> > VMs
> >> > > >>>>> running on the
> >> > > >>>>>> same
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>> device.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> is
> >> > > >>> neccesary
> >> > > >>>>> to set the
> >> > > >>>>>>>>>>> maximum
> >> > > >>>>>>>>>>>>>>>>>> rate
> >> > > >>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> VMs.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Looking at
> >> > > >>> the
> >> > > >>>>> code I see
> >> > > >>>>>> you
> >> > > >>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>> no
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Qemu
> >> > and
> >> > > >>>>> libvirt support
> >> > > >>>>>>>>> setting
> >> > > >>>>>>>>>>>>>>>>>> both a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps
> >> > > >>> which
> >> > > >>>>> could benefit a
> >> > > >>>>>>>>> lot of
> >> > > >>>>>>>>>>>>>>>>>> users.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> It's
> >> > also
> >> > > >>>>> strange, in the
> >> > > >>>>>>>>> polling
> >> > > >>>>>>>>>>>>>>>>>> side you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps,
> >> > but
> >> > > >>> on
> >> > > >>>>> the throttling
> >> > > >>>>>>>>> side
> >> > > >>>>>>>>>>>>>>>>>> you only
> >> > > >>>>>>>>>>>>>>>>>>>> go
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > IOps
> >> > > >>> are
> >> > > >>>>> usually much
> >> > > >>>>>>>>> more
> >> > > >>>>>>>>>>>>>>>>>> expensive
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> seems
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> like a
> >> > valid
> >> > > >>>>> use-case where
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>> admin
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps
> >> > vs
> >> > > >>> Read
> >> > > >>>>> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Since
> >> > this
> >> > > >>> only
> >> > > >>>>> supports KVM
> >> > > >>>>>>>>> at
> >> > > >>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>> point I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> great
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> value
> >> > to at
> >> > > >>>>> least have the
> >> > > >>>>>>>>>>>>>>>>>> mechanism in
> >> > > >>>>>>>>>>>>>>>>>>>>>> place
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> implementing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> this
> >> > later
> >> > > >>>>> would be a lot of
> >> > > >>>>>>>>>>> work.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> a
> >> > > >>> hypervisor
> >> > > >>>>> doesn't
> >> > > >>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>> setting
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> write
> >> > you
> >> > > >>> can
> >> > > >>>>> always sum
> >> > > >>>>>> both
> >> > > >>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>> and set
> >> > > >>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Can
> >> > you
> >> > > >>>>> explain why you
> >> > > >>>>>>>>>>>>> implemented
> >> > > >>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >> > > >>> feature
> >> > > >>>>> includes:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (1) set
> >> > the
> >> > > >>>>> maximum rate of
> >> > > >>>>>>>>> VMs
> >> > > >>>>>>>>>>>>> (in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> configuration)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (2)
> >> > change
> >> > > >>> the
> >> > > >>>>> maximum rate
> >> > > >>>>>> of
> >> > > >>>>>>>>>>> VMs
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (3)
> >> > limit
> >> > > >>> the
> >> > > >>>>> disk rate
> >> > > >>>>>> (total
> >> > > >>>>>>>>>>> bps
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>> iops)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> JIRA
> >> > ticket:
> >> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> issues.apache.org/****<http://issues.apache.org/****>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> jira/browse/CLOUDSTACK-1192<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1192>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> issues.apache.org/**jira/**browse/CLOUDSTACK-<http://issues.apache.org/**jira/**browse/CLOUDSTACK->
> >> > > >>> 1192<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >> > 1192
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1192<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >> > 1192>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> FS (I
> >> > will
> >> > > >>>>> update later) :
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>> VM+Disk+IO+Throttling<https://
<https://%0b>> >> <https://%0b/>> > >>>>>>>>>>>>> ****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cwiki.apache.org/confluence/****<http://cwiki.apache.org/confluence/****>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<http://apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >> > > >>>>> rottling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Merge
> >> > > >>> check
> >> > > >>>>> list :-
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> Did
> >> > you
> >> > > >>>>> check the branch's
> >> > > >>>>>>>>> RAT
> >> > > >>>>>>>>>>>>>>>>>> execution
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Yes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> Are
> >> > > >>> there
> >> > > >>>>> new dependencies
> >> > > >>>>>>>>>>>>>>>>>> introduced?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> No
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >> > What
> >> > > >>>>> automated testing
> >> > > >>>>>> (unit
> >> > > >>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> new
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > feature?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Unit
> >> > tests
> >> > > >>> are
> >> > > >>>>> added.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >> > What
> >> > > >>>>> testing has been done
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> check for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (1) set
> >> > the
> >> > > >>>>> bytes rate and
> >> > > >>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>> rate on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (2)
> >> > VM
> >> > > >>>>> operations, including
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > deploy,
> >> > > >>> stop,
> >> > > >>>>> start, reboot,
> >> > > >>>>>>>>>>>>>>>>>> destroy,
> >> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (3)
> >> > > >>> Volume
> >> > > >>>>> operations,
> >> > > >>>>>>>>> including
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Attach,
> >> > > >>>>> Detach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> To
> >> > review
> >> > > >>> the
> >> > > >>>>> code, you can
> >> > > >>>>>>>>> try
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> git diff
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Best
> >> > > >>> regards,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [1]
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>> VM+Disk+IO+Throttling<https://
<https://%0b>> >> <https://%0b/>> > >>>>>>>>>>>>> ****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cwiki.apache.org/confluence/****<http://cwiki.apache.org/confluence/****>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<http://apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >> > > >>>>> rottling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [2]
> >> > > >>>>>>>>> refs/heads/disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [3]
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> 1301
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >> > > >>>>> 1301
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> issues.apache.org/****jira/browse/CLOUDSTACK-<http://issues.apache.org/****jira/browse/CLOUDSTACK->
> >> > > >>> 1301<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1301>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> issues.apache.org/**jira/**browse/CLOUDSTACK-<http://issues.apache.org/**jira/**browse/CLOUDSTACK->
> >> > > >>> 1301<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >> > 1301
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1301<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >> > 1301>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > issues.apache.org/****jira/**browse/CLOUDSTACK-<http://issues.apache.org/****jira/**browse/CLOUDSTACK->
> >> > > >>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> > > >>>>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> **<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> > > >>>>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >> > 2071
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >> > > >>>>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 2071>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <h**ttps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071><
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >> > 2071>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> (**CLOUDSTACK-1301
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> > VM
> >> > > >>> Disk
> >> > > >>>>> I/O
> >> > > >>>>>> Throttling)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Mike
> >> > > >>>>> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >> > > >>>>> CloudStack Developer,
> >> > > >>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> > > >>> 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Advancing
> >> > > >>> the
> >> > > >>>>> way the world
> >> > > >>>>>>>>> uses
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> > > >>> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > > >>> CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> > 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> >> > the
> >> > > >>> way
> >> > > >>>>> the world uses
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> > Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > > >>> CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> > 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing the
> >> > > >>> way
> >> > > >>>>> the world uses
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> > Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >> > way
> >> > > >>> the
> >> > > >>>>> world uses the
> >> > > >>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>> SolidFire
> >> > > >>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >> > way
> >> > > >>> the
> >> > > >>>>> world uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >> > > >>>>> Developer, SolidFire
> >> > > >>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> >> > the
> >> > > >>>>> world uses the
> >> > > >>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >> > > >>> Developer,
> >> > > >>>>> SolidFire
> >> > > >>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> >> > the
> >> > > >>> world
> >> > > >>>>> uses the
> >> > > >>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >> > > >>> Developer,
> >> > > >>>>> SolidFire
> >> > > >>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way the
> >> > > >>> world
> >> > > >>>>> uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> > Developer,
> >> > > >>>>> SolidFire
> >> > > >>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> >> > world
> >> > > >>> uses
> >> > > >>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> > Developer,
> >> > > >>>>> SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> > world
> >> > > >>> uses
> >> > > >>>>> the cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >> > > >>>>> SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >> > uses
> >> > > >>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >> > > >>> SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >> > uses
> >> > > >>> the
> >> > > >>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >> > > >>> SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> > the
> >> > > >>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> > > >>> SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> > the
> >> > > >>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> > SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> > the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> > SolidFire
> >> > > >>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >> > > >>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >> > > >>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >> > Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >> > Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> --
> >> > > >>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> --
> >> > > >>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>>> o: 303.746.7302
> >> > > >>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>> *(tm)*
> >> > > >>>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> --
> >> > > >>>>>>> *Mike Tutkowski*
> >> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>>>> o: 303.746.7302
> >> > > >>>>>>> Advancing the way the world uses the
> >> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>> *(tm)*
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> --
> >> > > >>>>> *Mike Tutkowski*
> >> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>>>> o: 303.746.7302
> >> > > >>>>> Advancing the way the world uses the
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>> *(tm)*
> >> > > >>>>
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>> --
> >> > > >>> *Mike Tutkowski*
> >> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > >>> o: 303.746.7302
> >> > > >>> Advancing the way the world uses the
> >> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>> *(tm)*
> >> > > >>
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > *Mike Tutkowski*
> >> > > > *Senior CloudStack Developer, SolidFire Inc.*
> >> > > > e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > > > o: 303.746.7302
> >> > > > Advancing the way the world uses the
> >> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > > *(tm)*
> >> > >
> >> > >
> >> >
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *(tm)*****
> >>
> >>
> >>
> >> ****
> >>
> >> ** **
> >>
> >> --
> >> *Mike Tutkowski*****
> >>
> >> *Senior CloudStack Developer, SolidFire Inc.*****
> >>
> >> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>****
> >>
> >> o: 303.746.7302****
> >>
> >> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*****
> >>
> >>
> >>
> >> ****
> >>
> >> ** **
> >>
> >> --
> >> *Mike Tutkowski*****
> >>
> >> *Senior CloudStack Developer, SolidFire Inc.*****
> >>
> >> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>****
> >>
> >> o: 303.746.7302****
> >>
> >> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*****
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > o: 303.746.7302
> > Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
> >
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
--
Mike Tutkowski
Senior CloudStack Developer, SolidFire Inc.
e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
o: 303.746.7302
Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm)
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Well, that's sort of how I did it in the code that I originally submitted
(except I labelled the storage as Dynamic and that was how the hypervisor
knew what to do). :)
On Mon, Jun 10, 2013 at 12:58 PM, Edison Su <Ed...@citrix.com> wrote:
> I don't understand why we need to add a "managed" in the UI and in
> "storage_pool" table?
> AFAIK, what you guys trying to do is to enable attachVolume/DettachVolume
> if the volume is created on Solidfire, right?
> If the volume is created on Solidfire already, by createAsync, then why we
> can't just send out all the information about the volume(it's volumeTO
> which will be returned by each storage driver) to hypervisor, then at the
> hypervisor side, check the volumeTO, if the volumeTO is managed, then doing
> whatever necessary operations(create SR, create VDI etc), then attach the
> VDI to volume.
>
>
> > -----Original Message-----
> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > Sent: Monday, June 10, 2013 11:10 AM
> > To: John Burwell
> > Cc: Edison Su; dev@cloudstack.apache.org
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > I believe we're on the same page, John.
> >
> > I would add a new parameter to the createStoragePool API command (let's
> > call it "managed" and it can be true or false). If you pass in "true",
> but
> > the driver does not support being managed by CloudStack, an exception is
> > thrown by the driver. If you pass in "false", but the driver does not
> > support being managed outside of CloudStack, an exception is thrown by
> the
> > driver. It is possible one could write a driver that can be managed both
> > inside or outside of CloudStack (the SolidFire driver requires being
> > managed only by CloudStack).
> >
> > This managed value would end up in the storage_pool table in a new column
> > ("managed").
> >
> > An example usage: After the storage framework calls on the driver's
> > createAsync method, it will check to see if the driver is managed. If the
> > driver is managed, the storage framework will invoke the proper (new)
> > command of the hypervisor in use to set up, say, the SR for the SAN
> volume.
> >
> > There may be other cases when the isManaged() method is invoked on the
> > driver and - if true is returned - the storage framework sends a
> particular
> > command to the hypervisor in use.
> >
> > ********************
> >
> > As far as reworking the default storage plug-in (which sends commands to
> > hypervisors) for 4.2, I would check with Edison on his schedule.
> >
> > Thanks!
> >
> >
> > On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
> > wrote:
> >
> > > Mike,
> > >
> > > I want to make sure we are on the same page regarding isManaged. I see
> > > the addition of the following operations:
> > >
> > > * DataStoreDriver#supportsManagement() : boolean -- indicates whether
> > or
> > > not the driver support/implements the management functions
> > > * DataStore#isManaged() : boolean -- indicates whether or not a device
> is
> > > managed by CloudStack
> > >
> > > Only DataStores associated with DataStoreDrivers that support
> > management
> > > would be able to have the management flag enabled. This behavior
> allows
> > > operators to declare their intention to allow CloudStack to manage a
> > > device. This behavior would need to be exposed as flag on the HTTP
> API.
> > > Additionally, this flag would only be settable when a device is
> > > defined/created in the system (i.e. once a device is managed by
> CloudStack,
> > > you can't unmanaged it without dissociating it from CloudStack). In
> the
> > > future, we may consider relaxing that rule, but, in the short term, it
> > > greatly decreases the alternative flows.
> > >
> > > In terms of removing the dependencies from Storage to Hypervisor, I
> > would
> > > really like to see this accomplished in 4.2. It will be much more
> > > difficult to address this issue post 4.2 because we will have more
> plugins
> > > to address. I also see this issue as a critical architectural issues
> that
> > > for the management server. I recommend we identify the dependencies
> > that
> > > need to be inverted, and see if we can divide the work across a few
> more
> > > resources.
> > >
> > > Thanks,
> > > -John
> > >
> > > On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> > <mi...@solidfire.com>
> > > wrote:
> > >
> > > Sounds good, Edison :)
> > >
> > > I can implement John's isManaged() logic and resubmit my code for
> review.
> > >
> > > Perhaps in 4.3 I could address the direct-attach scenario. If it
> requires
> > > some refactoring of the storage framework, then I could do that.
> > >
> > > Also, the default storage plug-in does have hypervisor logic in it in
> 4.2.
> > > I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> > > with the above work I mentioned.
> > >
> > > Are you OK with that, guys?
> > >
> > >
> > > On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
> wrote:
> > >
> > >> Ok, I am Ok with the way you are talking about in 4.2.****
> > >>
> > >> ** **
> > >>
> > >> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> *Sent:* Sunday, June 09, 2013 8:59 PM
> > >> *To:* dev@cloudstack.apache.org
> > >> *Cc:* Edison Su; John Burwell
> > >> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> > >>
> > >> ** **
> > >>
> > >> Another thing we might want to consider for 4.2 is that the way the
> code
> > >> is currently implemented in master, I don't think it's even possible
> to
> > >> write a plug-in that connects storage directly to a VM because the
> > >> CloudStack storage framework will call the "attach" method of the
> > >> hypervisor in use and this method will fail because it assumes there
> is
> > >> (talking Xen here) an SR existent and in this situation there isn't
> one.*
> > >> ***
> > >>
> > >> ** **
> > >>
> > >> My thinking is that we should ignore the direct-attach-of-volume-to-VM
> > >> use case for 4.2. I think it will require too much of a rewrite at
> this
> > >> point in development (fewer than three weeks left until code
> > freeze).****
> > >>
> > >> ** **
> > >>
> > >> If the plug-in implements an isManaged() method (as John suggested),
> > the
> > >> storage framework can invoke the necessary commands on the
> > hypervisor in
> > >> use.****
> > >>
> > >> ** **
> > >>
> > >> For example, the createAsync method is called. The storage plug-in
> > >> creates a SAN volume and updates the CS DB. The storage framework
> > then asks
> > >> the driver if it's managed. If it is, the storage framework executes a
> > >> command against the hypervisor in use to have it create, say, an SR
> (with
> > a
> > >> single VDI that takes up the entire SR). Now, when the storage
> > framework
> > >> executes the "attach" command against the hypervisor, this command
> will
> > >> work, as well (because that logic assumes the prior existence of an
> SR and
> > >> it will, in fact, be present).****
> > >>
> > >> ** **
> > >>
> > >> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com> wrote:****
> > >>
> > >> Hi Edison and John,****
> > >>
> > >> ** **
> > >>
> > >> I wanted to point out something I thought was relevant to this
> > >> conversation.****
> > >>
> > >> ** **
> > >>
> > >> My plug-in supports both XenServer and ESX(i).****
> > >>
> > >> ** **
> > >>
> > >> These two hypervisors handle their respective hypervisor data
> structures
> > >> (XenServer SR or ESX(i) datastore) quite differently.****
> > >>
> > >> ** **
> > >>
> > >> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> > >> when you issue a remove-SR command to XenServer.****
> > >>
> > >> ** **
> > >>
> > >> When you later want to access the contents of that data, it is still
> > >> safely stored on the LUN and can be accessed from XenServer (from the
> > same
> > >> or different cluster (what XenServer calls a resource pool)) by
> creating a
> > >> new SR that is based on the same IQN and LUN.****
> > >>
> > >> ** **
> > >>
> > >> XenServer SRs exist at the cluster level.****
> > >>
> > >> ** **
> > >>
> > >> In ESX(i), the datastore that resides on the SAN LUN is actually
> > >> destroyed when you issue a remove-datastore command to ESX(i). In
> > other
> > >> words, the contents of the SAN LUN are destroyed.****
> > >>
> > >> ** **
> > >>
> > >> ESX(i) datastores exist at the datacenter level.****
> > >>
> > >> ** **
> > >>
> > >> The XenServer behavior (not destroying the contents of the LUN when
> > the
> > >> SR is removed) is desirable; the ESX(i) behavior (destroying the
> contents
> > >> of the LUN when the datastore is removed) is not desirable.****
> > >>
> > >> ** **
> > >>
> > >> I can get around the ESX(i) behavior by removing the referenced IQN
> > from
> > >> each host in the cluster when the CS storage framework notifies my
> plug-
> > in
> > >> of a detach-volume event. When the CS volume is later attached again
> > >> (either to the same or a different ESX(i) cluster), I can have each
> host in
> > >> the relevant ESX(i) cluster reference the IQN and the datastore will
> be
> > >> back again.****
> > >>
> > >> ** **
> > >>
> > >> This sequence of events is critical for my plug-in to support both
> > >> environments:****
> > >>
> > >> *XenServer flow (events for Driver class):*****
> > >>
> > >> createAsync (called right before first attach) = creates SAN
> volume****
> > >>
> > >> preAttachVolume = creates SR and creates VDI or just introduces SR****
> > >>
> > >> (CS volume gets attached and later detached)****
> > >>
> > >> postDetachVolume = removes SR (this does not delete data on SAN
> > volume)**
> > >> **
> > >>
> > >> deleteAsync (can be called when volume is not attached) = deletes the
> > SAN
> > >> volume****
> > >>
> > >> *VMware flow (events for Driver class):*****
> > >>
> > >> createAsync (called right before first attach) = creates SAN volume,
> > >> creates datastore, and creates VMDK file****
> > >>
> > >> preAttachVolume = adds iSCSI target to each host in cluster****
> > >>
> > >> (CS volume gets attached and later detached)****
> > >>
> > >> postDetachVolume = removes iSCSI target from each host in cluster****
> > >>
> > >> deleteAsync (can be called when volume is not attached) = deletes
> > >> datastore and deletes the SAN volume****
> > >>
> > >> ************************
> > >>
> > >> So, the way I have it written up here, the driver is talking to the
> > >> hypervisor. As John has pointed out, this may not be ideal.****
> > >>
> > >> ** **
> > >>
> > >> Whatever we decide on, it needs to be able to support this kind of
> flow.
> > >> :)****
> > >>
> > >> ** **
> > >>
> > >> We could go John's route, where the driver says if it's managed or
> not.**
> > >> **
> > >>
> > >> ** **
> > >>
> > >> If it is, then the storage framework could send the necessary
> messages to
> > >> the hypervisor in question at the right times (just migrate the
> hypervisor
> > >> logic above from the driver to the right parts of the storage
> framework).
> > >> ****
> > >>
> > >> ** **
> > >>
> > >> This is fine, except that it does not resolve Edison's use case of
> > >> desiring to have the storage LUN potentially connected directly to a
> VM.*
> > >> ***
> > >>
> > >> ** **
> > >>
> > >> I'm not sure we should design around that use case, though. Is it
> common
> > >> enough? Has anyone asked for it? Is it even desirable? How are usage
> > >> statistics gathered for such volumes? It's nice to offer the
> flexibility,
> > >> but is it a good idea in this case?****
> > >>
> > >> ** **
> > >>
> > >> Thanks!****
> > >>
> > >> ** **
> > >>
> > >> ** **
> > >>
> > >> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> wrote:**
> > >> **
> > >>
> > >>
> > >>
> > >> > -----Original Message-----
> > >> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> > Sent: Friday, June 07, 2013 4:08 PM
> > >> > To: dev@cloudstack.apache.org
> > >> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >> >
> > >> > Hi John,
> > >> >
> > >> > Yes, you are correct that Xen must layer an SR on top of the iSCSI
> > >> volume
> > >> > to use it. Same story for VMware: ESX(i) must layer a datastore on
> top
> > >> of
> > >> > the iSCSI volume to use it. I look at it like they are layering a
> > >> clustered
> > >> > file system on the SAN volume so the hypervisors can share access to
> > the
> > >> > contents of the volume.
> > >> >
> > >> > In 4.1, by the time the hypervisor-attach-volume logic is called,
> the
> > >> SR or
> > >> > datastore has already been created (usually manually by an admin).
> > >> >
> > >> > This pre-setup of the, say, SR is, of course, not acceptable in an
> > >> > environment where each CloudStack volume that a user creates is
> > mapped
> > >> > to a
> > >> > single SAN volume (via the SR).
> > >> >
> > >> > The question comes down to who should allocate the SR dynamically.
> > >> >
> > >> > We could have the storage framework ask the storage plug-in if it is
> > >> > managed. If it is, then the storage framework could send a message
> to
> > >> the
> > >> > hypervisor in question to create (let's talk Xen here) the SR ahead
> of
> > >> > time. Then, when the storage framework next sends the attach-volume
> > >> > command
> > >> > to the hypervisor, it should work without changes to that
> attach-volume
> > >> > logic (because - from the point of view of the attach logic - the
> SR is
> > >> > already existent, as expected).
> > >> >
> > >> > Now, as Edison has pointed out, this limits the power of the storage
> > >> > plug-in. A storage plug-in in this model cannot directly attach
> storage
> > >> to
> > >> > a VM (it must go through the hypervisor). Perhaps that is OK. We
> need
> > to
> > >> > make a call on that.
> > >>
> > >>
> > >> Yes, I think we need to address this usage case, and won't limit the
> > >> power of the storage plug-in.
> > >> John, what's your idea to fit this usage case into your abstraction?
> > >>
> > >>
> > >> >
> > >> > If we want to give more power to the storage vendor, we would have
> to
> > >> > have
> > >> > the storage framework call into the storage plug-in to send the
> > >> appropriate
> > >> > attach commands to the hypervisor.
> > >> >
> > >> > Let's discuss and come to a consensus (or at least agree on a path)
> as
> > >> soon
> > >> > as we can. :)
> > >> >
> > >> > Thanks!
> > >> >
> > >> >
> > >> >
> > >> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
> > >> wrote:
> > >> >
> > >> > > Mike,
> > >> > >
> > >> > > My understanding is the Xen expects an SR structure on all ISCSI
> > >> devices
> > >> > > -- at least that is how I read the code in your patch. Is my
> > >> understanding
> > >> > > correct? If so, the Xen plugin should be able to query the
> storage
> > >> device
> > >> > > to determine the presence of the SR structure and create it if it
> > >> does not
> > >> > > exist. Am I missing something in the implementation that makes
> that
> > >> type
> > >> > > of implementation impossible?
> > >> > >
> > >> > > Thanks,
> > >> > > -John
> > >> > >
> > >> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Yeah, if a storage vendor wanted to attach a volume directly to
> a
> > >> VM in
> > >> > > 4.2
> > >> > > > today, it would probably fail because the attach-volume logic
> > >> assumes
> > >> > the
> > >> > > > existence of the necessary hypervisor data structure (ex. SR on
> > >> Xen).
> > >> > > >
> > >> > > > If we wanted to enable such an attach, we could do it the way
> > Edison
> > >> > > > suggests.
> > >> > > >
> > >> > > >
> > >> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Edison.su@citrix.com
> >
> > >> wrote:
> > >> > > >
> > >> > > >>
> > >> > > >>
> > >> > > >>> -----Original Message-----
> > >> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> > > >>> Sent: Friday, June 07, 2013 2:37 PM
> > >> > > >>> To: dev@cloudstack.apache.org
> > >> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >> > > >>>
> > >> > > >>> As we only have three weeks until feature freeze, we should
> > come
> > >> to
> > >> > a
> > >> > > >>> consensus on this design point as soon as possible.
> > >> > > >>>
> > >> > > >>> Right now, if the storage framework asks my driver it is is
> > >> managed, it
> > >> > > >>> will say 'yes.' This means the framework will tell the driver
> to
> > >> > > perform
> > >> > > >>> its management activities. This then means the driver will
> call
> > >> into
> > >> > > the
> > >> > > >>> host (it doesn't know which hypervisor, by the way) to perform
> > the
> > >> > > >> activity
> > >> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> > >> > > >>>
> > >> > > >>> The driver doesn't know which hypervisor it's talking to, it
> just
> > >> > > sends a
> > >> > > >>> message to the host to perform the necessary pre-attach work.
> > >> > > >>
> > >> > > >> Could we just expose a method like
> > "attachVolume/dettachVolume" on
> > >> > the
> > >> > > >> PrimaryDataStoreDriver driver?
> > >> > > >> For most of cases, the implementation of each driver would just
> > >> send
> > >> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can
> > put
> > >> > the
> > >> > > >> implementation in a base class, so that can be shared by all of
> > >> this
> > >> > > >> drivers), and in each hypervisor resource code, which may just
> call
> > >> > > >> hypervisor's api to attach the volume to VM, while for certain
> > >> storage,
> > >> > > >> like SolidFire, may need to create a SR first, and create
> volume
> > >> on it,
> > >> > > >> then call hypervisor's API to attach volume to VM.
> > >> > > >> While for some other storage vendor, may want to bypass
> > hypervisor
> > >> > > during
> > >> > > >> attaching volume, so inside driver's attachvolume
> implementation,
> > >> the
> > >> > > >> driver can do some magic, such as, directly talk to an agent
> > >> inside VM
> > >> > > >> instance, then create a disk inside VM.
> > >> > > >>
> > >> > > >> How do you guys think?
> > >> > > >>
> > >> > > >>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <
> Edison.su@citrix.com>
> > >> > > wrote:
> > >> > > >>>
> > >> > > >>>>
> > >> > > >>>>
> > >> > > >>>>> -----Original Message-----
> > >> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> > >> > > >>>>> To: dev@cloudstack.apache.org
> > >> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >> > > >>>>>
> > >> > > >>>>> Hi John,
> > >> > > >>>>>
> > >> > > >>>>> How's about this:
> > >> > > >>>>>
> > >> > > >>>>> The driver can implement an isManaged() method. The
> > >> > > >>> VolumeManagerImpl
> > >> > > >>>>> can
> > >> > > >>>>> call into the driver to see if its managed. If it is, the
> > >> > > >>>> VolumeManagerImpl
> > >> > > >>>>> (which is responsible for calling into the hypervisor to
> attach
> > >> the
> > >> > > >> disk)
> > >> > > >>>>> can call into the hypervisor to create the necessary
> hypervisor
> > >> data
> > >> > > >>>>> structure (ex. for XenServer, a storage repository).
> > >> > > >>>>
> > >> > > >>>> The problem here is that storage vendor may work differently
> > with
> > >> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> > >> maybe
> > >> > > >> other
> > >> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> > >> > directly
> > >> > > >> to
> > >> > > >>>> VM instance, see the discuss(
> > >> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> > >> > > >>>
> > >> >
> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> > >> > > >>> %3E
> > >> > > >>>> ).
> > >> > > >>>> So I would let storage provider to implement attach disk to
> VM,
> > >> > > >> instead of
> > >> > > >>>> implemented by cloudstack itself.
> > >> > > >>>>
> > >> > > >>>>
> > >> > > >>>>>
> > >> > > >>>>> If that's what you're going for, that works for me. By the
> way,
> > >> > > >> Edison's
> > >> > > >>>>> default storage plug-in (which handles the default storage
> > >> behavior
> > >> > > >> in
> > >> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> > >> talks to
> > >> > > >>>>> hypervisors. You might want to contact him and inform him of
> > >> your
> > >> > > >>>> concerns
> > >> > > >>>>> or that logic (as is) will make it to production.
> > >> > > >>>>>
> > >> > > >>>>> Please let me know if what I wrote in above (for my
> solution)
> > >> is OK
> > >> > > >> with
> > >> > > >>>>> you. :)
> > >> > > >>>>>
> > >> > > >>>>> Thanks!
> > >> > > >>>>>
> > >> > > >>>>>
> > >> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> > >> jburwell@basho.com>
> > >> > > >>> wrote:
> > >> > > >>>>>
> > >> > > >>>>>> Mike,
> > >> > > >>>>>>
> > >> > > >>>>>> Please see my responses in-line below.
> > >> > > >>>>>>
> > >> > > >>>>>> Thanks,
> > >> > > >>>>>> -John
> > >> > > >>>>>>
> > >> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > >> > > >>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>> wrote:
> > >> > > >>>>>>
> > >> > > >>>>>>> Hey John,
> > >> > > >>>>>>>
> > >> > > >>>>>>> I still have a bit more testing I'd like to do before I
> build
> > >> up
> > >> > > >> a
> > >> > > >>>> patch
> > >> > > >>>>>>> file, but this is the gist of what I've done:
> > >> > > >>>>>>>
> > >> > > >>>>>>> * During a volume-attach operation, after
> > VolumeManagerImpl
> > >> > tells
> > >> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I
> > have
> > >> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> > driver if
> > >> it
> > >> > > >>>>> managed.
> > >> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> > >> > > >> whatever
> > >> > > >>>>>>> activity is required. In my case, this includes sending a
> > >> > > >> message to
> > >> > > >>>> the
> > >> > > >>>>>>> host where the VM is running to have, say XenServer, add a
> > >> > > >> storage
> > >> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of
> > the
> > >> > > >> SAN
> > >> > > >>>>>> volume,
> > >> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space
> it
> > >> can
> > >> > > >> on
> > >> > > >>>> the
> > >> > > >>>>>>> storage repository). After this, the normal attach-volume
> > >> > > >> message is
> > >> > > >>>> sent
> > >> > > >>>>>>> to the host by VolumeManagerImpl.
> > >> > > >>>>>>
> > >> > > >>>>>> There should be **no** code from a storage driver to a
> > >> hypervisor.
> > >> > > >> I
> > >> > > >>>>>> apologize for the repetition, but we simply can not have
> > >> hypervisor
> > >> > > >>>>>> specific code in the storage layer. The circular
> dependencies
> > >> > > >> between
> > >> > > >>>> the
> > >> > > >>>>>> two layers are not sustainable in the long term. Either
> the
> > >> > > >>>> VirtualManager
> > >> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> > >> > > >>> coordinate
> > >> > > >>>>>> volume creation and then populating the SR. Ideally, we
> can
> > >> > > >>>> generalize the
> > >> > > >>>>>> process flow for attaching volumes such that the Xen
> > hypervisor
> > >> > > >> plugin
> > >> > > >>>>>> would only implement callbacks to perform the attach action
> > and
> > >> > > >> create
> > >> > > >>>>> the
> > >> > > >>>>>> structure and SR. To my mind, the SolidFire driver should
> > >> only be
> > >> > > >>>>>> allocating space and providing information about contents
> > (e.g.
> > >> > > >> space
> > >> > > >>>>>> available, space consumed, streams to a URI, file handle
> for a
> > >> URI,
> > >> > > >>>> etc)
> > >> > > >>>>>> and capabilities.
> > >> > > >>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> * The reverse is performed for a detach-volume command.
> > >> > > >>>>>>>
> > >> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
> > >> driver.
> > >> > > >>>>>> Edison's
> > >> > > >>>>>>> default driver simply does a "return false;". We could
> add a
> > >> new
> > >> > > >>>>>> parameter
> > >> > > >>>>>>> to the createStoragePool API command, if we want, to
> > remove
> > >> > the
> > >> > > >>>>>> hard-coded
> > >> > > >>>>>>> return values in the drivers (although my driver will
> probably
> > >> > > >> just
> > >> > > >>>>>> ignore
> > >> > > >>>>>>> this parameter and always return true since it wouldn't
> make
> > >> > > >> sense
> > >> > > >>>> for it
> > >> > > >>>>>>> to ever return false). We'd need another column in the
> > >> > > >> storage_pool
> > >> > > >>>>> table
> > >> > > >>>>>>> to store this value.
> > >> > > >>>>>>
> > >> > > >>>>>> Yes, I think we should have a parameter added to the
> > >> > > >>> createStoragePool
> > >> > > >>>>>> surfaced to the HTTP API that allows DataStores to be
> > >> configured
> > >> > > >> for
> > >> > > >>>>>> management when their underlying drivers support it. To
> > >> simplify
> > >> > > >>>> things,
> > >> > > >>>>>> this flag should only be mutable when the DataStore is
> > >> created. It
> > >> > > >>>> would be
> > >> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged
> > after
> > >> > > >>>> creation.
> > >> > > >>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> Sound like I'm in sync with what you were thinking?
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > >> > > >>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>> wrote:
> > >> > > >>>>>>>
> > >> > > >>>>>>>> I agree, John. Just wanted to point out that I have a
> > working
> > >> > > >> GUI
> > >> > > >>>> for
> > >> > > >>>>>> you
> > >> > > >>>>>>>> to review (in that document), if you'd like to check it
> out.
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> Thanks!
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> > >> > > >> jburwell@basho.com>
> > >> > > >>>>>> wrote:
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>> Mike,
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> I would like the UIs of two features reviewed together
> to
> > >> > > >> ensure
> > >> > > >>>>>>>>> consistency across the concepts of hypervisor throttled
> > IOPs
> > >> > > >> and
> > >> > > >>>>>>>>> storage device provisioned IOPs. I see the potential
> for
> > >> > > >>>> confusion,
> > >> > > >>>>>>>>> and I think a side-by-side Ui review of these features
> will
> > >> > > >> help
> > >> > > >>>>>>>>> minimize any potential confusion.
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
> > >> itself
> > >> > > >> if
> > >> > > >>>> it
> > >> > > >>>>>>>>> is acceptable that a VM is only permitted utilize
> hypervisor
> > >> > > >>>> throttled
> > >> > > >>>>>>>>> IOPs or storage provisioned IOPs.
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> Thanks,
> > >> > > >>>>>>>>> -John
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > >> > > >>>>>>>>> <mi...@solidfire.com> wrote:
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
> > sent
> > >> > to
> > >> > > >>> you
> > >> > > >>>>> the
> > >> > > >>>>>>>>>> other day to see how the GUI looks for provisioned
> > storage
> > >> > > >> IOPS.
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail
> list
> > >> > > >> and we
> > >> > > >>>>>>>>> decided
> > >> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> > >> > Offering
> > >> > > >>>> dialog.
> > >> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> > >> > should
> > >> > > >>> be
> > >> > > >>>>> able
> > >> > > >>>>>> to
> > >> > > >>>>>>>>>> leverage this GUI going forward (even if they, say,
> only
> > >> use
> > >> > > >> Max
> > >> > > >>>>>> IOPS).
> > >> > > >>>>>>>>>> These fields are optional and can be ignored for
> storage
> > >> that
> > >> > > >>>> does not
> > >> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size
> field,
> > >> the
> > >> > > >>>> admin can
> > >> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> > Burst
> > >> > > >> IOPS.
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's
> feature
> > or
> > >> > > >> mine,
> > >> > > >>>> as
> > >> > > >>>>>> is
> > >> > > >>>>>>>>>> decided by the admin).
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> I'm not sure what we can do about these two features
> > >> > > >> expressing
> > >> > > >>>> the
> > >> > > >>>>>>>>> speed
> > >> > > >>>>>>>>>> in different terms. I've never seen a SAN express the
> > IOPS
> > >> for
> > >> > > >>>> QoS in
> > >> > > >>>>>>>>> any
> > >> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
> > >> read/write
> > >> > > >>>> IOPS).
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Thanks
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > >> > > >>> <jb...@basho.com>
> > >> > > >>>>>>>>> wrote:
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>> Wei,
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> > >> > > >>> Storage/Hypervisor
> > >> > > >>>>>> layer
> > >> > > >>>>>>>>>>> separation, but we still need to reconcile the
> behavior
> > of
> > >> > > >>>> hypervisor
> > >> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> > >> > > >> following
> > >> > > >>>>>> issues
> > >> > > >>>>>>>>>>> outstanding:
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > >> > > >>> read/write
> > >> > > >>>>>>>>> values
> > >> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as
> > total
> > >> > > >> IOPS.
> > >> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
> > to
> > >> > > >>> storage
> > >> > > >>>>>> volumes
> > >> > > >>>>>>>>>>> with provisioned IOPS?
> > >> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> > >> > > >>>> provisioned
> > >> > > >>>>>> IOPS
> > >> > > >>>>>>>>>>> that will permit providers to discriminate these
> > >> guaranteed
> > >> > > >>>>>> operations
> > >> > > >>>>>>>>> in
> > >> > > >>>>>>>>>>> the event they want to bill for it differently?
> > >> > > >>>>>>>>>>> 4. What is the user experience for throttled and
> > >> provisioned
> > >> > > >>> IOPS
> > >> > > >>>>>> that
> > >> > > >>>>>>>>>>> minimizes confusion of these concepts?
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> My thinking is that a VM can have either utilize
> > >> hypervisor
> > >> > > >>>> throttled
> > >> > > >>>>>>>>> IOPS
> > >> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> > >> solve
> > >> > > >>>> items 1
> > >> > > >>>>>>>>> and 2.
> > >> > > >>>>>>>>>>> Since the two facilities would not be permitted to
> > operate
> > >> > > >>>> together,
> > >> > > >>>>>>>>> they
> > >> > > >>>>>>>>>>> do not need to be semantically compatible. I think
> item
> > 3
> > >> > > >> can be
> > >> > > >>>>>>>>> resolved
> > >> > > >>>>>>>>>>> with an additional flag or two on the usage records.
> As
> > >> for
> > >> > > >>>> Item 4,
> > >> > > >>>>>> I
> > >> > > >>>>>>>>> am
> > >> > > >>>>>>>>>>> not familiar with how these two enhancements are
> > >> > depicted in
> > >> > > >>> the
> > >> > > >>>>> user
> > >> > > >>>>>>>>>>> interface. I think we need to review the UI
> > enhancements
> > >> > for
> > >> > > >>>> both
> > >> > > >>>>>>>>>>> enhancements and ensure they are consistent.
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> Do these solutions make sense?
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> > >> > <us...@gmail.com>
> > >> > > >>>>> wrote:
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>> John and Mike,
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> > >> > 2780/2729,
> > >> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > >> > > >>> CLOUDSTACK-
> > >> > > >>>>> 2875 ) this
> > >> > > >>>>>>>>> week.
> > >> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
> > and
> > >> > > >> ask
> > >> > > >>>> for
> > >> > > >>>>>>>>> second
> > >> > > >>>>>>>>>>>> merge request after testing.
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>> -Wei
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> > <mi...@solidfire.com>
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
> > >> John,
> > >> > > >> and
> > >> > > >>>>> have
> > >> > > >>>>>>>>> been
> > >> > > >>>>>>>>>>>>> re-working this section of code today.
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> Thanks for the comments,
> > >> > > >>>>>>>>>>>>> Mike
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > >> > > >>>>> <jb...@basho.com>
> > >> > > >>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> See my responses in-line below.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Thanks for the response.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> > >> > whether or
> > >> > > >>>>> not a
> > >> > > >>>>>>>>> Volume
> > >> > > >>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> > but an
> > >> > > >>> actual
> > >> > > >>>>>>>>> isManaged
> > >> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> > > >> resources
> > >> > > >>>> for
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>> if it is managed."
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> > >> > method
> > >> > > >>>>> added to
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns
> true,
> > >> > > >> then the
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever
> > name)
> > >> > > >>> method
> > >> > > >>>>> (which
> > >> > > >>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
> > well)
> > >> > and
> > >> > > >>> this
> > >> > > >>>>>> would
> > >> > > >>>>>>>>>>> call
> > >> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create,
> > say
> > >> > on
> > >> > > >>>>>>>>> XenServer,
> > >> > > >>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>> SR?
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> > >> > > >>>>> DataStoreDriver.
> > >> > > >>>>>>>>> Since
> > >> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> > storage,
> > >> > I
> > >> > > >>>> would
> > >> > > >>>>>> add
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> > >> > straw
> > >> > > >>> man
> > >> > > >>>>> term
> > >> > > >>>>>>>>> --
> > >> > > >>>>>>>>>>>>>> something other than volume) methods to
> > >> > > >>>>>>>>>>> allocate/create/deallocate/delete
> > >> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> > mutable
> > >> > > >>> property
> > >> > > >>>>> of
> > >> > > >>>>>>>>>>>>> DataStore
> > >> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> > >> > > >>> DataStoreDriver
> > >> > > >>>>> can be
> > >> > > >>>>>>>>>>>>> managed.
> > >> > > >>>>>>>>>>>>>> This approach allows operators to override
> > >> > manageability
> > >> > > >> of
> > >> > > >>>>>> devices.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
> > Xen
> > >> > > >> plugin
> > >> > > >>>>>> would
> > >> > > >>>>>>>>> be
> > >> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume
> > methods
> > >> > to
> > >> > > >>>>> create any
> > >> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There
> > >> should
> > >> > > >> be no
> > >> > > >>>>>>>>>>>>> dependencies
> > >> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I
> said
> > >> > > >> earlier,
> > >> > > >>>> such
> > >> > > >>>>>>>>>>>>> circular
> > >> > > >>>>>>>>>>>>>> dependencies will lead to a tangled,
> > unmaintainable
> > >> > mess.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with
> > you.
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Thanks again, John
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > >> > > >>>>> <jb...@basho.com>
> > >> > > >>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
> > layer
> > >> > that
> > >> > > >>>>>> supports n
> > >> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> > >> > > >> hypervisors.
> > >> > > >>>>>> Such
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and
> > untestable
> > >> > > >>> beast.
> > >> > > >>>>>>>>>>>>> Therefore, my
> > >> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to
> > evolve
> > >> > the
> > >> > > >>>>> Storage
> > >> > > >>>>>> layer
> > >> > > >>>>>>>>>>>>> towards
> > >> > > >>>>>>>>>>>>>>> this separation.
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> > >> > whether
> > >> > > >>> or
> > >> > > >>>>> not a
> > >> > > >>>>>>>>> Volume
> > >> > > >>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> > but
> > >> > an
> > >> > > >>> actual
> > >> > > >>>>>>>>>>> isManaged
> > >> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> > > >> resources
> > >> > > >>>> for
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
> > >> needs
> > >> > > >> to
> > >> > > >>>>>> indicate
> > >> > > >>>>>>>>>>>>> whether
> > >> > > >>>>>>>>>>>>>>> or not it supports management operations.
> > Finally, I
> > >> > > >> think
> > >> > > >>>> we
> > >> > > >>>>>>>>> need to
> > >> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect
> to
> > >> have
> > >> > > >>>>>> something
> > >> > > >>>>>>>>>>>>> that is
> > >> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> > capable
> > >> > > >>> managing
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> > >> > unmanaged).
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be
> > done
> > >> > in
> > >> > > >>> the
> > >> > > >>>>> Xen
> > >> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> > >> > Storage
> > >> > > >>>>> layer to
> > >> > > >>>>>>>>>>> perform
> > >> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> > create a
> > >> > > >> file,
> > >> > > >>>> etc).
> > >> > > >>>>>>>>>>> This
> > >> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> > >> > > >>>> hypervisor, as
> > >> > > >>>>>>>>> such,
> > >> > > >>>>>>>>>>>>>>> should be confined to its implementation. From
> > my
> > >> > > >>>>> perspective,
> > >> > > >>>>>>>>>>> nothing
> > >> > > >>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with
> > content.
> > >> > From
> > >> > > >>> its
> > >> > > >>>>>>>>>>>>> perspective,
> > >> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the
> > means
> > >> > to
> > >> > > >>>> query
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>> support the interpretation of the content by
> > higher-
> > >> > level
> > >> > > >>>>> layers
> > >> > > >>>>>>>>> (e.g.
> > >> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> > >> > composition
> > >> > > >>> of
> > >> > > >>>>>>>>>>> operations
> > >> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> > >> > Volume
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>> protocol
> > >> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> > >> > implemented
> > >> > > >> in
> > >> > > >>>> my
> > >> > > >>>>>>>>> patch,
> > >> > > >>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>> could do the following:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> > >> > > >>>>>>>>> VolumeManagerImpl
> > >> > > >>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
> > the
> > >> > > >>>>> hypervisor.
> > >> > > >>>>>> Before
> > >> > > >>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> > >> > dealing
> > >> > > >>> with
> > >> > > >>>>>>>>> Dynamic
> > >> > > >>>>>>>>>>> (or
> > >> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> > >> send a
> > >> > > >>>> "Create
> > >> > > >>>>>>>>> SR"
> > >> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> > >> > would
> > >> > > >>> then
> > >> > > >>>>>>>>> proceed
> > >> > > >>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and
> > we just
> > >> > > >> add
> > >> > > >>>>>> another
> > >> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
> > >> > > >> particular
> > >> > > >>>>> type
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>> storage.
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> What do you think?
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> Hey John,
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
> > >> > outlines
> > >> > > >>> how
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably
> > be
> > >> of
> > >> > > >> use
> > >> > > >>>> to
> > >> > > >>>>>>>>> you, as
> > >> > > >>>>>>>>>>>>> well,
> > >> > > >>>>>>>>>>>>>>>> as you perform the code review.
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> I have shared this document with you (you
> > should
> > >> > have
> > >> > > >>>>> received
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> Talk to you later!
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > >> > > >>>>>> jburwell@basho.com
> > >> > > >>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> > >> > thoughts,
> > >> > > >>> but
> > >> > > >>>>>> want
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> > >> solidfire,
> > >> > > >>>>>>>>>>>>> disk_io_throttle,
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
> > in a
> > >> > > >>>>> coherent
> > >> > > >>>>>>>>> form
> > >> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> > introduces
> > >> > the
> > >> > > >>>> notion
> > >> > > >>>>>> of
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> > introduces
> > >> > > >> the
> > >> > > >>>>> notion
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the
> > plug-in
> > >> > was
> > >> > > >>>>>>>>> incomplete,
> > >> > > >>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>> I had
> > >> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> > >> > > >> device.
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> > spending on
> > >> > > >> this.
> > >> > > >>>> :)
> > >> > > >>>>> Do
> > >> > > >>>>>>>>> you
> > >> > > >>>>>>>>>>>>>>>>>> have a
> > >> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should
> > accomplish
> > >> > > >>> what
> > >> > > >>>>> you're
> > >> > > >>>>>>>>> looking
> > >> > > >>>>>>>>>>>>>>>>>> for?
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
> > <
> > >> > > >>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> > introduces
> > >> > the
> > >> > > >>>> notion
> > >> > > >>>>>> of
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> > management
> > >> > > >>>>> capabilities
> > >> > > >>>>>>>>> (i.e.
> > >> > > >>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
> > it
> > >> > > >>> managed
> > >> > > >>>>> by
> > >> > > >>>>>>>>> CloudStack)
> > >> > > >>>>>>>>>>>>>>>>>> from the
> > >> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
> > with
> > >> > a
> > >> > > >>>>> semantic
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>> will
> > >> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> > managed.
> > >> > I
> > >> > > >>> also
> > >> > > >>>>> want
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
> > from
> > >> > the
> > >> > > >>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> definitions.
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
> > <
> > >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> > Burwell <
> > >> > > >>>>>>>>>>> jburwell@basho.com
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> > >> > questions
> > >> > > >>> for
> > >> > > >>>>> me
> > >> > > >>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>> regards to
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> > LUNs
> > >> > and
> > >> > > >>> SAN
> > >> > > >>>>>> volumes?
> > >> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
> > LUN
> > >> > (LUN
> > >> > > >>> 0).
> > >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> > Volumes
> > >> > and
> > >> > > >>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> Volumes?
> > >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> > management
> > >> > > >>> server or
> > >> > > >>>>>>>>> externally
> > >> > > >>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>> operator?
> > >> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
> > SAN
> > >> > > >>> volume
> > >> > > >>>>> (same
> > >> > > >>>>>> as a
> > >> > > >>>>>>>>>>> SAN
> > >> > > >>>>>>>>>>>>>>>>>> LUN) is
> > >> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via
> > the
> > >> > plug-in)
> > >> > > >>>>> the
> > >> > > >>>>>> first
> > >> > > >>>>>>>>>>> time
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> > hypervisor.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
> > in,
> > >> > but
> > >> > > >>>> still
> > >> > > >>>>>>>>> want
> > >> > > >>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> use a
> > >> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
> > already
> > >> > > >>> today
> > >> > > >>>>>> (prior
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> 4.2). The
> > >> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
> > and -
> > >> > in
> > >> > > >>> this
> > >> > > >>>>> case -
> > >> > > >>>>>>>>>>>>>>>>>> multiple VMs
> > >> > > >>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> > While
> > >> > you
> > >> > > >>> can do
> > >> > > >>>>> this
> > >> > > >>>>>>>>>>>>> today,
> > >> > > >>>>>>>>>>>>>>>>>> it is
> > >> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
> > QoS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> > management
> > >> > server
> > >> > > >>> or
> > >> > > >>>>> externally
> > >> > > >>>>>>>>> by
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> operator?
> > >> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> > SAN
> > >> > > >>> volumes
> > >> > > >>>>> are
> > >> > > >>>>>>>>>>>>> completely
> > >> > > >>>>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
> > in).
> > >> > There
> > >> > > >>> is
> > >> > > >>>>> no
> > >> > > >>>>>> admin
> > >> > > >>>>>>>>>>>>>>>>>>>> interaction.
> > >> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
> > SAN
> > >> > > >>> volume
> > >> > > >>>>> and a
> > >> > > >>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
> > that
> > >> > > >>>>> supports
> > >> > > >>>>>> true
> > >> > > >>>>>>>>>>> QoS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
> > are
> > >> > > >> related
> > >> > > >>>>> and
> > >> > > >>>>>>>>>>>>> expected
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> operate.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> > Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> > >> > multiple
> > >> > > >>> VMs
> > >> > > >>>>> with
> > >> > > >>>>>>>>>>>>>>>>>> provisioned
> > >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> > storage."
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
> > map a
> > >> > > >>> single
> > >> > > >>>>> SAN
> > >> > > >>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>> (LUN) to a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
> > have
> > >> > > >>> multiple
> > >> > > >>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> > >> > guarantee
> > >> > > >>> QoS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> > >> > volume
> > >> > > >>> house
> > >> > > >>>>> more
> > >> > > >>>>>>>>> than
> > >> > > >>>>>>>>>>>>> one
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
> > today
> > >> > > >>> without
> > >> > > >>>>> any of
> > >> > > >>>>>> my
> > >> > > >>>>>>>>>>>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>>>> code.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> > >> > Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> > >> > volume
> > >> > > >>> for
> > >> > > >>>>>>>>>>> CloudStack's
> > >> > > >>>>>>>>>>>>>>>>>> use
> > >> > > >>>>>>>>>>>>>>>>>>>> onto
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> > created."
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> > >> > Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> > >> > concerns me
> > >> > > >>> --
> > >> > > >>>>>> concepts
> > >> > > >>>>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
> > layer.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> > >> > Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
> > is
> > >> > today
> > >> > > >>>> only
> > >> > > >>>>>> used
> > >> > > >>>>>>>>>>> (as
> > >> > > >>>>>>>>>>>>>>>>>> far as
> > >> > > >>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
> > have
> > >> > not
> > >> > > >>>>> PreSetup an
> > >> > > >>>>>>>>> SR).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> > volume
> > >> > from
> > >> > > >>>>> VMware, it
> > >> > > >>>>>>>>> uses
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> vmfs
> > >> > > >>>>>>>>>>>>>>>>>>>>>> type.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> > volume
> > >> > from
> > >> > > >>> KVM,
> > >> > > >>>>> it uses
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> > thinking
> > >> > > >>> here
> > >> > > >>>>> was to
> > >> > > >>>>>>>>> make a
> > >> > > >>>>>>>>>>>>>>>>>> new type
> > >> > > >>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
> > ability
> > >> > > >>> Edison
> > >> > > >>>>> added
> > >> > > >>>>>>>>> into
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> > more
> > >> > > >>> specificy,
> > >> > > >>>>> though:
> > >> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> > Mike
> > >> > > >>> Tutkowski
> > >> > > >>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> > shouldn't
> > >> > know
> > >> > > >>> or
> > >> > > >>>>> care
> > >> > > >>>>>> that
> > >> > > >>>>>>>>> it
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>> being
> > >> > > >>>>>>>>>>>>>>>>>>>>>> used
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
> > answer
> > >> > > >>>>> questions
> > >> > > >>>>>>>>> about
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
> > the
> > >> > > >>>>> SolidFire
> > >> > > >>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>> needs to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
> > the
> > >> > > >>> hypervisor
> > >> > > >>>>> logic
> > >> > > >>>>>>>>>>> knows
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>> treat
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> > >> > constraint
> > >> > > >>>>> and just
> > >> > > >>>>>>>>>>>>>>>>>> calling it
> > >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
> > way
> > >> > for
> > >> > > >>> the
> > >> > > >>>>> hypervisor
> > >> > > >>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> > >> > perform
> > >> > > >>> the
> > >> > > >>>>> necessary
> > >> > > >>>>>>>>>>>>>>>>>> activities.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> > John
> > >> > > >>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > >> > > >>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
> > be
> > >> > able to
> > >> > > >>>>> manage a
> > >> > > >>>>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > >> > > >>> CloudStack
> > >> > > >>>>> (e.g. the
> > >> > > >>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>> shared
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> > operator has
> > >> > > >>> chosen
> > >> > > >>>>> to
> > >> > > >>>>>>>>> dedicate
> > >> > > >>>>>>>>>>>>>>>>>> only a
> > >> > > >>>>>>>>>>>>>>>>>>>>>> portion
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> > reasoning
> > >> > > >>> make
> > >> > > >>>>> sense?"
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> > creating a
> > >> > SAN
> > >> > > >>>>> volume via
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> > hypervisor)
> > >> > would
> > >> > > >>>>> work unless
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
> > say,
> > >> > an
> > >> > > >>> SR.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
> > of
> > >> > > >>>>> CloudStack, an
> > >> > > >>>>>>>>>>>>> operator
> > >> > > >>>>>>>>>>>>>>>>>>>>>> allocates
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
> > and
> > >> > > >>> exposes it
> > >> > > >>>>>> through
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> LUN. They
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
> > over
> > >> > to
> > >> > > >>>>> CloudStack,
> > >> > > >>>>>>>>> but
> > >> > > >>>>>>>>>>>>>>>>>> not allow
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
> > LUNs.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
> > to my
> > >> > > >>>>> changes, we
> > >> > > >>>>>>>>> would
> > >> > > >>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>> passing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
> > the
> > >> > > >>>>> necessary
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>> (like
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> > maybe
> > >> > have
> > >> > > >>> the
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>>>> framework
> > >> > > >>>>>>>>>>>>>>>>>>>> itself
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> > >> > support
> > >> > > >>> from
> > >> > > >>>>> the
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> side and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> > specifically
> > >> > to
> > >> > > >>> create
> > >> > > >>>>> the
> > >> > > >>>>>>>>> SR
> > >> > > >>>>>>>>>>>>>>>>>> before the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> > management
> > >> > plugin
> > >> > > >>>>> should
> > >> > > >>>>>> have a
> > >> > > >>>>>>>>>>>>> rich
> > >> > > >>>>>>>>>>>>>>>>>> enough
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> > determine
> > >> > > >>> available
> > >> > > >>>>> for
> > >> > > >>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>> storage.
> > >> > > >>>>>>>>>>>>>>>>>>>> For
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > >> > > >>> interrogation of
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> > device
> > >> > itself
> > >> > > >>>>> shouldn't
> > >> > > >>>>>>>>> know
> > >> > > >>>>>>>>>>>>>>>>>> or care
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> > simply be
> > >> > able
> > >> > > >>> to
> > >> > > >>>>>> answer
> > >> > > >>>>>>>>>>>>>>>>>> questions
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> > SolidFire
> > >> > > >>> driver
> > >> > > >>>>> (which
> > >> > > >>>>>>>>> is a
> > >> > > >>>>>>>>>>>>>>>>>> type of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> > >> > done, a
> > >> > > >>> new
> > >> > > >>>>> Primary
> > >> > > >>>>>>>>>>>>>>>>>> Storage shows
> > >> > > >>>>>>>>>>>>>>>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> > Offering
> > >> > > >>> that
> > >> > > >>>>>> references
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>> tag
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> > Storage.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> > >> > CloudStack
> > >> > > >>>>> volume. This
> > >> > > >>>>>>>>> leads
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> a new
> > >> > > >>>>>>>>>>>>>>>>>>>>>> row
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> > >> > CloudStack
> > >> > > >>>>> volume to a
> > >> > > >>>>>>>>> VM
> > >> > > >>>>>>>>>>>>>>>>>> (attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>> disk).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> > framework
> > >> > calling
> > >> > > >>> the
> > >> > > >>>>> plug-in
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> create a
> > >> > > >>>>>>>>>>>>>>>>>>>> new
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
> > case, a
> > >> > > >>> SAN).
> > >> > > >>>>> The
> > >> > > >>>>>>>>> plug-in
> > >> > > >>>>>>>>>>>>>>>>>> also
> > >> > > >>>>>>>>>>>>>>>>>>>>>> updates
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> > applicable
> > >> > data
> > >> > > >>>>> (like the
> > >> > > >>>>>>>>> IQN of
> > >> > > >>>>>>>>>>>>>>>>>> the SAN
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> > invoked if
> > >> > the
> > >> > > >>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> volume is in
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> > attach,
> > >> > the
> > >> > > >>>>> volume
> > >> > > >>>>>>>>> will be
> > >> > > >>>>>>>>>>>>>>>>>> in the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
> > and
> > >> > the
> > >> > > >>>>> plug-in
> > >> > > >>>>>>>>> code
> > >> > > >>>>>>>>>>>>>>>>>> will not
> > >> > > >>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> > volume.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
> > is run
> > >> > and
> > >> > > >>>>> detects
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
> > the
> > >> > form
> > >> > > >>> of a
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
> > PM,
> > >> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> > sync on
> > >> > > >>>>> terminology,
> > >> > > >>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>>>>>>> in these
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> > physical
> > >> > > >>>>> allocation on
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
> > try to
> > >> > > >>> mean
> > >> > > >>>>> 'SAN
> > >> > > >>>>>>>>> volume'.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
> > end
> > >> > user
> > >> > > >>> can
> > >> > > >>>>> make in
> > >> > > >>>>>>>>>>>>>>>>>> CloudStack, I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
> > PM,
> > >> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > >> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> > very
> > >> > well
> > >> > > >>> make
> > >> > > >>>>> sense, but
> > >> > > >>>>>>>>> I'm
> > >> > > >>>>>>>>>>>>>>>>>> having a
> > >> > > >>>>>>>>>>>>>>>>>>>>>> hard
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
> > Edison
> > >> > in on
> > >> > > >>>>> this
> > >> > > >>>>>>>>> conversation
> > >> > > >>>>>>>>>>>>>>>>>> as he
> > >> > > >>>>>>>>>>>>>>>>>>>> was
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > >> > > >>> approach I
> > >> > > >>>>> took.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
> > PM,
> > >> > > >>> John
> > >> > > >>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> > combining
> > >> > two
> > >> > > >>>>> distinct
> > >> > > >>>>>>>>> concepts
> > >> > > >>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> > >> > protocols.
> > >> > > >>> In
> > >> > > >>>>> both
> > >> > > >>>>>>>>> cases, we
> > >> > > >>>>>>>>>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> > >> > system to
> > >> > > >>>>>> create/delete
> > >> > > >>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> > requires
> > >> > the
> > >> > > >>>>> volume to
> > >> > > >>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>> volume to be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> > >> > context.
> > >> > > >>> To
> > >> > > >>>>> ensure
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>>>>> are in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
> > these
> > >> > > >>>>> definitions,
> > >> > > >>>>>>>>> refers to
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>> physical
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> > >> > correct?
> > >> > > >>>>> Minimally,
> > >> > > >>>>>> we
> > >> > > >>>>>>>>>>>>> must
> > >> > > >>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>> able
> > >> > > >>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> > storage
> > >> > device
> > >> > > >>> to
> > >> > > >>>>> move bits
> > >> > > >>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>> one place
> > >> > > >>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > >> > > >>> Optionally, a
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>> may
> > >> > > >>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> > >> > Therefore,
> > >> > > >>>>> we can have
> > >> > > >>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> unmanaged
> > >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
> > SR,
> > >> > and
> > >> > > >>> we
> > >> > > >>>>> can have a
> > >> > > >>>>>>>>>>>>> managed
> > >> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> > CloudStack is
> > >> > > >>> capable
> > >> > > >>>>> of
> > >> > > >>>>>>>>> allocating
> > >> > > >>>>>>>>>>>>>>>>>> LUNs and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> > >> > CloudStack
> > >> > > >>> may
> > >> > > >>>>> be able
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> manage a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> > leave it
> > >> > > >>>>> unmanaged by
> > >> > > >>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>> (e.g.
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
> > and
> > >> > the
> > >> > > >>>>> operator has
> > >> > > >>>>>>>>> chosen
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> > CloudStack).
> > >> > Does
> > >> > > >>> my
> > >> > > >>>>>> reasoning
> > >> > > >>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>> sense?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> > above
> > >> > are
> > >> > > >>>>> reasonable, it
> > >> > > >>>>>>>>> seems
> > >> > > >>>>>>>>>>>>>>>>>>>> appropriate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> > >> > concerns
> > >> > > >>> from
> > >> > > >>>>>>>>> StoragePoolType,
> > >> > > >>>>>>>>>>>>>>>>>> add the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> > >> > attached
> > >> > > >>> driver
> > >> > > >>>>> that
> > >> > > >>>>>>>>>>> indicates
> > >> > > >>>>>>>>>>>>>>>>>> whether
> > >> > > >>>>>>>>>>>>>>>>>>>>>> or
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
> > and
> > >> > > >>> establish
> > >> > > >>>>> a
> > >> > > >>>>>>>>> abstraction
> > >> > > >>>>>>>>>>>>>>>>>>>>>> representing a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> > separate
> > >> > that
> > >> > > >>> is
> > >> > > >>>>>>>>> associated
> > >> > > >>>>>>>>>>>>>>>>>> with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>> volume.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> > >> > hypervisor
> > >> > > >>>>> drivers can
> > >> > > >>>>>>>>>>>>> declare
> > >> > > >>>>>>>>>>>>>>>>>> which
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> > >> > encounter
> > >> > > >>> a
> > >> > > >>>>> device
> > >> > > >>>>>> managed
> > >> > > >>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> > >> > exposed
> > >> > > >>> by
> > >> > > >>>>> the driver to
> > >> > > >>>>>>>>>>>>> automate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
> > make
> > >> > > >>> sense,
> > >> > > >>>>> then we can
> > >> > > >>>>>>>>> sit
> > >> > > >>>>>>>>>>>>>>>>>> down and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> > >> > Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > >> > > >>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> > >> > between
> > >> > > >>> the
> > >> > > >>>>> current
> > >> > > >>>>>> iSCSI
> > >> > > >>>>>>>>>>>>> type
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> > has to
> > >> > go in
> > >> > > >>> and
> > >> > > >>>>> create
> > >> > > >>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> Primary
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> > point in
> > >> > time,
> > >> > > >>>>> the
> > >> > > >>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>> volume must
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> > >> > allocated).
> > >> > > >>>>> Future
> > >> > > >>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
> > was
> > >> > created
> > >> > > >>>>> behind the
> > >> > > >>>>>>>>>>>>> scenes.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
> > has
> > >> > to
> > >> > > >>> go in
> > >> > > >>>>> and
> > >> > > >>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>> Primary
> > >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
> > and
> > >> > delete
> > >> > > >>>>> volumes on
> > >> > > >>>>>>>>> its
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> > enabled via
> > >> > the
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> framework). When
> > >> > > >>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> > volume
> > >> > that
> > >> > > >>> was
> > >> > > >>>>> created,
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> framework
> > >> > > >>>>>>>>>>>>>>>>>>>>>> tells
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> > >> > volume.
> > >> > > >>> After
> > >> > > >>>>> this is
> > >> > > >>>>>>>>> done,
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> > question is
> > >> > called.
> > >> > > >>>>> No
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> > >> > volume
> > >> > > >>> was
> > >> > > >>>>> just
> > >> > > >>>>>>>>> created.
> > >> > > >>>>>>>>>>>>> The
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> > created.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 3:21
> > >> > PM,
> > >> > > >>> Mike
> > >> > > >>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
> > so I
> > >> > should
> > >> > > >>>>> probably
> > >> > > >>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>> defined them
> > >> > > >>>>>>>>>>>>>>>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
> > that is
> > >> > > >>> pre-
> > >> > > >>>>> allocated
> > >> > > >>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>> an admin
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
> > a
> > >> > > >>> hypervisor
> > >> > > >>>>> data
> > >> > > >>>>>>>>> structure
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> > XenServer
> > >> > SR),
> > >> > > >>> then
> > >> > > >>>>> that
> > >> > > >>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> > CloudStack.
> > >> > Disks
> > >> > > >>> (VDI)
> > >> > > >>>>> are later
> > >> > > >>>>>>>>>>>>> placed
> > >> > > >>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> > needed. In
> > >> > > >>> these
> > >> > > >>>>> cases, the
> > >> > > >>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> > structure is
> > >> > > >>> already
> > >> > > >>>>> in place
> > >> > > >>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> simply
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> > >> > structure
> > >> > > >>> to
> > >> > > >>>>> the VM in
> > >> > > >>>>>>>>>>>>>>>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> > Storage
> > >> > that is
> > >> > > >>> not
> > >> > > >>>>>>>>>>> pre-allocated.
> > >> > > >>>>>>>>>>>>>>>>>> Instead
> > >> > > >>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
> > this
> > >> > could
> > >> > > >>> be a
> > >> > > >>>>> SAN
> > >> > > >>>>>> (not
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> volume on
> > >> > > >>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> > hypervisor
> > >> > data
> > >> > > >>>>> structure
> > >> > > >>>>>>>>> must be
> > >> > > >>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> > >> > because
> > >> > > >>> these
> > >> > > >>>>> types of
> > >> > > >>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>> have not
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
> > data
> > >> > > >>>>> structure by an
> > >> > > >>>>>>>>>>> admin.
> > >> > > >>>>>>>>>>>>>>>>>> Once
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> > >> > XenServer
> > >> > > >>> for
> > >> > > >>>>> this
> > >> > > >>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>> attaches
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
> > to the
> > >> > VM
> > >> > > >>> in
> > >> > > >>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 3:13
> > >> > PM,
> > >> > > >>>>> John Burwell
> > >> > > >>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> > >> > implementation of
> > >> > > >>> the
> > >> > > >>>>> Dynamic
> > >> > > >>>>>> type
> > >> > > >>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>> behavior
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> > which
> > >> > why I
> > >> > > >>> ask
> > >> > > >>>>> about the
> > >> > > >>>>>>>>>>>>>>>>>> difference.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> > what is
> > >> > the
> > >> > > >>>>> definition
> > >> > > >>>>>> of
> > >> > > >>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>> Dynamic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
> > PM,
> > >> > > >>> Mike
> > >> > > >>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
> > iSCSI
> > >> > type
> > >> > > >>> is
> > >> > > >>>>>> uniquely
> > >> > > >>>>>>>>>>> used
> > >> > > >>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> > Primary
> > >> > > >>> Storage
> > >> > > >>>>> that is
> > >> > > >>>>>>>>> directly
> > >> > > >>>>>>>>>>>>>>>>>> based on
> > >> > > >>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> > skip the
> > >> > > >>> step of
> > >> > > >>>>> going
> > >> > > >>>>>> to
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
> > based
> > >> > on
> > >> > > >>> that
> > >> > > >>>>> iSCSI
> > >> > > >>>>>>>>> target as
> > >> > > >>>>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
> > only
> > >> > > >>>>> supported for
> > >> > > >>>>>>>>>>>>>>>>>> XenServer.
> > >> > > >>>>>>>>>>>>>>>>>>>> For
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> > must
> > >> > first go
> > >> > > >>> to
> > >> > > >>>>> the
> > >> > > >>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>> perform
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
> > what
> > >> > RBD is.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > >> > 2:13
> > >> > > >>> PM,
> > >> > > >>>>> John
> > >> > > >>>>>> Burwell
> > >> > > >>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
> > the
> > >> > code,
> > >> > > >>> what
> > >> > > >>>>> is the
> > >> > > >>>>>>>>>>> difference
> > >> > > >>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> > Why
> > >> > isn't
> > >> > > >>> RBD
> > >> > > >>>>> considered
> > >> > > >>>>>>>>>>>>> Dynamic?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> > 3:46
> > >> > PM,
> > >> > > >>> Mike
> > >> > > >>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> > >> > storage is
> > >> > > >>>>> defined in
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > >> > > >>>>> StoragePoolType {
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
> > //
> > >> > local
> > >> > > >>>>> directory
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > NetworkFilesystem(true),
> > >> > > >>> //
> > >> > > >>>>> NFS or CIFS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> > >> > shared
> > >> > > >>> LUN,
> > >> > > >>>>> with a
> > >> > > >>>>>>>>>>> clusterfs
> > >> > > >>>>>>>>>>>>>>>>>>>> overlay
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
> > e.g.,
> > >> > ZFS
> > >> > > >>>>> Comstar
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
> > iso
> > >> > image
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> > >> > XenServer
> > >> > > >>> local
> > >> > > >>>>> LVM SR
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > SharedMountPoint(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> > >> > VMware
> > >> > > >>> VMFS
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
> > for
> > >> > > >>>>> XenServer, Storage
> > >> > > >>>>>>>>> Pool
> > >> > > >>>>>>>>>>>>>>>>>> is set
> > >> > > >>>>>>>>>>>>>>>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> > >> > XenServer
> > >> > > >>> local
> > >> > > >>>>> EXT SR
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> > >> > dynamic,
> > >> > > >>>>> zone-wide
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > StoragePoolType(boolean
> > >> > > >>>>> shared) {
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> > shared;
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> > >> > isShared() {
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > >> > 1:41
> > >> > > >>> PM,
> > >> > > >>>>> Mike
> > >> > > >>>>>>>>> Tutkowski
> > >> > > >>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> > let's say
> > >> > > >>> another
> > >> > > >>>>> storage
> > >> > > >>>>>>>>>>> company
> > >> > > >>>>>>>>>>>>>>>>>> wants
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> > leverage its
> > >> > > >>> Quality
> > >> > > >>>>> of
> > >> > > >>>>>> Service
> > >> > > >>>>>>>>>>>>>>>>>> feature. It
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > storage, as
> > >> > well.
> > >> > > >>>>> They would
> > >> > > >>>>>>>>> need
> > >> > > >>>>>>>>>>>>> only
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> > made
> > >> > the
> > >> > > >>>>> necessary
> > >> > > >>>>>>>>> changes to
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
> > plug-
> > >> > in.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > >> > at
> > >> > > >>> 1:39
> > >> > > >>>>> PM, Mike
> > >> > > >>>>>>>>>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> > imply
> > >> > the
> > >> > > >>>>> XenServer code
> > >> > > >>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> > >> > attach
> > >> > > >>> logic is
> > >> > > >>>>> now aware
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>> dynamic,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
> > an
> > >> > > >>>>> implementation of
> > >> > > >>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>> kind of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
> > to 4.2
> > >> > > >>> with
> > >> > > >>>>> Edison's
> > >> > > >>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> framework
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
> > a
> > >> > new
> > >> > > >>>>> framework that
> > >> > > >>>>>>>>>>> supported
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> > >> > dynamically.
> > >> > > >>>>> However, when I
> > >> > > >>>>>>>>>>>>>>>>>> visited with
> > >> > > >>>>>>>>>>>>>>>>>>>>>> him
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> > realized
> > >> > that
> > >> > > >>> it
> > >> > > >>>>> was not
> > >> > > >>>>>>>>>>>>>>>>>> complete. We
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> > CloudStack
> > >> > could
> > >> > > >>> do
> > >> > > >>>>> with these
> > >> > > >>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>> unless
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> > >> > recognize
> > >> > > >>> this
> > >> > > >>>>> new type of
> > >> > > >>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> > >> > structure.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > >> > at
> > >> > > >>> 1:28
> > >> > > >>>>> PM, John
> > >> > > >>>>>>>>> Burwell
> > >> > > >>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
> > odd
> > >> > to me
> > >> > > >>>>> that any
> > >> > > >>>>>>>>>>> operation
> > >> > > >>>>>>>>>>>>>>>>>> in the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
> > care
> > >> > > >>> about
> > >> > > >>>>> details. I
> > >> > > >>>>>>>>> expect
> > >> > > >>>>>>>>>>>>>>>>>> to see
> > >> > > >>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
> > of
> > >> > > >>> operations
> > >> > > >>>>> that can be
> > >> > > >>>>>>>>>>>>>>>>>>>> composed/driven
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > implementations to
> > >> > > >>>>> allocate
> > >> > > >>>>>> space/create
> > >> > > >>>>>>>>>>>>>>>>>> structures
> > >> > > >>>>>>>>>>>>>>>>>>>>>> per
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
> > this
> > >> > > >>>>> dependency, we are
> > >> > > >>>>>>>>> going
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> end
> > >> > > >>>>>>>>>>>>>>>>>>>>>> with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
> > will
> > >> > make
> > >> > > >>> the
> > >> > > >>>>> system
> > >> > > >>>>>>>>>>>>> increasingly
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
> > I
> > >> > > >>>>> understanding that the
> > >> > > >>>>>>>>> Xen
> > >> > > >>>>>>>>>>>>>>>>>> specific
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > >> > > >>>>> CitrixResourceBase
> > >> > > >>>>>> class?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > >> > 2013 at
> > >> > > >>> 3:12
> > >> > > >>>>> PM, Mike
> > >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
> > this
> > >> > in a
> > >> > > >>> bit
> > >> > > >>>>> more
> > >> > > >>>>>>>>> detail:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
> > and
> > >> > aside
> > >> > > >>> from
> > >> > > >>>>> one setup
> > >> > > >>>>>>>>>>>>> method
> > >> > > >>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
> > a
> > >> > > >>> volume on
> > >> > > >>>>> the
> > >> > > >>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> system,
> > >> > > >>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>> go
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
> > data
> > >> > > >>> structure
> > >> > > >>>>> to make
> > >> > > >>>>>> use
> > >> > > >>>>>>>>> of
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > >> > > >>> XenServer
> > >> > > >>>>> or a
> > >> > > >>>>>> datastore
> > >> > > >>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>> ESX(i)).
> > >> > > >>>>>>>>>>>>>>>>>>>>>> VMs
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> > >> > storage
> > >> > > >>>>> system's volume.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
> > new
> > >> > > >>> storage
> > >> > > >>>>> framework,
> > >> > > >>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>> need
> > >> > > >>>>>>>>>>>>>>>>>>>> no
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
> > easily
> > >> > > >>> create
> > >> > > >>>>> a 1:1
> > >> > > >>>>>>>>>>>>> relationship
> > >> > > >>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
> > the
> > >> > VM's
> > >> > > >>> data
> > >> > > >>>>> disk
> > >> > > >>>>>>>>> (necessary
> > >> > > >>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
> > write
> > >> > a
> > >> > > >>> plug-
> > >> > > >>>>> in that is
> > >> > > >>>>>>>>> called
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> > >> > needed.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
> > that
> > >> > the
> > >> > > >>>>> storage
> > >> > > >>>>>> framework
> > >> > > >>>>>>>>> did
> > >> > > >>>>>>>>>>>>>>>>>> not
> > >> > > >>>>>>>>>>>>>>>>>>>>>> address
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > >> > > >>> hypervisor-
> > >> > > >>>>> specific data
> > >> > > >>>>>>>>>>>>>>>>>> structure
> > >> > > >>>>>>>>>>>>>>>>>>>> when
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > attach/detach.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
> > the
> > >> > case,
> > >> > > >>> I've
> > >> > > >>>>> been
> > >> > > >>>>>>>>> enhancing
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> to do
> > >> > > >>>>>>>>>>>>>>>>>>>> so.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
> > and
> > >> > > >>>>> submitted. I've got
> > >> > > >>>>>>>>> ESX(i)
> > >> > > >>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> my
> > >> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
> > the
> > >> > 4.2
> > >> > > >>>>> freeze date.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> > help a
> > >> > bit? :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> > 3,
> > >> > 2013 at
> > >> > > >>>>> 1:03 PM, Mike
> > >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> > >> > plug-in -
> > >> > > >>> by
> > >> > > >>>>> itself - is
> > >> > > >>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
> > with
> > >> > the
> > >> > > >>>>> volume-attach
> > >> > > >>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>> (in the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> > calls
> > >> > into
> > >> > > >>> the
> > >> > > >>>>> plug-in to
> > >> > > >>>>>>>>> have
> > >> > > >>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>> create a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
> > time
> > >> > > >>> comes to
> > >> > > >>>>> attach the
> > >> > > >>>>>>>>> volume
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
> > smart
> > >> > > >>> enough
> > >> > > >>>>> to recognize
> > >> > > >>>>>>>>> it's
> > >> > > >>>>>>>>>>>>>>>>>> being
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> > >> > volume
> > >> > > >>> has
> > >> > > >>>>> just been
> > >> > > >>>>>>>>> created)
> > >> > > >>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>> create,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> > (for
> > >> > > >>>>> XenServer) or a
> > >> > > >>>>>>>>> datastore
> > >> > > >>>>>>>>>>>>>>>>>> (for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
> > was
> > >> > just
> > >> > > >>>>> created.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> > >> > spending
> > >> > > >>> most
> > >> > > >>>>> of my time
> > >> > > >>>>>>>>>>>>> recently
> > >> > > >>>>>>>>>>>>>>>>>> making
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> > >> > code.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> > clear it
> > >> > up?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> > Jun 3,
> > >> > 2013
> > >> > > >>> at
> > >> > > >>>>> 12:48 PM,
> > >> > > >>>>>> John
> > >> > > >>>>>>>>>>>>>>>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> > explain
> > >> > > >>> why
> > >> > > >>>>> the the storage
> > >> > > >>>>>>>>>>>>> driver
> > >> > > >>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> > 2013,
> > >> > at
> > >> > > >>> 1:21
> > >> > > >>>>> PM, Mike
> > >> > > >>>>>>>>>>>>> Tutkowski
> > >> > > >>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> > >> > ultimately I
> > >> > > >>>>> would like to
> > >> > > >>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>> all
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
> > I
> > >> > think
> > >> > > >>> I'm
> > >> > > >>>>> just out of
> > >> > > >>>>>>>>> time
> > >> > > >>>>>>>>>>>>>>>>>> for 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> > now this
> > >> > > >>> plug-in
> > >> > > >>>>> supports
> > >> > > >>>>>>>>>>>>> XenServer.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> > to 4.2
> > >> > > >>> feature
> > >> > > >>>>> freeze, I
> > >> > > >>>>>>>>> have
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> working
> > >> > > >>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
> > to
> > >> > be
> > >> > > >>> clear,
> > >> > > >>>>> this is
> > >> > > >>>>>> all
> > >> > > >>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> regards
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > Compute
> > >> > > >>>>> Offerings post
> > >> > > >>>>>> 4.2.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> > Jun 3,
> > >> > > >>> 2013
> > >> > > >>>>> at 11:14 AM,
> > >> > > >>>>>>>>> Kelcey
> > >> > > >>>>>>>>>>>>>>>>>> Jamison
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > kelcey@bbits.ca
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
> > any
> > >> > plan
> > >> > > >>> on
> > >> > > >>>>> supporting
> > >> > > >>>>>>>>> KVM in
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>> patch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> > Original
> > >> > > >>>>> Message -----
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
> > "Mike
> > >> > > >>>>> Tutkowski" <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > >> > > >>>>> dev@cloudstack.apache.org
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> > >> > Monday,
> > >> > > >>> June
> > >> > > >>>>> 3, 2013
> > >> > > >>>>>>>>> 10:12:32 AM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
> > Re:
> > >> > > >>> [MERGE]
> > >> > > >>>>>>>>>>> disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>> MASTER
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
> > on
> > >> > > >>> merging
> > >> > > >>>>> Wei's feature
> > >> > > >>>>>>>>>>> first,
> > >> > > >>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mine.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
> > feature
> > >> > is
> > >> > > >>> for
> > >> > > >>>>> KVM only,
> > >> > > >>>>>>>>> then
> > >> > > >>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> is a
> > >> > > >>>>>>>>>>>>>>>>>>>> non
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
> > 4.2.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> > Jun
> > >> > 3,
> > >> > > >>> 2013
> > >> > > >>>>> at 8:55 AM,
> > >> > > >>>>>> Wei
> > >> > > >>>>>>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > ustcweizhou@gmail.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> > the
> > >> > billing,
> > >> > > >>> as
> > >> > > >>>>> no one works
> > >> > > >>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>> billing
> > >> > > >>>>>>>>>>>>>>>>>>>> now,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > billing
> > >> > by
> > >> > > >>>>> themselves. They
> > >> > > >>>>>>>>> can
> > >> > > >>>>>>>>>>>>> get
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > disk_offering of
> > >> > > >>> a
> > >> > > >>>>> VMs and
> > >> > > >>>>>> volumes
> > >> > > >>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
> > user
> > >> > the
> > >> > > >>>>> exact
> > >> > > >>>>>> limitation
> > >> > > >>>>>>>>>>>>> value
> > >> > > >>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > >> > limitation
> > >> > > >>> for
> > >> > > >>>>> nics as
> > >> > > >>>>>> well.
> > >> > > >>>>>>>>> I
> > >> > > >>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>> work
> > >> > > >>>>>>>>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> > part of
> > >> > I/O
> > >> > > >>>>> throttling?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
> > my
> > >> > > >>>>> misunstand the second
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
> > with
> > >> > > >>> what
> > >> > > >>>>> you said about
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>> two
> > >> > > >>>>>>>>>>>>>>>>>>>> features.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/6/3
> > >> > John
> > >> > > >>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Jun 3,
> > >> > 2013,
> > >> > > >>> at
> > >> > > >>>>> 2:13 AM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> > John,
> > >> > Mike
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > hope
> > >> > Mike's
> > >> > > >>>>> aswer helps you.
> > >> > > >>>>>>>>> I am
> > >> > > >>>>>>>>>>>>>>>>>> trying
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> > >> > think
> > >> > > >>> billing
> > >> > > >>>>> should
> > >> > > >>>>>>>>> depend
> > >> > > >>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>>> IO
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > limitation.
> > >> > > >>> Please
> > >> > > >>>>> review
> > >> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
> > >> > > >>>>>>>>>>>>>>>>>>>> you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_io_stat
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> > the
> > >> > IO
> > >> > > >>>>> statistics
> > >> > > >>>>>> including
> > >> > > >>>>>>>>>>>>>>>>>> bytes/iops
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > individual
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > virtual
> > >> > > >>> machine.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Going by
> > >> > the
> > >> > > >>> AWS
> > >> > > >>>>> model,
> > >> > > >>>>>> customers
> > >> > > >>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>> billed
> > >> > > >>>>>>>>>>>>>>>>>>>>>> more
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > provisioned
> > >> > > >>> IOPS,
> > >> > > >>>>> as well as,
> > >> > > >>>>>> for
> > >> > > >>>>>>>>>>>>> those
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://aws.amazon.com/ebs/).
> > >> > > >>>>>> I
> > >> > > >>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>> imagine
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > option to
> > >> > > >>> employ
> > >> > > >>>>> similar cost
> > >> > > >>>>>>>>>>> models.
> > >> > > >>>>>>>>>>>>>>>>>> Could
> > >> > > >>>>>>>>>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > billing
> > >> > model
> > >> > > >>> in
> > >> > > >>>>> the current
> > >> > > >>>>>>>>> patch?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > Do
> > >> > you
> > >> > > >>> mean
> > >> > > >>>>> IOPS runtime
> > >> > > >>>>>>>>>>> change?
> > >> > > >>>>>>>>>>>>>>>>>> KVM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > limitation for
> > >> > > >>> a
> > >> > > >>>>> running
> > >> > > >>>>>> virtual
> > >> > > >>>>>>>>>>>>>>>>>> machine
> > >> > > >>>>>>>>>>>>>>>>>>>>>> through
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > However,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > CloudStack
> > >> > > >>> does
> > >> > > >>>>> not support
> > >> > > >>>>>>>>>>> changing
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > offering
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > (computer
> > >> > > >>>>> offering or disk
> > >> > > >>>>>>>>>>>>> offering).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > meant at
> > >> > the
> > >> > > >>>>> Java interface
> > >> > > >>>>>>>>> level.
> > >> > > >>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>> apologize
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > more
> > >> > > >>>>> generalize allocation
> > >> > > >>>>>>>>>>>>>>>>>> algorithms
> > >> > > >>>>>>>>>>>>>>>>>>>> with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > describe
> > >> > the
> > >> > > >>>>> service guarantees
> > >> > > >>>>>>>>>>>>>>>>>> provided by a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > It is a
> > >> > > >>> good
> > >> > > >>>>> question.
> > >> > > >>>>>>>>> Maybe it
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>> better
> > >> > > >>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> disk_io_throttling as Mike
> > >> > > >>>>>>>>> needs to
> > >> > > >>>>>>>>>>>>>>>>>> consider
> > >> > > >>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > hypervisor
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > type, I
> > >> > think.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> > >> > expand
> > >> > > >>> on
> > >> > > >>>>> my thoughts
> > >> > > >>>>>> in a
> > >> > > >>>>>>>>>>>>> later
> > >> > > >>>>>>>>>>>>>>>>>>>>>> response
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > touch
> > >> > points
> > >> > > >>>>> between these two
> > >> > > >>>>>>>>>>>>>>>>>> features. I
> > >> > > >>>>>>>>>>>>>>>>>>>>>> think
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > need
> > >> > to be
> > >> > > >>>>> merged before
> > >> > > >>>>>>>>>>>>>>>>>> SolidFire, but
> > >> > > >>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > coordination
> > >> > > >>>>> between the
> > >> > > >>>>>> branches
> > >> > > >>>>>>>>>>>>>>>>>> (possibly
> > >> > > >>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> disk_io_throttling)
> > >> > > >>>>> to
> > >> > > >>>>>>>>> coordinate on
> > >> > > >>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>> issue.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/6/3
> > >> > > >>> John
> > >> > > >>>>> Burwell <
> > >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Mike,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > The
> > >> > things I
> > >> > > >>>>> want to
> > >> > > >>>>>> understand
> > >> > > >>>>>>>>>>> are
> > >> > > >>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
> > Is
> > >> > there
> > >> > > >>>>> value in
> > >> > > >>>>>> capturing
> > >> > > >>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>> policies
> > >> > > >>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > common
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > data
> > >> > model
> > >> > > >>> (e.g.
> > >> > > >>>>> for
> > >> > > >>>>>>>>> billing/usage
> > >> > > >>>>>>>>>>>>>>>>>>>> purposes,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> > >> > Should
> > >> > > >>> there
> > >> > > >>>>> be a common
> > >> > > >>>>>>>>>>>>>>>>>> interface model
> > >> > > >>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > provisioning
> > >> > > >>> at
> > >> > > >>>>> runtime?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
> > How
> > >> > are
> > >> > > >>>>> conflicting
> > >> > > >>>>>>>>> provisioned
> > >> > > >>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > hypervisor
> > >> > > >>> and
> > >> > > >>>>> storage device
> > >> > > >>>>>>>>>>>>>>>>>> reconciled?
> > >> > > >>>>>>>>>>>>>>>>>>>> In
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > where
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > lead
> > >> > to
> > >> > > >>>>> believe (and
> > >> > > >>>>>> billed)
> > >> > > >>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>> more
> > >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > storage
> > >> > > >>> device
> > >> > > >>>>> has been
> > >> > > >>>>>>>>> configured
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> > a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > consistent
> > >> > > >>>>> configuration
> > >> > > >>>>>>>>> between a
> > >> > > >>>>>>>>>>>>>>>>>> VM and a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > time,
> > >> > but a
> > >> > > >>>>> later
> > >> > > >>>>>> modification
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> inconsistency.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Jun
> > >> > 2,
> > >> > > >>> 2013,
> > >> > > >>>>> at 8:38 PM,
> > >> > > >>>>>>>>> Mike
> > >> > > >>>>>>>>>>>>>>>>>> Tutkowski
> > >> > > >>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> mike.tutkowski@solidfire.com>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> > John,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > believe
> > >> > > >>> Wei's
> > >> > > >>>>> feature deals
> > >> > > >>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>> controlling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > the
> > >> > > >>> hypervisor
> > >> > > >>>>> side.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > My
> > >> > feature
> > >> > > >>> is
> > >> > > >>>>> focused on
> > >> > > >>>>>>>>>>>>> controlling
> > >> > > >>>>>>>>>>>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>>>>>> from
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > hope
> > >> > that
> > >> > > >>>>> helps. :)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Sun,
> > >> > Jun
> > >> > > >>> 2,
> > >> > > >>>>> 2013 at 6:35
> > >> > > >>>>>> PM,
> > >> > > >>>>>>>>>>>>> John
> > >> > > >>>>>>>>>>>>>>>>>>>> Burwell
> > >> > > >>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > jburwell@basho.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > My
> > >> > > >>> opinion is
> > >> > > >>>>> that no
> > >> > > >>>>>> features
> > >> > > >>>>>>>>>>>>>>>>>> should be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > functional
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > issues
> > >> > > >>> have
> > >> > > >>>>> been resolved
> > >> > > >>>>>> and
> > >> > > >>>>>>>>> it
> > >> > > >>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>> ready
> > >> > > >>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > total
> > >> > Ops
> > >> > > >>> vs
> > >> > > >>>>> discrete
> > >> > > >>>>>>>>> read/write
> > >> > > >>>>>>>>>>>>>>>>>> ops issue
> > >> > > >>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> > >> > reviewed
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > I
> > >> > > >>> don't
> > >> > > >>>>> think this
> > >> > > >>>>>>>>> criteria
> > >> > > >>>>>>>>>>>>>>>>>> has been
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Also,
> > >> > how
> > >> > > >>>>> does this work
> > >> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> > >> > > >>>>>>>>>>>>>>>>>> )?
> > >> > > >>>>>>>>>>>>>>>>>>>> As I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > also
> > >> > > >>> involves
> > >> > > >>>>> provisioned
> > >> > > >>>>>>>>> IOPS. I
> > >> > > >>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>> like
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > scenario
> > >> > > >>>>> where provisioned
> > >> > > >>>>>>>>> IOPS
> > >> > > >>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> KVM and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > unnecessarily
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> incompatible.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > Jun 1,
> > >> > > >>> 2013,
> > >> > > >>>>> at 6:47 AM,
> > >> > > >>>>>>>>> Wei
> > >> > > >>>>>>>>>>>>>>>>>> ZHOU <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > ustcweizhou@gmail.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Sure. I
> > >> > will
> > >> > > >>>>> change it next
> > >> > > >>>>>>>>> week.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > 2013/6/1
> > >> > > >>>>> Wido den Hollander
> > >> > > >>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 06/01/2013 08:24 AM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Exactly. I
> > >> > > >>> have
> > >> > > >>>>> pushed the
> > >> > > >>>>>>>>>>>>> features
> > >> > > >>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > anyone
> > >> > > >>>>> object thems for
> > >> > > >>>>>>>>>>>>> technical
> > >> > > >>>>>>>>>>>>>>>>>>>> reason
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > them.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > For
> > >> > the
> > >> > > >>> sake
> > >> > > >>>>> of clarity I
> > >> > > >>>>>> just
> > >> > > >>>>>>>>>>>>> want
> > >> > > >>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>> mention
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > change
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > the
> > >> > total
> > >> > > >>> IOps
> > >> > > >>>>> to R/W IOps
> > >> > > >>>>>>>>> asap
> > >> > > >>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>> that we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > only
> > >> > total
> > >> > > >>> IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > You
> > >> > laid
> > >> > > >>> the
> > >> > > >>>>> groundwork for
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>> I/O
> > >> > > >>>>>>>>>>>>>>>>>>>>>> throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > should
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > however
> > >> > > >>>>> prevent that we
> > >> > > >>>>>> create
> > >> > > >>>>>>>>>>>>>>>>>> legacy from
> > >> > > >>>>>>>>>>>>>>>>>>>>>> day
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > 2013/5/31
> > >> > > >>>>> Wido den
> > >> > > >>>>>> Hollander <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 05/31/2013 03:59 PM, John
> > >> > > >>>>>>>>>>>>> Burwell
> > >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > +1 --
> > >> > this
> > >> > > >>>>> enhancement must
> > >> > > >>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> discretely
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > don't
> > >> > see
> > >> > > >>> how
> > >> > > >>>>> it could be
> > >> > > >>>>>>>>> fixed
> > >> > > >>>>>>>>>>>>>>>>>> later
> > >> > > >>>>>>>>>>>>>>>>>>>>>> because
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > correctly
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > split
> > >> > total
> > >> > > >>> IOPS
> > >> > > >>>>> into read
> > >> > > >>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> write.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > with a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > total
> > >> > > >>>>> unless/until we
> > >> > > >>>>>> decided
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> break
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > What
> > >> > Wei
> > >> > > >>>>> meant was merging
> > >> > > >>>>>> it
> > >> > > >>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>> master
> > >> > > >>>>>>>>>>>>>>>>>>>>>> now
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 4.2
> > >> > branch
> > >> > > >>>>> and add Read /
> > >> > > >>>>>>>>> Write
> > >> > > >>>>>>>>>>>>> IOps
> > >> > > >>>>>>>>>>>>>>>>>>>> before
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > will be
> > >> > > >>>>> released with Read
> > >> > > >>>>>> and
> > >> > > >>>>>>>>>>>>> Write
> > >> > > >>>>>>>>>>>>>>>>>>>> instead
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > This is
> > >> > to
> > >> > > >>>>> make the May 31st
> > >> > > >>>>>>>>>>>>> feature
> > >> > > >>>>>>>>>>>>>>>>>>>> freeze
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > moves
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (see
> > >> > other
> > >> > > >>>>> threads) then it
> > >> > > >>>>>>>>> won't
> > >> > > >>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>> necessary
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > also
> > >> > > >>>>> completely agree that
> > >> > > >>>>>>>>>>> there
> > >> > > >>>>>>>>>>>>>>>>>> is no
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk
> > >> > I/O.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Thanks,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > John
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > May 31,
> > >> > > >>>>> 2013, at 9:51 AM,
> > >> > > >>>>>>>>> Wido
> > >> > > >>>>>>>>>>>>>>>>>> den
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wido@widodh.nl
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 05/31/2013 03:13 PM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi
> > >> > Wido,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Thanks.
> > >> > > >>> Good
> > >> > > >>>>> question.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > thought
> > >> > > >>>>> about at the
> > >> > > >>>>>>>>>>> beginning.
> > >> > > >>>>>>>>>>>>>>>>>>>> Finally I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > difference
> > >> > > >>> of
> > >> > > >>>>> read and write
> > >> > > >>>>>>>>>>>>> mainly
> > >> > > >>>>>>>>>>>>>>>>>>>> because
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > not
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > care
> > >> > the
> > >> > > >>>>> difference of sent
> > >> > > >>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>> received
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > That
> > >> > > >>>>> reasoning seems odd.
> > >> > > >>>>>>>>>>>>>>>>>> Networking and
> > >> > > >>>>>>>>>>>>>>>>>>>>>> disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > different.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Disk
> > >> > I/O is
> > >> > > >>>>> much more
> > >> > > >>>>>>>>> expensive
> > >> > > >>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>> most
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > bandwith.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> Implementing
> > >> > > >>>>> it will be some
> > >> > > >>>>>>>>>>>>>>>>>> copy-paste
> > >> > > >>>>>>>>>>>>>>>>>>>>>> work.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> implemented
> > >> > > >>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > few
> > >> > days.
> > >> > > >>> For
> > >> > > >>>>> the deadline
> > >> > > >>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>> feature
> > >> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > after
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > that ,
> > >> > if
> > >> > > >>>>> needed.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> > >> > think it's
> > >> > > >>> a
> > >> > > >>>>> feature we
> > >> > > >>>>>>>>> can't
> > >> > > >>>>>>>>>>>>>>>>>> miss. But
> > >> > > >>>>>>>>>>>>>>>>>>>>>> if
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > window
> > >> > > >>> we
> > >> > > >>>>> have to make sure
> > >> > > >>>>>> we
> > >> > > >>>>>>>>>>>>> don't
> > >> > > >>>>>>>>>>>>>>>>>>>> release
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > fix
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > in
> > >> > 4.3,
> > >> > > >>> that
> > >> > > >>>>> will confuse
> > >> > > >>>>>>>>>>>>> users.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > 2013/5/31
> > >> > > >>>>> Wido den
> > >> > > >>>>>> Hollander <
> > >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi Wei,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > On
> > >> > > >>>>> 05/30/2013 06:03 PM, Wei
> > >> > > >>>>>>>>> ZHOU
> > >> > > >>>>>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Hi,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >> > would
> > >> > > >>> like to
> > >> > > >>>>> merge
> > >> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >> > nobody
> > >> > > >>>>> object, I will
> > >> > > >>>>>> merge
> > >> > > >>>>>>>>>>>>> into
> > >> > > >>>>>>>>>>>>>>>>>> master
> > >> > > >>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > The
> > >> > > >>> purpose
> > >> > > >>>>> is :
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Virtual
> > >> > > >>>>> machines are running
> > >> > > >>>>>>>>> on
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>> same
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > storage or
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > share
> > >> > > >>> strage).
> > >> > > >>>>> Because of
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>> rate
> > >> > > >>>>>>>>>>>>>>>>>>>>>> limitation
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > iops),
> > >> > if
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > one
> > >> > VM
> > >> > > >>> has
> > >> > > >>>>> large disk
> > >> > > >>>>>>>>> operation,
> > >> > > >>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>> may
> > >> > > >>>>>>>>>>>>>>>>>>>>>> affect
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > performance
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > other
> > >> > VMs
> > >> > > >>>>> running on the
> > >> > > >>>>>> same
> > >> > > >>>>>>>>>>>>>>>>>> storage
> > >> > > >>>>>>>>>>>>>>>>>>>>>> device.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> > is
> > >> > > >>> neccesary
> > >> > > >>>>> to set the
> > >> > > >>>>>>>>>>> maximum
> > >> > > >>>>>>>>>>>>>>>>>> rate
> > >> > > >>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > VMs.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Looking at
> > >> > > >>> the
> > >> > > >>>>> code I see
> > >> > > >>>>>> you
> > >> > > >>>>>>>>>>> make
> > >> > > >>>>>>>>>>>>>>>>>> no
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Qemu
> > >> > and
> > >> > > >>>>> libvirt support
> > >> > > >>>>>>>>> setting
> > >> > > >>>>>>>>>>>>>>>>>> both a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps
> > >> > > >>> which
> > >> > > >>>>> could benefit a
> > >> > > >>>>>>>>> lot of
> > >> > > >>>>>>>>>>>>>>>>>> users.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > It's
> > >> > also
> > >> > > >>>>> strange, in the
> > >> > > >>>>>>>>> polling
> > >> > > >>>>>>>>>>>>>>>>>> side you
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps,
> > >> > but
> > >> > > >>> on
> > >> > > >>>>> the throttling
> > >> > > >>>>>>>>> side
> > >> > > >>>>>>>>>>>>>>>>>> you only
> > >> > > >>>>>>>>>>>>>>>>>>>> go
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Write
> > >> > IOps
> > >> > > >>> are
> > >> > > >>>>> usually much
> > >> > > >>>>>>>>> more
> > >> > > >>>>>>>>>>>>>>>>>> expensive
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > seems
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > like a
> > >> > valid
> > >> > > >>>>> use-case where
> > >> > > >>>>>>>>> that
> > >> > > >>>>>>>>>>>>> an
> > >> > > >>>>>>>>>>>>>>>>>> admin
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > write
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > IOps
> > >> > vs
> > >> > > >>> Read
> > >> > > >>>>> IOps.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Since
> > >> > this
> > >> > > >>> only
> > >> > > >>>>> supports KVM
> > >> > > >>>>>>>>> at
> > >> > > >>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>> point I
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > great
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > value
> > >> > to at
> > >> > > >>>>> least have the
> > >> > > >>>>>>>>>>>>>>>>>> mechanism in
> > >> > > >>>>>>>>>>>>>>>>>>>>>> place
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> implementing
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > this
> > >> > later
> > >> > > >>>>> would be a lot of
> > >> > > >>>>>>>>>>> work.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > a
> > >> > > >>> hypervisor
> > >> > > >>>>> doesn't
> > >> > > >>>>>>>>> support
> > >> > > >>>>>>>>>>>>>>>>>> setting
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > write
> > >> > you
> > >> > > >>> can
> > >> > > >>>>> always sum
> > >> > > >>>>>> both
> > >> > > >>>>>>>>> up
> > >> > > >>>>>>>>>>>>>>>>>> and set
> > >> > > >>>>>>>>>>>>>>>>>>>>>> that
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Can
> > >> > you
> > >> > > >>>>> explain why you
> > >> > > >>>>>>>>>>>>> implemented
> > >> > > >>>>>>>>>>>>>>>>>> it
> > >> > > >>>>>>>>>>>>>>>>>>>> this
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wido
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > The
> > >> > > >>> feature
> > >> > > >>>>> includes:
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (1) set
> > >> > the
> > >> > > >>>>> maximum rate of
> > >> > > >>>>>>>>> VMs
> > >> > > >>>>>>>>>>>>> (in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> configuration)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (2)
> > >> > change
> > >> > > >>> the
> > >> > > >>>>> maximum rate
> > >> > > >>>>>> of
> > >> > > >>>>>>>>>>> VMs
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (3)
> > >> > limit
> > >> > > >>> the
> > >> > > >>>>> disk rate
> > >> > > >>>>>> (total
> > >> > > >>>>>>>>>>> bps
> > >> > > >>>>>>>>>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>> iops)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > JIRA
> > >> > ticket:
> > >> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> issues.apache.org/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> jira/browse/CLOUDSTACK-1192<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1192>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>> 1192<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > >> > 1192
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1192<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > https://issues.apache.org/jira/browse/CLOUDSTACK-
> > >> > 1192>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > FS (I
> > >> > will
> > >> > > >>>>> update later) :
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>
> > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>> VM+Disk+IO+Throttling<https://
> > >> <https://%0b/>> > >>>>>>>>>>>>> ****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cwiki.apache.org/confluence/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >> > > >>>>> rottling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Merge
> > >> > > >>> check
> > >> > > >>>>> list :-
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > Did
> > >> > you
> > >> > > >>>>> check the branch's
> > >> > > >>>>>>>>> RAT
> > >> > > >>>>>>>>>>>>>>>>>> execution
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Yes
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > Are
> > >> > > >>> there
> > >> > > >>>>> new dependencies
> > >> > > >>>>>>>>>>>>>>>>>> introduced?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > No
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > >> > What
> > >> > > >>>>> automated testing
> > >> > > >>>>>> (unit
> > >> > > >>>>>>>>>>> and
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > new
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > feature?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Unit
> > >> > tests
> > >> > > >>> are
> > >> > > >>>>> added.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > >> > What
> > >> > > >>>>> testing has been done
> > >> > > >>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>>> check for
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (1) set
> > >> > the
> > >> > > >>>>> bytes rate and
> > >> > > >>>>>>>>> IOPS
> > >> > > >>>>>>>>>>>>>>>>>> rate on
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (2)
> > >> > VM
> > >> > > >>>>> operations, including
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > deploy,
> > >> > > >>> stop,
> > >> > > >>>>> start, reboot,
> > >> > > >>>>>>>>>>>>>>>>>> destroy,
> > >> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (3)
> > >> > > >>> Volume
> > >> > > >>>>> operations,
> > >> > > >>>>>>>>> including
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Attach,
> > >> > > >>>>> Detach
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > To
> > >> > review
> > >> > > >>> the
> > >> > > >>>>> code, you can
> > >> > > >>>>>>>>> try
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > git diff
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Best
> > >> > > >>> regards,
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Wei
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > [1]
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>
> > >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>
> > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>> VM+Disk+IO+Throttling<https://
> > >> <https://%0b/>> > >>>>>>>>>>>>> ****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cwiki.apache.org/confluence/****
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>
> > >> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >> > > >>>>> rottling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > [2]
> > >> > > >>>>>>>>> refs/heads/disk_io_throttling
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > [3]
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> > 1301
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1301
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/****jira/browse/CLOUDSTACK-
> > >> > > >>> 1301<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1301>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>> 1301<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > >> > 1301
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 1301<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > https://issues.apache.org/jira/browse/CLOUDSTACK-
> > >> > 1301>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <ht**tps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > issues.apache.org/****jira/**browse/CLOUDSTACK-
> > >> > > >>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>>>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > **<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >> > > >>>>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > >> > 2071
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <**
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >> > > >>>>> 2071
> > >> > > >>>>>>>>>>>>>>>>>>>> <
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >> > > >>>>> 2071>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> <h**ttps://
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > https://issues.apache.org/jira/browse/CLOUDSTACK-
> > >> > 2071>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> (**CLOUDSTACK-1301
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > >> > VM
> > >> > > >>> Disk
> > >> > > >>>>> I/O
> > >> > > >>>>>> Throttling)
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > *Mike
> > >> > > >>>>> Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > *Senior
> > >> > > >>>>> CloudStack Developer,
> > >> > > >>>>>>>>>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>>>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >> > > >>> 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > Advancing
> > >> > > >>> the
> > >> > > >>>>> way the world
> > >> > > >>>>>>>>> uses
> > >> > > >>>>>>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >> > > >>> Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > > >>> CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Advancing
> > >> > the
> > >> > > >>> way
> > >> > > >>>>> the world uses
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > > >>> CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Advancing the
> > >> > > >>> way
> > >> > > >>>>> the world uses
> > >> > > >>>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>>>>> SolidFire
> > >> > > >>>>>>>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > >> > way
> > >> > > >>> the
> > >> > > >>>>> world uses the
> > >> > > >>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >> > CloudStack
> > >> > > >>>>> Developer,
> > >> > > >>>>>> SolidFire
> > >> > > >>>>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > >> > way
> > >> > > >>> the
> > >> > > >>>>> world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > >> > > >>>>> Developer, SolidFire
> > >> > > >>>>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > >> > the
> > >> > > >>>>> world uses the
> > >> > > >>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > >> > > >>> Developer,
> > >> > > >>>>> SolidFire
> > >> > > >>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > >> > the
> > >> > > >>> world
> > >> > > >>>>> uses the
> > >> > > >>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > >> > > >>> Developer,
> > >> > > >>>>> SolidFire
> > >> > > >>>>>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>>>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way the
> > >> > > >>> world
> > >> > > >>>>> uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >> > Developer,
> > >> > > >>>>> SolidFire
> > >> > > >>>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > > >>> mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > >> > world
> > >> > > >>> uses
> > >> > > >>>>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >> > Developer,
> > >> > > >>>>> SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > >> > world
> > >> > > >>> uses
> > >> > > >>>>> the cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > >> > > >>>>> SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > >> > uses
> > >> > > >>> the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > >> > > >>> SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > >> > uses
> > >> > > >>> the
> > >> > > >>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > >> > > >>> SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > >> > the
> > >> > > >>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >> > > >>> SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > >> > the
> > >> > > >>>>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >> > SolidFire
> > >> > > >>>>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > >> > the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >> > SolidFire
> > >> > > >>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > >> > > >>> cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > >> > > >>> Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > >> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > cloud<
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > >> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> > >> > > >>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > >> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>>>>>>>
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>>>>
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> --
> > >> > > >>>>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>>>> *(tm)*
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> --
> > >> > > >>>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>>> o: 303.746.7302
> > >> > > >>>>>>>> Advancing the way the world uses the cloud<
> > >> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>>> *(tm)*
> > >> > > >>>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> --
> > >> > > >>>>>>> *Mike Tutkowski*
> > >> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>>>> o: 303.746.7302
> > >> > > >>>>>>> Advancing the way the world uses the
> > >> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>>>> *(tm)*
> > >> > > >>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>>>
> > >> > > >>>>> --
> > >> > > >>>>> *Mike Tutkowski*
> > >> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>>>> e: mike.tutkowski@solidfire.com
> > >> > > >>>>> o: 303.746.7302
> > >> > > >>>>> Advancing the way the world uses the
> > >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>>>> *(tm)*
> > >> > > >>>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> --
> > >> > > >>> *Mike Tutkowski*
> > >> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > >>> e: mike.tutkowski@solidfire.com
> > >> > > >>> o: 303.746.7302
> > >> > > >>> Advancing the way the world uses the
> > >> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > >>> *(tm)*
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > *Mike Tutkowski*
> > >> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > > > e: mike.tutkowski@solidfire.com
> > >> > > > o: 303.746.7302
> > >> > > > Advancing the way the world uses the
> > >> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > > > *(tm)*
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > *Mike Tutkowski*
> > >> > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > e: mike.tutkowski@solidfire.com
> > >> > o: 303.746.7302
> > >> > Advancing the way the world uses the
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > *(tm)*****
> > >>
> > >>
> > >>
> > >> ****
> > >>
> > >> ** **
> > >>
> > >> --
> > >> *Mike Tutkowski*****
> > >>
> > >> *Senior CloudStack Developer, SolidFire Inc.*****
> > >>
> > >> e: mike.tutkowski@solidfire.com****
> > >>
> > >> o: 303.746.7302****
> > >>
> > >> Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> *(tm)*****
> > >>
> > >>
> > >>
> > >> ****
> > >>
> > >> ** **
> > >>
> > >> --
> > >> *Mike Tutkowski*****
> > >>
> > >> *Senior CloudStack Developer, SolidFire Inc.*****
> > >>
> > >> e: mike.tutkowski@solidfire.com****
> > >>
> > >> o: 303.746.7302****
> > >>
> > >> Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> *(tm)*****
> > >>
> > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *(tm)*
> > >
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
OK, thanks for that info, John.
On Mon, Jun 10, 2013 at 2:25 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Yes, a vendor may hand a non-managed plugin. For example, on the
> secondary storage side, S3 and Swift don't support the concept of being
> managed -- an operator is required to create a bucket and point CloudStack
> at it. Another example would be a generic iSCSI device (e.g. OpenFiler,
> FreeNAS) which CloudStack can not directly manage, but can consume a LUN
> that an operator has created for it.
>
> From an operational perspective, an administrator may elect to say, "Hey,
> I know you can manage that device for me, but it is has a wider use than
> CloudStack can manage. Let me handle it, and just provide the information
> CloudStack needs." In this scenario, the operator is accepting a greater
> management burden in exchange for control. I don't think many will elect
> to manage these devices themselves, but this ability will be extremely
> important for the edge cases that require it. Speaking as someone who has
> run cloud operations, I can testify to this need -- nothing aggravates you
> more than a system insisting on managing a device or service without the
> opportunity to opt-out
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 3:19 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > So, would a storage vendor write a non-CloudStack-managed plug-in and
> maybe
> > just implement the snapshot feature or something?
> >
> > In this case, they'd have to set the hypervisor up ahead of time with a
> > large portion of their storage, but they'd still get the vendor-snapshot
> > ability.
> >
> > Just trying to undertand this more.
> >
> > Thanks
> >
> >
> > On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Edison,
> >>
> >> TL;DR Always give operators/system administrators the ability to manage
> >> things themselves, if they like.
> >>
> >> As more manageable drivers emerge, I think it will be import to allow
> >> operators to opt-out of CloudStack device management. Therefore, a
> driver
> >> declares whether or not it can support device management (many, in fact
> >> most, won't). For those drivers that do support management, allow
> >> operators to opt-in/out of CloudStack management on a per device basis.
> To
> >> support this notion, we need a flag on the storage_pool table and
> >> associated classes to indicate whether or not the operator wants
> CloudStack
> >> to manage the device.
> >>
> >> Does this explanation make more sense?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
> >>
> >>> I don't understand why we need to add a "managed" in the UI and in
> >> "storage_pool" table?
> >>> AFAIK, what you guys trying to do is to enable
> >> attachVolume/DettachVolume if the volume is created on Solidfire, right?
> >>> If the volume is created on Solidfire already, by createAsync, then why
> >> we can't just send out all the information about the volume(it's
> volumeTO
> >> which will be returned by each storage driver) to hypervisor, then at
> the
> >> hypervisor side, check the volumeTO, if the volumeTO is managed, then
> doing
> >> whatever necessary operations(create SR, create VDI etc), then attach
> the
> >> VDI to volume.
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>> Sent: Monday, June 10, 2013 11:10 AM
> >>>> To: John Burwell
> >>>> Cc: Edison Su; dev@cloudstack.apache.org
> >>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>
> >>>> I believe we're on the same page, John.
> >>>>
> >>>> I would add a new parameter to the createStoragePool API command
> (let's
> >>>> call it "managed" and it can be true or false). If you pass in "true",
> >> but
> >>>> the driver does not support being managed by CloudStack, an exception
> is
> >>>> thrown by the driver. If you pass in "false", but the driver does not
> >>>> support being managed outside of CloudStack, an exception is thrown by
> >> the
> >>>> driver. It is possible one could write a driver that can be managed
> both
> >>>> inside or outside of CloudStack (the SolidFire driver requires being
> >>>> managed only by CloudStack).
> >>>>
> >>>> This managed value would end up in the storage_pool table in a new
> >> column
> >>>> ("managed").
> >>>>
> >>>> An example usage: After the storage framework calls on the driver's
> >>>> createAsync method, it will check to see if the driver is managed. If
> >> the
> >>>> driver is managed, the storage framework will invoke the proper (new)
> >>>> command of the hypervisor in use to set up, say, the SR for the SAN
> >> volume.
> >>>>
> >>>> There may be other cases when the isManaged() method is invoked on the
> >>>> driver and - if true is returned - the storage framework sends a
> >> particular
> >>>> command to the hypervisor in use.
> >>>>
> >>>> ********************
> >>>>
> >>>> As far as reworking the default storage plug-in (which sends commands
> to
> >>>> hypervisors) for 4.2, I would check with Edison on his schedule.
> >>>>
> >>>> Thanks!
> >>>>
> >>>>
> >>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> I want to make sure we are on the same page regarding isManaged. I
> see
> >>>>> the addition of the following operations:
> >>>>>
> >>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates whether
> >>>> or
> >>>>> not the driver support/implements the management functions
> >>>>> * DataStore#isManaged() : boolean -- indicates whether or not a
> device
> >> is
> >>>>> managed by CloudStack
> >>>>>
> >>>>> Only DataStores associated with DataStoreDrivers that support
> >>>> management
> >>>>> would be able to have the management flag enabled. This behavior
> >> allows
> >>>>> operators to declare their intention to allow CloudStack to manage a
> >>>>> device. This behavior would need to be exposed as flag on the HTTP
> >> API.
> >>>>> Additionally, this flag would only be settable when a device is
> >>>>> defined/created in the system (i.e. once a device is managed by
> >> CloudStack,
> >>>>> you can't unmanaged it without dissociating it from CloudStack). In
> >> the
> >>>>> future, we may consider relaxing that rule, but, in the short term,
> it
> >>>>> greatly decreases the alternative flows.
> >>>>>
> >>>>> In terms of removing the dependencies from Storage to Hypervisor, I
> >>>> would
> >>>>> really like to see this accomplished in 4.2. It will be much more
> >>>>> difficult to address this issue post 4.2 because we will have more
> >> plugins
> >>>>> to address. I also see this issue as a critical architectural issues
> >> that
> >>>>> for the management server. I recommend we identify the dependencies
> >>>> that
> >>>>> need to be inverted, and see if we can divide the work across a few
> >> more
> >>>>> resources.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> >>>> <mi...@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>> Sounds good, Edison :)
> >>>>>
> >>>>> I can implement John's isManaged() logic and resubmit my code for
> >> review.
> >>>>>
> >>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
> >> requires
> >>>>> some refactoring of the storage framework, then I could do that.
> >>>>>
> >>>>> Also, the default storage plug-in does have hypervisor logic in it in
> >> 4.2.
> >>>>> I recommend we leave this as is for 4.2 and it can be refactored in
> 4.3
> >>>>> with the above work I mentioned.
> >>>>>
> >>>>> Are you OK with that, guys?
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
> >> wrote:
> >>>>>
> >>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
> >>>>>> *To:* dev@cloudstack.apache.org
> >>>>>> *Cc:* Edison Su; John Burwell
> >>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> Another thing we might want to consider for 4.2 is that the way the
> >> code
> >>>>>> is currently implemented in master, I don't think it's even possible
> >> to
> >>>>>> write a plug-in that connects storage directly to a VM because the
> >>>>>> CloudStack storage framework will call the "attach" method of the
> >>>>>> hypervisor in use and this method will fail because it assumes there
> >> is
> >>>>>> (talking Xen here) an SR existent and in this situation there isn't
> >> one.*
> >>>>>> ***
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> My thinking is that we should ignore the
> direct-attach-of-volume-to-VM
> >>>>>> use case for 4.2. I think it will require too much of a rewrite at
> >> this
> >>>>>> point in development (fewer than three weeks left until code
> >>>> freeze).****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> If the plug-in implements an isManaged() method (as John suggested),
> >>>> the
> >>>>>> storage framework can invoke the necessary commands on the
> >>>> hypervisor in
> >>>>>> use.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> For example, the createAsync method is called. The storage plug-in
> >>>>>> creates a SAN volume and updates the CS DB. The storage framework
> >>>> then asks
> >>>>>> the driver if it's managed. If it is, the storage framework
> executes a
> >>>>>> command against the hypervisor in use to have it create, say, an SR
> >> (with
> >>>> a
> >>>>>> single VDI that takes up the entire SR). Now, when the storage
> >>>> framework
> >>>>>> executes the "attach" command against the hypervisor, this command
> >> will
> >>>>>> work, as well (because that logic assumes the prior existence of an
> >> SR and
> >>>>>> it will, in fact, be present).****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:****
> >>>>>>
> >>>>>> Hi Edison and John,****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> I wanted to point out something I thought was relevant to this
> >>>>>> conversation.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> My plug-in supports both XenServer and ESX(i).****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> These two hypervisors handle their respective hypervisor data
> >> structures
> >>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> In XenServer, the SR that resides on the SAN LUN is simply
> "forgotten"
> >>>>>> when you issue a remove-SR command to XenServer.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> When you later want to access the contents of that data, it is still
> >>>>>> safely stored on the LUN and can be accessed from XenServer (from
> the
> >>>> same
> >>>>>> or different cluster (what XenServer calls a resource pool)) by
> >> creating a
> >>>>>> new SR that is based on the same IQN and LUN.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> XenServer SRs exist at the cluster level.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
> >>>>>> destroyed when you issue a remove-datastore command to ESX(i). In
> >>>> other
> >>>>>> words, the contents of the SAN LUN are destroyed.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> ESX(i) datastores exist at the datacenter level.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> The XenServer behavior (not destroying the contents of the LUN when
> >>>> the
> >>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
> >> contents
> >>>>>> of the LUN when the datastore is removed) is not desirable.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> I can get around the ESX(i) behavior by removing the referenced IQN
> >>>> from
> >>>>>> each host in the cluster when the CS storage framework notifies my
> >> plug-
> >>>> in
> >>>>>> of a detach-volume event. When the CS volume is later attached again
> >>>>>> (either to the same or a different ESX(i) cluster), I can have each
> >> host in
> >>>>>> the relevant ESX(i) cluster reference the IQN and the datastore will
> >> be
> >>>>>> back again.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> This sequence of events is critical for my plug-in to support both
> >>>>>> environments:****
> >>>>>>
> >>>>>> *XenServer flow (events for Driver class):*****
> >>>>>>
> >>>>>> createAsync (called right before first attach) = creates SAN
> >> volume****
> >>>>>>
> >>>>>> preAttachVolume = creates SR and creates VDI or just introduces
> SR****
> >>>>>>
> >>>>>> (CS volume gets attached and later detached)****
> >>>>>>
> >>>>>> postDetachVolume = removes SR (this does not delete data on SAN
> >>>> volume)**
> >>>>>> **
> >>>>>>
> >>>>>> deleteAsync (can be called when volume is not attached) = deletes
> the
> >>>> SAN
> >>>>>> volume****
> >>>>>>
> >>>>>> *VMware flow (events for Driver class):*****
> >>>>>>
> >>>>>> createAsync (called right before first attach) = creates SAN volume,
> >>>>>> creates datastore, and creates VMDK file****
> >>>>>>
> >>>>>> preAttachVolume = adds iSCSI target to each host in cluster****
> >>>>>>
> >>>>>> (CS volume gets attached and later detached)****
> >>>>>>
> >>>>>> postDetachVolume = removes iSCSI target from each host in
> cluster****
> >>>>>>
> >>>>>> deleteAsync (can be called when volume is not attached) = deletes
> >>>>>> datastore and deletes the SAN volume****
> >>>>>>
> >>>>>> ************************
> >>>>>>
> >>>>>> So, the way I have it written up here, the driver is talking to the
> >>>>>> hypervisor. As John has pointed out, this may not be ideal.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> Whatever we decide on, it needs to be able to support this kind of
> >> flow.
> >>>>>> :)****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> We could go John's route, where the driver says if it's managed or
> >> not.**
> >>>>>> **
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> If it is, then the storage framework could send the necessary
> >> messages to
> >>>>>> the hypervisor in question at the right times (just migrate the
> >> hypervisor
> >>>>>> logic above from the driver to the right parts of the storage
> >> framework).
> >>>>>> ****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> This is fine, except that it does not resolve Edison's use case of
> >>>>>> desiring to have the storage LUN potentially connected directly to a
> >> VM.*
> >>>>>> ***
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> I'm not sure we should design around that use case, though. Is it
> >> common
> >>>>>> enough? Has anyone asked for it? Is it even desirable? How are usage
> >>>>>> statistics gathered for such volumes? It's nice to offer the
> >> flexibility,
> >>>>>> but is it a good idea in this case?****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> Thanks!****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> >> wrote:**
> >>>>>> **
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>> Sent: Friday, June 07, 2013 4:08 PM
> >>>>>>> To: dev@cloudstack.apache.org
> >>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>
> >>>>>>> Hi John,
> >>>>>>>
> >>>>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
> >>>>>> volume
> >>>>>>> to use it. Same story for VMware: ESX(i) must layer a datastore on
> >> top
> >>>>>> of
> >>>>>>> the iSCSI volume to use it. I look at it like they are layering a
> >>>>>> clustered
> >>>>>>> file system on the SAN volume so the hypervisors can share access
> to
> >>>> the
> >>>>>>> contents of the volume.
> >>>>>>>
> >>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called,
> the
> >>>>>> SR or
> >>>>>>> datastore has already been created (usually manually by an admin).
> >>>>>>>
> >>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
> >>>>>>> environment where each CloudStack volume that a user creates is
> >>>> mapped
> >>>>>>> to a
> >>>>>>> single SAN volume (via the SR).
> >>>>>>>
> >>>>>>> The question comes down to who should allocate the SR dynamically.
> >>>>>>>
> >>>>>>> We could have the storage framework ask the storage plug-in if it
> is
> >>>>>>> managed. If it is, then the storage framework could send a message
> to
> >>>>>> the
> >>>>>>> hypervisor in question to create (let's talk Xen here) the SR ahead
> >> of
> >>>>>>> time. Then, when the storage framework next sends the attach-volume
> >>>>>>> command
> >>>>>>> to the hypervisor, it should work without changes to that
> >> attach-volume
> >>>>>>> logic (because - from the point of view of the attach logic - the
> SR
> >> is
> >>>>>>> already existent, as expected).
> >>>>>>>
> >>>>>>> Now, as Edison has pointed out, this limits the power of the
> storage
> >>>>>>> plug-in. A storage plug-in in this model cannot directly attach
> >> storage
> >>>>>> to
> >>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We
> need
> >>>> to
> >>>>>>> make a call on that.
> >>>>>>
> >>>>>>
> >>>>>> Yes, I think we need to address this usage case, and won't limit the
> >>>>>> power of the storage plug-in.
> >>>>>> John, what's your idea to fit this usage case into your abstraction?
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> If we want to give more power to the storage vendor, we would have
> to
> >>>>>>> have
> >>>>>>> the storage framework call into the storage plug-in to send the
> >>>>>> appropriate
> >>>>>>> attach commands to the hypervisor.
> >>>>>>>
> >>>>>>> Let's discuss and come to a consensus (or at least agree on a path)
> >> as
> >>>>>> soon
> >>>>>>> as we can. :)
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> My understanding is the Xen expects an SR structure on all ISCSI
> >>>>>> devices
> >>>>>>>> -- at least that is how I read the code in your patch. Is my
> >>>>>> understanding
> >>>>>>>> correct? If so, the Xen plugin should be able to query the
> storage
> >>>>>> device
> >>>>>>>> to determine the presence of the SR structure and create it if it
> >>>>>> does not
> >>>>>>>> exist. Am I missing something in the implementation that makes
> that
> >>>>>> type
> >>>>>>>> of implementation impossible?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Yeah, if a storage vendor wanted to attach a volume directly to a
> >>>>>> VM in
> >>>>>>>> 4.2
> >>>>>>>>> today, it would probably fail because the attach-volume logic
> >>>>>> assumes
> >>>>>>> the
> >>>>>>>>> existence of the necessary hypervisor data structure (ex. SR on
> >>>>>> Xen).
> >>>>>>>>>
> >>>>>>>>> If we wanted to enable such an attach, we could do it the way
> >>>> Edison
> >>>>>>>>> suggests.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> As we only have three weeks until feature freeze, we should
> >>>> come
> >>>>>> to
> >>>>>>> a
> >>>>>>>>>>> consensus on this design point as soon as possible.
> >>>>>>>>>>>
> >>>>>>>>>>> Right now, if the storage framework asks my driver it is is
> >>>>>> managed, it
> >>>>>>>>>>> will say 'yes.' This means the framework will tell the driver
> to
> >>>>>>>> perform
> >>>>>>>>>>> its management activities. This then means the driver will call
> >>>>>> into
> >>>>>>>> the
> >>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to perform
> >>>> the
> >>>>>>>>>> activity
> >>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
> >>>>>>>>>>>
> >>>>>>>>>>> The driver doesn't know which hypervisor it's talking to, it
> just
> >>>>>>>> sends a
> >>>>>>>>>>> message to the host to perform the necessary pre-attach work.
> >>>>>>>>>>
> >>>>>>>>>> Could we just expose a method like
> >>>> "attachVolume/dettachVolume" on
> >>>>>>> the
> >>>>>>>>>> PrimaryDataStoreDriver driver?
> >>>>>>>>>> For most of cases, the implementation of each driver would just
> >>>>>> send
> >>>>>>>>>> attachvolumecommand/dettachvolumecmd to hypervisor(we can
> >>>> put
> >>>>>>> the
> >>>>>>>>>> implementation in a base class, so that can be shared by all of
> >>>>>> this
> >>>>>>>>>> drivers), and in each hypervisor resource code, which may just
> >> call
> >>>>>>>>>> hypervisor's api to attach the volume to VM, while for certain
> >>>>>> storage,
> >>>>>>>>>> like SolidFire, may need to create a SR first, and create volume
> >>>>>> on it,
> >>>>>>>>>> then call hypervisor's API to attach volume to VM.
> >>>>>>>>>> While for some other storage vendor, may want to bypass
> >>>> hypervisor
> >>>>>>>> during
> >>>>>>>>>> attaching volume, so inside driver's attachvolume
> implementation,
> >>>>>> the
> >>>>>>>>>> driver can do some magic, such as, directly talk to an agent
> >>>>>> inside VM
> >>>>>>>>>> instance, then create a disk inside VM.
> >>>>>>>>>>
> >>>>>>>>>> How do you guys think?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <
> Edison.su@citrix.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> How's about this:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The driver can implement an isManaged() method. The
> >>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
> >>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>> (which is responsible for calling into the hypervisor to
> attach
> >>>>>> the
> >>>>>>>>>> disk)
> >>>>>>>>>>>>> can call into the hypervisor to create the necessary
> hypervisor
> >>>>>> data
> >>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
> >>>>>>>>>>>>
> >>>>>>>>>>>> The problem here is that storage vendor may work differently
> >>>> with
> >>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> >>>>>> maybe
> >>>>>>>>>> other
> >>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the LUN
> >>>>>>> directly
> >>>>>>>>>> to
> >>>>>>>>>>>> VM instance, see the discuss(
> >>>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>>>>>>>>>>
> >>>>>>>
> >>>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>>>>>>>>>> %3E
> >>>>>>>>>>>> ).
> >>>>>>>>>>>> So I would let storage provider to implement attach disk to
> VM,
> >>>>>>>>>> instead of
> >>>>>>>>>>>> implemented by cloudstack itself.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If that's what you're going for, that works for me. By the
> way,
> >>>>>>>>>> Edison's
> >>>>>>>>>>>>> default storage plug-in (which handles the default storage
> >>>>>> behavior
> >>>>>>>>>> in
> >>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> >>>>>> talks to
> >>>>>>>>>>>>> hypervisors. You might want to contact him and inform him of
> >>>>>> your
> >>>>>>>>>>>> concerns
> >>>>>>>>>>>>> or that logic (as is) will make it to production.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Please let me know if what I wrote in above (for my solution)
> >>>>>> is OK
> >>>>>>>>>> with
> >>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please see my responses in-line below.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I
> build
> >>>>>> up
> >>>>>>>>>> a
> >>>>>>>>>>>> patch
> >>>>>>>>>>>>>>> file, but this is the gist of what I've done:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * During a volume-attach operation, after
> >>>> VolumeManagerImpl
> >>>>>>> tells
> >>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a volume, I
> >>>> have
> >>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> >>>> driver if
> >>>>>> it
> >>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> >>>>>>>>>> whatever
> >>>>>>>>>>>>>>> activity is required. In my case, this includes sending a
> >>>>>>>>>> message to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> host where the VM is running to have, say XenServer, add a
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the IQN of
> >>>> the
> >>>>>>>>>> SAN
> >>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space
> it
> >>>>>> can
> >>>>>>>>>> on
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> storage repository). After this, the normal attach-volume
> >>>>>>>>>> message is
> >>>>>>>>>>>> sent
> >>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There should be **no** code from a storage driver to a
> >>>>>> hypervisor.
> >>>>>>>>>> I
> >>>>>>>>>>>>>> apologize for the repetition, but we simply can not have
> >>>>>> hypervisor
> >>>>>>>>>>>>>> specific code in the storage layer. The circular
> dependencies
> >>>>>>>>>> between
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> two layers are not sustainable in the long term. Either the
> >>>>>>>>>>>> VirtualManager
> >>>>>>>>>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >>>>>>>>>>> coordinate
> >>>>>>>>>>>>>> volume creation and then populating the SR. Ideally, we can
> >>>>>>>>>>>> generalize the
> >>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
> >>>> hypervisor
> >>>>>>>>>> plugin
> >>>>>>>>>>>>>> would only implement callbacks to perform the attach action
> >>>> and
> >>>>>>>>>> create
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver should
> >>>>>> only be
> >>>>>>>>>>>>>> allocating space and providing information about contents
> >>>> (e.g.
> >>>>>>>>>> space
> >>>>>>>>>>>>>> available, space consumed, streams to a URI, file handle
> for a
> >>>>>> URI,
> >>>>>>>>>>>> etc)
> >>>>>>>>>>>>>> and capabilities.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * The reverse is performed for a detach-volume command.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in my
> >>>>>> driver.
> >>>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>> default driver simply does a "return false;". We could add
> a
> >>>>>> new
> >>>>>>>>>>>>>> parameter
> >>>>>>>>>>>>>>> to the createStoragePool API command, if we want, to
> >>>> remove
> >>>>>>> the
> >>>>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>>>> return values in the drivers (although my driver will
> >> probably
> >>>>>>>>>> just
> >>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>> this parameter and always return true since it wouldn't
> make
> >>>>>>>>>> sense
> >>>>>>>>>>>> for it
> >>>>>>>>>>>>>>> to ever return false). We'd need another column in the
> >>>>>>>>>> storage_pool
> >>>>>>>>>>>>> table
> >>>>>>>>>>>>>>> to store this value.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes, I think we should have a parameter added to the
> >>>>>>>>>>> createStoragePool
> >>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
> >>>>>> configured
> >>>>>>>>>> for
> >>>>>>>>>>>>>> management when their underlying drivers support it. To
> >>>>>> simplify
> >>>>>>>>>>>> things,
> >>>>>>>>>>>>>> this flag should only be mutable when the DataStore is
> >>>>>> created. It
> >>>>>>>>>>>> would be
> >>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to unmanaged
> >>>> after
> >>>>>>>>>>>> creation.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
> >>>> working
> >>>>>>>>>> GUI
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>> to review (in that document), if you'd like to check it
> out.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed together to
> >>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor throttled
> >>>> IOPs
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the potential for
> >>>>>>>>>>>> confusion,
> >>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these features
> will
> >>>>>>>>>> help
> >>>>>>>>>>>>>>>>> minimize any potential confusion.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will work
> >>>>>> itself
> >>>>>>>>>> if
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
> >> hypervisor
> >>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
> >>>> sent
> >>>>>>> to
> >>>>>>>>>>> you
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> other day to see how the GUI looks for provisioned
> >>>> storage
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-mail
> >> list
> >>>>>>>>>> and we
> >>>>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> >>>>>>> Offering
> >>>>>>>>>>>> dialog.
> >>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS, so they
> >>>>>>> should
> >>>>>>>>>>> be
> >>>>>>>>>>>>> able
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say, only
> >>>>>> use
> >>>>>>>>>> Max
> >>>>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for storage
> >>>>>> that
> >>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
> >>>>>> the
> >>>>>>>>>>>> admin can
> >>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> >>>> Burst
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature
> >>>> or
> >>>>>>>>>> mine,
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> decided by the admin).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two features
> >>>>>>>>>> expressing
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express the
> >>>> IOPS
> >>>>>> for
> >>>>>>>>>>>> QoS in
> >>>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
> >>>>>> read/write
> >>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
> >>>>>>>>>>> Storage/Hypervisor
> >>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the behavior
> >>>> of
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> >>>>>>>>>> following
> >>>>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>>>> outstanding:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are expressed as
> >>>> total
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
> >>>> to
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for throttled and
> >>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> that will permit providers to discriminate these
> >>>>>> guaranteed
> >>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
> >>>>>> provisioned
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
> >>>>>> hypervisor
> >>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> >>>>>> solve
> >>>>>>>>>>>> items 1
> >>>>>>>>>>>>>>>>> and 2.
> >>>>>>>>>>>>>>>>>>> Since the two facilities would not be permitted to
> >>>> operate
> >>>>>>>>>>>> together,
> >>>>>>>>>>>>>>>>> they
> >>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I think
> item
> >>>> 3
> >>>>>>>>>> can be
> >>>>>>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage records.
> As
> >>>>>> for
> >>>>>>>>>>>> Item 4,
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>>>> not familiar with how these two enhancements are
> >>>>>>> depicted in
> >>>>>>>>>>> the
> >>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
> >>>> enhancements
> >>>>>>> for
> >>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >>>>>>> <us...@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> >>>>>>> 2780/2729,
> >>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >>>>>>>>>>> CLOUDSTACK-
> >>>>>>>>>>>>> 2875 ) this
> >>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
> >>>> and
> >>>>>>>>>> ask
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> second
> >>>>>>>>>>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> >>>> <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going with this,
> >>>>>> John,
> >>>>>>>>>> and
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> >>>>>>> whether or
> >>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> >>>> but an
> >>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >>>>>>>>>> resources
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> >>>>>>> method
> >>>>>>>>>>>>> added to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns
> true,
> >>>>>>>>>> then the
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or whatever
> >>>> name)
> >>>>>>>>>>> method
> >>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
> >>>> well)
> >>>>>>> and
> >>>>>>>>>>> this
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to create,
> >>>> say
> >>>>>>> on
> >>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >>>>>>>>>>>>> DataStoreDriver.
> >>>>>>>>>>>>>>>>> Since
> >>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> >>>> storage,
> >>>>>>> I
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> >>>>>>> straw
> >>>>>>>>>>> man
> >>>>>>>>>>>>> term
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> >>>> mutable
> >>>>>>>>>>> property
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the underlying
> >>>>>>>>>>> DataStoreDriver
> >>>>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
> >>>>>>> manageability
> >>>>>>>>>> of
> >>>>>>>>>>>>>> devices.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
> >>>> Xen
> >>>>>>>>>> plugin
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> responsible for composing DataStore/Volume
> >>>> methods
> >>>>>>> to
> >>>>>>>>>>>>> create any
> >>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR. There
> >>>>>> should
> >>>>>>>>>> be no
> >>>>>>>>>>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> >>>>>>>>>> earlier,
> >>>>>>>>>>>> such
> >>>>>>>>>>>>>>>>>>>>> circular
> >>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
> >>>> unmaintainable
> >>>>>>> mess.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page with
> >>>> you.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
> >>>> layer
> >>>>>>> that
> >>>>>>>>>>>>>> supports n
> >>>>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors of m
> >>>>>>>>>> hypervisors.
> >>>>>>>>>>>>>> Such
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
> >>>> untestable
> >>>>>>>>>>> beast.
> >>>>>>>>>>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven to
> >>>> evolve
> >>>>>>> the
> >>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> >>>>>>> whether
> >>>>>>>>>>> or
> >>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> >>>> but
> >>>>>>> an
> >>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >>>>>>>>>> resources
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
> >>>>>> needs
> >>>>>>>>>> to
> >>>>>>>>>>>>>> indicate
> >>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
> >>>> Finally, I
> >>>>>>>>>> think
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to elect
> to
> >>>>>> have
> >>>>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> >>>> capable
> >>>>>>>>>>> managing
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
> >>>>>>> unmanaged).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume should be
> >>>> done
> >>>>>>> in
> >>>>>>>>>>> the
> >>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> >>>>>>> Storage
> >>>>>>>>>>>>> layer to
> >>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> >>>> create a
> >>>>>>>>>> file,
> >>>>>>>>>>>> etc).
> >>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >>>>>>>>>>>> hypervisor, as
> >>>>>>>>>>>>>>>>> such,
> >>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation. From
> >>>> my
> >>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>> nothing
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
> >>>> content.
> >>>>>>> From
> >>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides the
> >>>> means
> >>>>>>> to
> >>>>>>>>>>>> query
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the content by
> >>>> higher-
> >>>>>>> level
> >>>>>>>>>>>>> layers
> >>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> >>>>>>> composition
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on the
> >>>>>>> Volume
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> protocol
> >>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> >>>>>>> implemented
> >>>>>>>>>> in
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>> patch,
> >>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >>>>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
> >>>> the
> >>>>>>>>>>>>> hypervisor.
> >>>>>>>>>>>>>> Before
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if we're
> >>>>>>> dealing
> >>>>>>>>>>> with
> >>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>> (or
> >>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> >>>>>> send a
> >>>>>>>>>>>> "Create
> >>>>>>>>>>>>>>>>> SR"
> >>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> >>>>>>> would
> >>>>>>>>>>> then
> >>>>>>>>>>>>>>>>> proceed
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same and
> >>>> we just
> >>>>>>>>>> add
> >>>>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called for this
> >>>>>>>>>> particular
> >>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer today that
> >>>>>>> outlines
> >>>>>>>>>>> how
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will probably
> >>>> be
> >>>>>> of
> >>>>>>>>>> use
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> you, as
> >>>>>>>>>>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you (you
> >>>> should
> >>>>>>> have
> >>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> >>>>>>> thoughts,
> >>>>>>>>>>> but
> >>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> >>>>>> solidfire,
> >>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
> >>>> in a
> >>>>>>>>>>>>> coherent
> >>>>>>>>>>>>>>>>> form
> >>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> >>>> introduces
> >>>>>>> the
> >>>>>>>>>>>> notion
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> >>>> introduces
> >>>>>>>>>> the
> >>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages the
> >>>> plug-in
> >>>>>>> was
> >>>>>>>>>>>>>>>>> incomplete,
> >>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> >>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> >>>> spending on
> >>>>>>>>>> this.
> >>>>>>>>>>>> :)
> >>>>>>>>>>>>> Do
> >>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
> >>>> accomplish
> >>>>>>>>>>> what
> >>>>>>>>>>>>> you're
> >>>>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
> >>>> <
> >>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> >>>> introduces
> >>>>>>> the
> >>>>>>>>>>>> notion
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> >>>> management
> >>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>> (i.e.
> >>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
> >>>> it
> >>>>>>>>>>> managed
> >>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
> >>>> with
> >>>>>>> a
> >>>>>>>>>>>>> semantic
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> >>>> managed.
> >>>>>>> I
> >>>>>>>>>>> also
> >>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
> >>>> from
> >>>>>>> the
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> >>>>>>> questions
> >>>>>>>>>>> for
> >>>>>>>>>>>>> me
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> >>>> LUNs
> >>>>>>> and
> >>>>>>>>>>> SAN
> >>>>>>>>>>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
> >>>> LUN
> >>>>>>> (LUN
> >>>>>>>>>>> 0).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> >>>> Volumes
> >>>>>>> and
> >>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> >>>> management
> >>>>>>>>>>> server or
> >>>>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
> >>>> SAN
> >>>>>>>>>>> volume
> >>>>>>>>>>>>> (same
> >>>>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server (via
> >>>> the
> >>>>>>> plug-in)
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> >>>> hypervisor.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
> >>>> in,
> >>>>>>> but
> >>>>>>>>>>>> still
> >>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
> >>>> already
> >>>>>>>>>>> today
> >>>>>>>>>>>>>> (prior
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
> >>>> and -
> >>>>>>> in
> >>>>>>>>>>> this
> >>>>>>>>>>>>> case -
> >>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> >>>> While
> >>>>>>> you
> >>>>>>>>>>> can do
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
> >>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> >>>> management
> >>>>>>> server
> >>>>>>>>>>> or
> >>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> >>>> SAN
> >>>>>>>>>>> volumes
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
> >>>> in).
> >>>>>>> There
> >>>>>>>>>>> is
> >>>>>>>>>>>>> no
> >>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
> >>>> SAN
> >>>>>>>>>>> volume
> >>>>>>>>>>>>> and a
> >>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
> >>>> that
> >>>>>>>>>>>>> supports
> >>>>>>>>>>>>>> true
> >>>>>>>>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
> >>>> are
> >>>>>>>>>> related
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> >>>>>>> multiple
> >>>>>>>>>>> VMs
> >>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> >>>> storage."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
> >>>> map a
> >>>>>>>>>>> single
> >>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
> >>>> have
> >>>>>>>>>>> multiple
> >>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> >>>>>>> guarantee
> >>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> >>>>>>> volume
> >>>>>>>>>>> house
> >>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
> >>>> today
> >>>>>>>>>>> without
> >>>>>>>>>>>>> any of
> >>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> >>>>>>> volume
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> >>>> created."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> >>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> >>>>>>> concerns me
> >>>>>>>>>>> --
> >>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
> >>>> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
> >>>> is
> >>>>>>> today
> >>>>>>>>>>>> only
> >>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
> >>>> have
> >>>>>>> not
> >>>>>>>>>>>>> PreSetup an
> >>>>>>>>>>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >>>> volume
> >>>>>>> from
> >>>>>>>>>>>>> VMware, it
> >>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >>>> volume
> >>>>>>> from
> >>>>>>>>>>> KVM,
> >>>>>>>>>>>>> it uses
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> >>>> thinking
> >>>>>>>>>>> here
> >>>>>>>>>>>>> was to
> >>>>>>>>>>>>>>>>> make a
> >>>>>>>>>>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
> >>>> ability
> >>>>>>>>>>> Edison
> >>>>>>>>>>>>> added
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> >>>> more
> >>>>>>>>>>> specificy,
> >>>>>>>>>>>>> though:
> >>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> >>>> Mike
> >>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> >>>> shouldn't
> >>>>>>> know
> >>>>>>>>>>> or
> >>>>>>>>>>>>> care
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
> >>>> answer
> >>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
> >>>> the
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
> >>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> >>>>>>> constraint
> >>>>>>>>>>>>> and just
> >>>>>>>>>>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
> >>>> way
> >>>>>>> for
> >>>>>>>>>>> the
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> >>>>>>> perform
> >>>>>>>>>>> the
> >>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> >>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
> >>>> be
> >>>>>>> able to
> >>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>> (e.g. the
> >>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> >>>> operator has
> >>>>>>>>>>> chosen
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> >>>> reasoning
> >>>>>>>>>>> make
> >>>>>>>>>>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> >>>> creating a
> >>>>>>> SAN
> >>>>>>>>>>>>> volume via
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> >>>> hypervisor)
> >>>>>>> would
> >>>>>>>>>>>>> work unless
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
> >>>> say,
> >>>>>>> an
> >>>>>>>>>>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
> >>>> of
> >>>>>>>>>>>>> CloudStack, an
> >>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
> >>>> and
> >>>>>>>>>>> exposes it
> >>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
> >>>> over
> >>>>>>> to
> >>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
> >>>> LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
> >>>> to my
> >>>>>>>>>>>>> changes, we
> >>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
> >>>> the
> >>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> >>>> maybe
> >>>>>>> have
> >>>>>>>>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> >>>>>>> support
> >>>>>>>>>>> from
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> >>>> specifically
> >>>>>>> to
> >>>>>>>>>>> create
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> SR
> >>>>>>>>>>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> >>>> management
> >>>>>>> plugin
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> >>>> determine
> >>>>>>>>>>> available
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >>>>>>>>>>> interrogation of
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> >>>> device
> >>>>>>> itself
> >>>>>>>>>>>>> shouldn't
> >>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> >>>> simply be
> >>>>>>> able
> >>>>>>>>>>> to
> >>>>>>>>>>>>>> answer
> >>>>>>>>>>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
> >>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> >>>> SolidFire
> >>>>>>>>>>> driver
> >>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> >>>>>>> done, a
> >>>>>>>>>>> new
> >>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> >>>> Offering
> >>>>>>>>>>> that
> >>>>>>>>>>>>>> references
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> >>>> Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> >>>>>>> CloudStack
> >>>>>>>>>>>>> volume. This
> >>>>>>>>>>>>>>>>> leads
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> >>>>>>> CloudStack
> >>>>>>>>>>>>> volume to a
> >>>>>>>>>>>>>>>>> VM
> >>>>>>>>>>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> >>>> framework
> >>>>>>> calling
> >>>>>>>>>>> the
> >>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
> >>>> case, a
> >>>>>>>>>>> SAN).
> >>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> >>>> applicable
> >>>>>>> data
> >>>>>>>>>>>>> (like the
> >>>>>>>>>>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> >>>> invoked if
> >>>>>>> the
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> >>>> attach,
> >>>>>>> the
> >>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
> >>>> and
> >>>>>>> the
> >>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> >>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
> >>>> is run
> >>>>>>> and
> >>>>>>>>>>>>> detects
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
> >>>> the
> >>>>>>> form
> >>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
> >>>> PM,
> >>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> >>>> sync on
> >>>>>>>>>>>>> terminology,
> >>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> >>>> physical
> >>>>>>>>>>>>> allocation on
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
> >>>> try to
> >>>>>>>>>>> mean
> >>>>>>>>>>>>> 'SAN
> >>>>>>>>>>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
> >>>> end
> >>>>>>> user
> >>>>>>>>>>> can
> >>>>>>>>>>>>> make in
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
> >>>> PM,
> >>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> >>>> very
> >>>>>>> well
> >>>>>>>>>>> make
> >>>>>>>>>>>>> sense, but
> >>>>>>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
> >>>> Edison
> >>>>>>> in on
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> >>>>>>>>>>> approach I
> >>>>>>>>>>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
> >>>> PM,
> >>>>>>>>>>> John
> >>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> >>>> combining
> >>>>>>> two
> >>>>>>>>>>>>> distinct
> >>>>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> >>>>>>> protocols.
> >>>>>>>>>>> In
> >>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> >>>>>>> system to
> >>>>>>>>>>>>>> create/delete
> >>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> >>>> requires
> >>>>>>> the
> >>>>>>>>>>>>> volume to
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> >>>>>>> context.
> >>>>>>>>>>> To
> >>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
> >>>> these
> >>>>>>>>>>>>> definitions,
> >>>>>>>>>>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> >>>>>>> correct?
> >>>>>>>>>>>>> Minimally,
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> >>>> storage
> >>>>>>> device
> >>>>>>>>>>> to
> >>>>>>>>>>>>> move bits
> >>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >>>>>>>>>>> Optionally, a
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> >>>>>>> Therefore,
> >>>>>>>>>>>>> we can have
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
> >>>> SR,
> >>>>>>> and
> >>>>>>>>>>> we
> >>>>>>>>>>>>> can have a
> >>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> >>>> CloudStack is
> >>>>>>>>>>> capable
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> >>>>>>> CloudStack
> >>>>>>>>>>> may
> >>>>>>>>>>>>> be able
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> >>>> leave it
> >>>>>>>>>>>>> unmanaged by
> >>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
> >>>> and
> >>>>>>> the
> >>>>>>>>>>>>> operator has
> >>>>>>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> >>>> CloudStack).
> >>>>>>> Does
> >>>>>>>>>>> my
> >>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> >>>> above
> >>>>>>> are
> >>>>>>>>>>>>> reasonable, it
> >>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> >>>>>>> concerns
> >>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> >>>>>>> attached
> >>>>>>>>>>> driver
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
> >>>> and
> >>>>>>>>>>> establish
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> >>>> separate
> >>>>>>> that
> >>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> associated
> >>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> >>>>>>> hypervisor
> >>>>>>>>>>>>> drivers can
> >>>>>>>>>>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> >>>>>>> encounter
> >>>>>>>>>>> a
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> >>>>>>> exposed
> >>>>>>>>>>> by
> >>>>>>>>>>>>> the driver to
> >>>>>>>>>>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
> >>>> make
> >>>>>>>>>>> sense,
> >>>>>>>>>>>>> then we can
> >>>>>>>>>>>>>>>>> sit
> >>>>>>>>>>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> >>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> >>>>>>> between
> >>>>>>>>>>> the
> >>>>>>>>>>>>> current
> >>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> >>>> has to
> >>>>>>> go in
> >>>>>>>>>>> and
> >>>>>>>>>>>>> create
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> >>>> point in
> >>>>>>> time,
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> >>>>>>> allocated).
> >>>>>>>>>>>>> Future
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
> >>>> was
> >>>>>>> created
> >>>>>>>>>>>>> behind the
> >>>>>>>>>>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
> >>>> has
> >>>>>>> to
> >>>>>>>>>>> go in
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
> >>>> and
> >>>>>>> delete
> >>>>>>>>>>>>> volumes on
> >>>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> >>>> enabled via
> >>>>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> >>>> volume
> >>>>>>> that
> >>>>>>>>>>> was
> >>>>>>>>>>>>> created,
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> >>>>>>> volume.
> >>>>>>>>>>> After
> >>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>> done,
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> >>>> question is
> >>>>>>> called.
> >>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> >>>>>>> volume
> >>>>>>>>>>> was
> >>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> >>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>>> 3:21
> >>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
> >>>> so I
> >>>>>>> should
> >>>>>>>>>>>>> probably
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
> >>>> that is
> >>>>>>>>>>> pre-
> >>>>>>>>>>>>> allocated
> >>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
> >>>> a
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> >>>> XenServer
> >>>>>>> SR),
> >>>>>>>>>>> then
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> >>>> CloudStack.
> >>>>>>> Disks
> >>>>>>>>>>> (VDI)
> >>>>>>>>>>>>> are later
> >>>>>>>>>>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> >>>> needed. In
> >>>>>>>>>>> these
> >>>>>>>>>>>>> cases, the
> >>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >>>> structure is
> >>>>>>>>>>> already
> >>>>>>>>>>>>> in place
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> >>>>>>> structure
> >>>>>>>>>>> to
> >>>>>>>>>>>>> the VM in
> >>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> >>>> Storage
> >>>>>>> that is
> >>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
> >>>> this
> >>>>>>> could
> >>>>>>>>>>> be a
> >>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>> (not
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> >>>> hypervisor
> >>>>>>> data
> >>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>> must be
> >>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> >>>>>>> because
> >>>>>>>>>>> these
> >>>>>>>>>>>>> types of
> >>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
> >>>> data
> >>>>>>>>>>>>> structure by an
> >>>>>>>>>>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> >>>>>>> XenServer
> >>>>>>>>>>> for
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
> >>>> to the
> >>>>>>> VM
> >>>>>>>>>>> in
> >>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>>> 3:13
> >>>>>>> PM,
> >>>>>>>>>>>>> John Burwell
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >>>>>>> implementation of
> >>>>>>>>>>> the
> >>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> >>>> which
> >>>>>>> why I
> >>>>>>>>>>> ask
> >>>>>>>>>>>>> about the
> >>>>>>>>>>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> >>>> what is
> >>>>>>> the
> >>>>>>>>>>>>> definition
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
> >>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
> >>>> iSCSI
> >>>>>>> type
> >>>>>>>>>>> is
> >>>>>>>>>>>>>> uniquely
> >>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> >>>> Primary
> >>>>>>>>>>> Storage
> >>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>> directly
> >>>>>>>>>>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> >>>> skip the
> >>>>>>>>>>> step of
> >>>>>>>>>>>>> going
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
> >>>> based
> >>>>>>> on
> >>>>>>>>>>> that
> >>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
> >>>> only
> >>>>>>>>>>>>> supported for
> >>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> >>>> must
> >>>>>>> first go
> >>>>>>>>>>> to
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
> >>>> what
> >>>>>>> RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> >>>> at
> >>>>>>> 2:13
> >>>>>>>>>>> PM,
> >>>>>>>>>>>>> John
> >>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
> >>>> the
> >>>>>>> code,
> >>>>>>>>>>> what
> >>>>>>>>>>>>> is the
> >>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> >>>> Why
> >>>>>>> isn't
> >>>>>>>>>>> RBD
> >>>>>>>>>>>>> considered
> >>>>>>>>>>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> >>>> 3:46
> >>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> >>>>>>> storage is
> >>>>>>>>>>>>> defined in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> >>>>>>>>>>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
> >>>> //
> >>>>>>> local
> >>>>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> NetworkFilesystem(true),
> >>>>>>>>>>> //
> >>>>>>>>>>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> >>>>>>> shared
> >>>>>>>>>>> LUN,
> >>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
> >>>> e.g.,
> >>>>>>> ZFS
> >>>>>>>>>>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
> >>>> iso
> >>>>>>> image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> >>>>>>> XenServer
> >>>>>>>>>>> local
> >>>>>>>>>>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> >>>>>>> VMware
> >>>>>>>>>>> VMFS
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
> >>>> for
> >>>>>>>>>>>>> XenServer, Storage
> >>>>>>>>>>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> >>>>>>> XenServer
> >>>>>>>>>>> local
> >>>>>>>>>>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> >>>>>>> dynamic,
> >>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> StoragePoolType(boolean
> >>>>>>>>>>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> >>>> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> >>>>>>> isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>> 2013 at
> >>>>>>> 1:41
> >>>>>>>>>>> PM,
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> >>>> let's say
> >>>>>>>>>>> another
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> >>>> leverage its
> >>>>>>>>>>> Quality
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>> Service
> >>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>> storage, as
> >>>>>>> well.
> >>>>>>>>>>>>> They would
> >>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> >>>> made
> >>>>>>> the
> >>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
> >>>> plug-
> >>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>> 2013
> >>>>>>> at
> >>>>>>>>>>> 1:39
> >>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> >>>> imply
> >>>>>>> the
> >>>>>>>>>>>>> XenServer code
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> >>>>>>> attach
> >>>>>>>>>>> logic is
> >>>>>>>>>>>>> now aware
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
> >>>> an
> >>>>>>>>>>>>> implementation of
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
> >>>> to 4.2
> >>>>>>>>>>> with
> >>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
> >>>> a
> >>>>>>> new
> >>>>>>>>>>>>> framework that
> >>>>>>>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> >>>>>>> dynamically.
> >>>>>>>>>>>>> However, when I
> >>>>>>>>>>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> >>>> realized
> >>>>>>> that
> >>>>>>>>>>> it
> >>>>>>>>>>>>> was not
> >>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> >>>> CloudStack
> >>>>>>> could
> >>>>>>>>>>> do
> >>>>>>>>>>>>> with these
> >>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> >>>>>>> recognize
> >>>>>>>>>>> this
> >>>>>>>>>>>>> new type of
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >>>>>>> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>> 2013
> >>>>>>> at
> >>>>>>>>>>> 1:28
> >>>>>>>>>>>>> PM, John
> >>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
> >>>> odd
> >>>>>>> to me
> >>>>>>>>>>>>> that any
> >>>>>>>>>>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
> >>>> care
> >>>>>>>>>>> about
> >>>>>>>>>>>>> details. I
> >>>>>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
> >>>> of
> >>>>>>>>>>> operations
> >>>>>>>>>>>>> that can be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> implementations to
> >>>>>>>>>>>>> allocate
> >>>>>>>>>>>>>> space/create
> >>>>>>>>>>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
> >>>> this
> >>>>>>>>>>>>> dependency, we are
> >>>>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
> >>>> will
> >>>>>>> make
> >>>>>>>>>>> the
> >>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
> >>>> I
> >>>>>>>>>>>>> understanding that the
> >>>>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >>>>>>>>>>>>> CitrixResourceBase
> >>>>>>>>>>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>>>>> 2013 at
> >>>>>>>>>>> 3:12
> >>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
> >>>> this
> >>>>>>> in a
> >>>>>>>>>>> bit
> >>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
> >>>> and
> >>>>>>> aside
> >>>>>>>>>>> from
> >>>>>>>>>>>>> one setup
> >>>>>>>>>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
> >>>> a
> >>>>>>>>>>> volume on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
> >>>> data
> >>>>>>>>>>> structure
> >>>>>>>>>>>>> to make
> >>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> >>>>>>>>>>> XenServer
> >>>>>>>>>>>>> or a
> >>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> >>>>>>> storage
> >>>>>>>>>>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
> >>>> new
> >>>>>>>>>>> storage
> >>>>>>>>>>>>> framework,
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
> >>>> easily
> >>>>>>>>>>> create
> >>>>>>>>>>>>> a 1:1
> >>>>>>>>>>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
> >>>> the
> >>>>>>> VM's
> >>>>>>>>>>> data
> >>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>> (necessary
> >>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
> >>>> write
> >>>>>>> a
> >>>>>>>>>>> plug-
> >>>>>>>>>>>>> in that is
> >>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> >>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
> >>>> that
> >>>>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >>>>>>>>>>> hypervisor-
> >>>>>>>>>>>>> specific data
> >>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
> >>>> the
> >>>>>>> case,
> >>>>>>>>>>> I've
> >>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>> enhancing
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
> >>>> and
> >>>>>>>>>>>>> submitted. I've got
> >>>>>>>>>>>>>>>>> ESX(i)
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
> >>>> the
> >>>>>>> 4.2
> >>>>>>>>>>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> >>>> help a
> >>>>>>> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> >>>> 3,
> >>>>>>> 2013 at
> >>>>>>>>>>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> >>>>>>> plug-in -
> >>>>>>>>>>> by
> >>>>>>>>>>>>> itself - is
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
> >>>> with
> >>>>>>> the
> >>>>>>>>>>>>> volume-attach
> >>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>> calls
> >>>>>>> into
> >>>>>>>>>>> the
> >>>>>>>>>>>>> plug-in to
> >>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
> >>>> time
> >>>>>>>>>>> comes to
> >>>>>>>>>>>>> attach the
> >>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
> >>>> smart
> >>>>>>>>>>> enough
> >>>>>>>>>>>>> to recognize
> >>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> >>>>>>> volume
> >>>>>>>>>>> has
> >>>>>>>>>>>>> just been
> >>>>>>>>>>>>>>>>> created)
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> >>>> (for
> >>>>>>>>>>>>> XenServer) or a
> >>>>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
> >>>> was
> >>>>>>> just
> >>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> >>>>>>> spending
> >>>>>>>>>>> most
> >>>>>>>>>>>>> of my time
> >>>>>>>>>>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> >>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> >>>> clear it
> >>>>>>> up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >>>> Jun 3,
> >>>>>>> 2013
> >>>>>>>>>>> at
> >>>>>>>>>>>>> 12:48 PM,
> >>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> >>>> explain
> >>>>>>>>>>> why
> >>>>>>>>>>>>> the the storage
> >>>>>>>>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> >>>> 2013,
> >>>>>>> at
> >>>>>>>>>>> 1:21
> >>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> >>>>>>> ultimately I
> >>>>>>>>>>>>> would like to
> >>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
> >>>> I
> >>>>>>> think
> >>>>>>>>>>> I'm
> >>>>>>>>>>>>> just out of
> >>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> >>>> now this
> >>>>>>>>>>> plug-in
> >>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> >>>> to 4.2
> >>>>>>>>>>> feature
> >>>>>>>>>>>>> freeze, I
> >>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
> >>>> to
> >>>>>>> be
> >>>>>>>>>>> clear,
> >>>>>>>>>>>>> this is
> >>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>> Compute
> >>>>>>>>>>>>> Offerings post
> >>>>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >>>> Jun 3,
> >>>>>>>>>>> 2013
> >>>>>>>>>>>>> at 11:14 AM,
> >>>>>>>>>>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
> >>>> any
> >>>>>>> plan
> >>>>>>>>>>> on
> >>>>>>>>>>>>> supporting
> >>>>>>>>>>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> >>>> Original
> >>>>>>>>>>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
> >>>> "Mike
> >>>>>>>>>>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >>>>>>>>>>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> >>>>>>> Monday,
> >>>>>>>>>>> June
> >>>>>>>>>>>>> 3, 2013
> >>>>>>>>>>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
> >>>> Re:
> >>>>>>>>>>> [MERGE]
> >>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
> >>>> on
> >>>>>>>>>>> merging
> >>>>>>>>>>>>> Wei's feature
> >>>>>>>>>>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
> >>>> feature
> >>>>>>> is
> >>>>>>>>>>> for
> >>>>>>>>>>>>> KVM only,
> >>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
> >>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >>>> Jun
> >>>>>>> 3,
> >>>>>>>>>>> 2013
> >>>>>>>>>>>>> at 8:55 AM,
> >>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>> the
> >>>>>>> billing,
> >>>>>>>>>>> as
> >>>>>>>>>>>>> no one works
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>> billing
> >>>>>>> by
> >>>>>>>>>>>>> themselves. They
> >>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> disk_offering of
> >>>>>>>>>>> a
> >>>>>>>>>>>>> VMs and
> >>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
> >>>> user
> >>>>>>> the
> >>>>>>>>>>>>> exact
> >>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>> limitation
> >>>>>>>>>>> for
> >>>>>>>>>>>>> nics as
> >>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>> part of
> >>>>>>> I/O
> >>>>>>>>>>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
> >>>> my
> >>>>>>>>>>>>> misunstand the second
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
> >>>> with
> >>>>>>>>>>> what
> >>>>>>>>>>>>> you said about
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> 2013/6/3
> >>>>>>> John
> >>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>> Jun 3,
> >>>>>>> 2013,
> >>>>>>>>>>> at
> >>>>>>>>>>>>> 2:13 AM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> >>>> John,
> >>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>> hope
> >>>>>>> Mike's
> >>>>>>>>>>>>> aswer helps you.
> >>>>>>>>>>>>>>>>> I am
> >>>>>>>>>>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> >>>>>>> think
> >>>>>>>>>>> billing
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>> depend
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> limitation.
> >>>>>>>>>>> Please
> >>>>>>>>>>>>> review
> >>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>> the
> >>>>>>> IO
> >>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> virtual
> >>>>>>>>>>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Going by
> >>>>>>> the
> >>>>>>>>>>> AWS
> >>>>>>>>>>>>> model,
> >>>>>>>>>>>>>> customers
> >>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> provisioned
> >>>>>>>>>>> IOPS,
> >>>>>>>>>>>>> as well as,
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://aws.amazon.com/ebs/).
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> option to
> >>>>>>>>>>> employ
> >>>>>>>>>>>>> similar cost
> >>>>>>>>>>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> billing
> >>>>>>> model
> >>>>>>>>>>> in
> >>>>>>>>>>>>> the current
> >>>>>>>>>>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> >>>> Do
> >>>>>>> you
> >>>>>>>>>>> mean
> >>>>>>>>>>>>> IOPS runtime
> >>>>>>>>>>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> limitation for
> >>>>>>>>>>> a
> >>>>>>>>>>>>> running
> >>>>>>>>>>>>>> virtual
> >>>>>>>>>>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> CloudStack
> >>>>>>>>>>> does
> >>>>>>>>>>>>> not support
> >>>>>>>>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> (computer
> >>>>>>>>>>>>> offering or disk
> >>>>>>>>>>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>> meant at
> >>>>>>> the
> >>>>>>>>>>>>> Java interface
> >>>>>>>>>>>>>>>>> level.
> >>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>> more
> >>>>>>>>>>>>> generalize allocation
> >>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> describe
> >>>>>>> the
> >>>>>>>>>>>>> service guarantees
> >>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> >>>> It is a
> >>>>>>>>>>> good
> >>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>> Maybe it
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> disk_io_throttling as Mike
> >>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> type, I
> >>>>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> >>>>>>> expand
> >>>>>>>>>>> on
> >>>>>>>>>>>>> my thoughts
> >>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> touch
> >>>>>>> points
> >>>>>>>>>>>>> between these two
> >>>>>>>>>>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>> need
> >>>>>>> to be
> >>>>>>>>>>>>> merged before
> >>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> coordination
> >>>>>>>>>>>>> between the
> >>>>>>>>>>>>>> branches
> >>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> disk_io_throttling)
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> 2013/6/3
> >>>>>>>>>>> John
> >>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> The
> >>>>>>> things I
> >>>>>>>>>>>>> want to
> >>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
> >>>> Is
> >>>>>>> there
> >>>>>>>>>>>>> value in
> >>>>>>>>>>>>>> capturing
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> data
> >>>>>>> model
> >>>>>>>>>>> (e.g.
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> >>>>>>> Should
> >>>>>>>>>>> there
> >>>>>>>>>>>>> be a common
> >>>>>>>>>>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> provisioning
> >>>>>>>>>>> at
> >>>>>>>>>>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
> >>>> How
> >>>>>>> are
> >>>>>>>>>>>>> conflicting
> >>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> hypervisor
> >>>>>>>>>>> and
> >>>>>>>>>>>>> storage device
> >>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>> lead
> >>>>>>> to
> >>>>>>>>>>>>> believe (and
> >>>>>>>>>>>>>> billed)
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> storage
> >>>>>>>>>>> device
> >>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> consistent
> >>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>> between a
> >>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> time,
> >>>>>>> but a
> >>>>>>>>>>>>> later
> >>>>>>>>>>>>>> modification
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>> Jun
> >>>>>>> 2,
> >>>>>>>>>>> 2013,
> >>>>>>>>>>>>> at 8:38 PM,
> >>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> >>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>> believe
> >>>>>>>>>>> Wei's
> >>>>>>>>>>>>> feature deals
> >>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> My
> >>>>>>> feature
> >>>>>>>>>>> is
> >>>>>>>>>>>>> focused on
> >>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>> hope
> >>>>>>> that
> >>>>>>>>>>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>> Sun,
> >>>>>>> Jun
> >>>>>>>>>>> 2,
> >>>>>>>>>>>>> 2013 at 6:35
> >>>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> My
> >>>>>>>>>>> opinion is
> >>>>>>>>>>>>> that no
> >>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues
> >>>>>>>>>>> have
> >>>>>>>>>>>>> been resolved
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> total
> >>>>>>> Ops
> >>>>>>>>>>> vs
> >>>>>>>>>>>>> discrete
> >>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> >>>>>>> reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido,
> >>>>>>> I
> >>>>>>>>>>> don't
> >>>>>>>>>>>>> think this
> >>>>>>>>>>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Also,
> >>>>>>> how
> >>>>>>>>>>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> also
> >>>>>>>>>>> involves
> >>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> scenario
> >>>>>>>>>>>>> where provisioned
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> On
> >>>>>>> Jun 1,
> >>>>>>>>>>> 2013,
> >>>>>>>>>>>>> at 6:47 AM,
> >>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Sure. I
> >>>>>>> will
> >>>>>>>>>>>>> change it next
> >>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> 2013/6/1
> >>>>>>>>>>>>> Wido den Hollander
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> On
> >>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Exactly. I
> >>>>>>>>>>> have
> >>>>>>>>>>>>> pushed the
> >>>>>>>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>> anyone
> >>>>>>>>>>>>> object thems for
> >>>>>>>>>>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> For
> >>>>>>> the
> >>>>>>>>>>> sake
> >>>>>>>>>>>>> of clarity I
> >>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> the
> >>>>>>> total
> >>>>>>>>>>> IOps
> >>>>>>>>>>>>> to R/W IOps
> >>>>>>>>>>>>>>>>> asap
> >>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> only
> >>>>>>> total
> >>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> You
> >>>>>>> laid
> >>>>>>>>>>> the
> >>>>>>>>>>>>> groundwork for
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> however
> >>>>>>>>>>>>> prevent that we
> >>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> 2013/5/31
> >>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> On
> >>>>>>>>>>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> +1 --
> >>>>>>> this
> >>>>>>>>>>>>> enhancement must
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> don't
> >>>>>>> see
> >>>>>>>>>>> how
> >>>>>>>>>>>>> it could be
> >>>>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> split
> >>>>>>> total
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>> into read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> total
> >>>>>>>>>>>>> unless/until we
> >>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> What
> >>>>>>> Wei
> >>>>>>>>>>>>> meant was merging
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> 4.2
> >>>>>>> branch
> >>>>>>>>>>>>> and add Read /
> >>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> will be
> >>>>>>>>>>>>> released with Read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> This is
> >>>>>>> to
> >>>>>>>>>>>>> make the May 31st
> >>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (see
> >>>>>>> other
> >>>>>>>>>>>>> threads) then it
> >>>>>>>>>>>>>>>>> won't
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>> also
> >>>>>>>>>>>>> completely agree that
> >>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> disk
> >>>>>>> I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> On
> >>>>>>> May 31,
> >>>>>>>>>>>>> 2013, at 9:51 AM,
> >>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> On
> >>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Hi
> >>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Thanks.
> >>>>>>>>>>> Good
> >>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>> thought
> >>>>>>>>>>>>> about at the
> >>>>>>>>>>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> difference
> >>>>>>>>>>> of
> >>>>>>>>>>>>> read and write
> >>>>>>>>>>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> care
> >>>>>>> the
> >>>>>>>>>>>>> difference of sent
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> That
> >>>>>>>>>>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Disk
> >>>>>>> I/O is
> >>>>>>>>>>>>> much more
> >>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> Implementing
> >>>>>>>>>>>>> it will be some
> >>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> implemented
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> few
> >>>>>>> days.
> >>>>>>>>>>> For
> >>>>>>>>>>>>> the deadline
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> that ,
> >>>>>>> if
> >>>>>>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>> think it's
> >>>>>>>>>>> a
> >>>>>>>>>>>>> feature we
> >>>>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> window
> >>>>>>>>>>> we
> >>>>>>>>>>>>> have to make sure
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>> in
> >>>>>>> 4.3,
> >>>>>>>>>>> that
> >>>>>>>>>>>>> will confuse
> >>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> 2013/5/31
> >>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> On
> >>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>> would
> >>>>>>>>>>> like to
> >>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>> nobody
> >>>>>>>>>>>>> object, I will
> >>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> The
> >>>>>>>>>>> purpose
> >>>>>>>>>>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Virtual
> >>>>>>>>>>>>> machines are running
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> share
> >>>>>>>>>>> strage).
> >>>>>>>>>>>>> Because of
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> iops),
> >>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> one
> >>>>>>> VM
> >>>>>>>>>>> has
> >>>>>>>>>>>>> large disk
> >>>>>>>>>>>>>>>>> operation,
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> other
> >>>>>>> VMs
> >>>>>>>>>>>>> running on the
> >>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>> is
> >>>>>>>>>>> neccesary
> >>>>>>>>>>>>> to set the
> >>>>>>>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Looking at
> >>>>>>>>>>> the
> >>>>>>>>>>>>> code I see
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Qemu
> >>>>>>> and
> >>>>>>>>>>>>> libvirt support
> >>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> IOps
> >>>>>>>>>>> which
> >>>>>>>>>>>>> could benefit a
> >>>>>>>>>>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> It's
> >>>>>>> also
> >>>>>>>>>>>>> strange, in the
> >>>>>>>>>>>>>>>>> polling
> >>>>>>>>>>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> IOps,
> >>>>>>> but
> >>>>>>>>>>> on
> >>>>>>>>>>>>> the throttling
> >>>>>>>>>>>>>>>>> side
> >>>>>>>>>>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Write
> >>>>>>> IOps
> >>>>>>>>>>> are
> >>>>>>>>>>>>> usually much
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> like a
> >>>>>>> valid
> >>>>>>>>>>>>> use-case where
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> IOps
> >>>>>>> vs
> >>>>>>>>>>> Read
> >>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Since
> >>>>>>> this
> >>>>>>>>>>> only
> >>>>>>>>>>>>> supports KVM
> >>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> value
> >>>>>>> to at
> >>>>>>>>>>>>> least have the
> >>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> this
> >>>>>>> later
> >>>>>>>>>>>>> would be a lot of
> >>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>> a
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> doesn't
> >>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> write
> >>>>>>> you
> >>>>>>>>>>> can
> >>>>>>>>>>>>> always sum
> >>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Can
> >>>>>>> you
> >>>>>>>>>>>>> explain why you
> >>>>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> The
> >>>>>>>>>>> feature
> >>>>>>>>>>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (1) set
> >>>>>>> the
> >>>>>>>>>>>>> maximum rate of
> >>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (2)
> >>>>>>> change
> >>>>>>>>>>> the
> >>>>>>>>>>>>> maximum rate
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (3)
> >>>>>>> limit
> >>>>>>>>>>> the
> >>>>>>>>>>>>> disk rate
> >>>>>>>>>>>>>> (total
> >>>>>>>>>>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> JIRA
> >>>>>>> ticket:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>> 1192
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> FS (I
> >>>>>>> will
> >>>>>>>>>>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Merge
> >>>>>>>>>>> check
> >>>>>>>>>>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >>>> Did
> >>>>>>> you
> >>>>>>>>>>>>> check the branch's
> >>>>>>>>>>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >>>> Are
> >>>>>>>>>>> there
> >>>>>>>>>>>>> new dependencies
> >>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >>>>>>> What
> >>>>>>>>>>>>> automated testing
> >>>>>>>>>>>>>> (unit
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Unit
> >>>>>>> tests
> >>>>>>>>>>> are
> >>>>>>>>>>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >>>>>>> What
> >>>>>>>>>>>>> testing has been done
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (1) set
> >>>>>>> the
> >>>>>>>>>>>>> bytes rate and
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (2)
> >>>>>>> VM
> >>>>>>>>>>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> deploy,
> >>>>>>>>>>> stop,
> >>>>>>>>>>>>> start, reboot,
> >>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (3)
> >>>>>>>>>>> Volume
> >>>>>>>>>>>>> operations,
> >>>>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Attach,
> >>>>>>>>>>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> To
> >>>>>>> review
> >>>>>>>>>>> the
> >>>>>>>>>>>>> code, you can
> >>>>>>>>>>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Best
> >>>>>>>>>>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> [2]
> >>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> >>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>>>>> VM
> >>>>>>>>>>> Disk
> >>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Mike
> >>>>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Senior
> >>>>>>>>>>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Advancing
> >>>>>>>>>>> the
> >>>>>>>>>>>>> way the world
> >>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Advancing
> >>>>>>> the
> >>>>>>>>>>> way
> >>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Advancing the
> >>>>>>>>>>> way
> >>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >>>> the
> >>>>>>> way
> >>>>>>>>>>> the
> >>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >>>> the
> >>>>>>> way
> >>>>>>>>>>> the
> >>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>>>>>>>>> Developer, SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>>> way
> >>>>>>> the
> >>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>>> way
> >>>>>>> the
> >>>>>>>>>>> world
> >>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>>> way the
> >>>>>>>>>>> world
> >>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >>>> the
> >>>>>>> world
> >>>>>>>>>>> uses
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>>>>> world
> >>>>>>>>>>> uses
> >>>>>>>>>>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>> world
> >>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>> world
> >>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>>> uses
> >>>>>>> the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>>> uses
> >>>>>>> the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>>> uses
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *(tm)*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *(tm)*****
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*****
> >>>>>>
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>
> >>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>
> >>>>>> o: 303.746.7302****
> >>>>>>
> >>>>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *(tm)*****
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*****
> >>>>>>
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>
> >>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>
> >>>>>> o: 303.746.7302****
> >>>>>>
> >>>>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *(tm)*****
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *(tm)*
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Chip Childers <ch...@sungard.com>.
On Wed, Jun 12, 2013 at 12:26:32AM -0600, Mike Tutkowski wrote:
> Thanks for all the work on this everyone!
+1
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
Will you be on today's Go to meeting? We can talk about your stuff.
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Tuesday, June 11, 2013 10:20 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> I am OK with it either way.
>
> Edison, do you still have a preference?
>
> Thanks!
>
>
> On Tue, Jun 11, 2013 at 11:14 AM, John Burwell <jb...@basho.com>
> wrote:
>
> > Mike,
> >
> > So my dependency graph below is incorrect. If there is no dependency
> > between object_store and solidfire, why wouldn't merge them
> > separately? I ask because the object_store patch is already very
> > large. As a reviewer try to comprehend the changes, a series of
> > smaller of patches is easier to digest .
> >
> > Thanks,
> > -John
> >
> > On Jun 11, 2013, at 1:10 PM, Mike Tutkowski
> > <mi...@solidfire.com>
> > wrote:
> >
> > > Hey John,
> > >
> > > The SolidFire patch does not depend on the object_store branch, but
> > > - as Edison mentioned - it might be easier if we merge the SolidFire
> > > branch
> > into
> > > the object_store branch before object_store goes into master.
> > >
> > > I'm not sure how the disk_io_throttling fits into this merge strategy.
> > > Perhaps Wei can chime in on that.
> > >
> > >
> > > On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
> > wrote:
> > >
> > >> Mike,
> > >>
> > >> We have a delicate merge dance to perform. The disk_io_throttling,
> > >> solidfire, and object_store appear to have a number of overlapping
> > >> elements. I understand the dependencies between the patches to be
> > >> as
> > >> follows:
> > >>
> > >> object_store <- solidfire -> disk_io_throttling
> > >>
> > >> Am I correct that the device management aspects of SolidFire are
> > additive
> > >> to the object_store branch or there are circular dependency between
> > >> the branches? Once we understand the dependency graph, we can
> > >> determine the best approach to land the changes in master.
> > >>
> > >> Thanks,
> > >> -John
> > >>
> > >>
> > >> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > >> wrote:
> > >>
> > >>> Also, if we are good with Edison merging my code into his branch
> > >>> before going into master, I am good with that.
> > >>>
> > >>> We can remove the StoragePoolType.Dynamic code after his merge
> and
> > >>> we
> > can
> > >>> deal with Burst IOPS then, as well.
> > >>>
> > >>>
> > >>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> > >>> mike.tutkowski@solidfire.com> wrote:
> > >>>
> > >>>> Let me make sure I follow where we're going here:
> > >>>>
> > >>>> 1) There should be NO references to hypervisor code in the
> > >>>> storage plug-ins code (this includes the default storage plug-in,
> > >>>> which
> > >> currently
> > >>>> sends several commands to the hypervisor in use (although it does
> > >>>> not
> > >> know
> > >>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> > >>>>
> > >>>> 2) managed=true or managed=false can be placed in the url field
> > >>>> (if
> > not
> > >>>> present, we default to false). This info is stored in the
> > >>>> storage_pool_details table.
> > >>>>
> > >>>> 3) When the "attach" command is sent to the hypervisor in
> > >>>> question, we pass the managed property along (this takes the
> > >>>> place of the StoragePoolType.Dynamic check).
> > >>>>
> > >>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> > managed
> > >>>> property. If true for an attach, the necessary hypervisor data
> > >> structure is
> > >>>> created and the rest of the attach command executes to attach the
> > >> volume.
> > >>>>
> > >>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> > >>>> volume,
> > the
> > >>>> same check is made. If managed, the hypervisor data structure is
> > >> removed.
> > >>>>
> > >>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
> > >>>> it is stored in the volumes and disk_offerings table. If we have
> > >>>> some idea, that'd be cool.
> > >>>>
> > >>>> Thanks!
> > >>>>
> > >>>>
> > >>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> > >>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>
> > >>>>> "+1 -- Burst IOPS can be implemented while avoiding
> > >>>>> implementation attributes. I always wondered about the details
> > >>>>> field. I think we
> > >> should
> > >>>>> beef up the description in the documentation regarding the
> > >>>>> expected
> > >> format
> > >>>>> of the field. In 4.1, I noticed that the details are not
> > >>>>> returned on
> > >> the
> > >>>>> createStoratePool updateStoragePool, or listStoragePool response.
> > Why
> > >>>>> don't we return it? It seems like it would be useful for
> > >>>>> clients to
> > be
> > >>>>> able to inspect the contents of the details field."
> > >>>>>
> > >>>>> Not sure how this would work storing Burst IOPS here.
> > >>>>>
> > >>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> > >>>>> Offering basis. For each Disk Offering created, you have to be
> > >>>>> able to
> > associate
> > >>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
> > >>>>> it
> > >> could
> > >>>>> go there?
> > >>>>>
> > >>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
> > >>>>> if
> > >> it's
> > >>>>> not stored like the Min and Max fields are in the DB.
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> *Mike Tutkowski*
> > >>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>> e: mike.tutkowski@solidfire.com
> > >>>> o: 303.746.7302
> > >>>> Advancing the way the world uses the cloud<
> > >> http://solidfire.com/solution/overview/?video=play>
> > >>>> *(tm)*
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> *Mike Tutkowski*
> > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>> e: mike.tutkowski@solidfire.com
> > >>> o: 303.746.7302
> > >>> Advancing the way the world uses the
> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>> *(tm)*
> > >>
> > >>
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *(tm)*
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I am OK with it either way.
Edison, do you still have a preference?
Thanks!
On Tue, Jun 11, 2013 at 11:14 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> So my dependency graph below is incorrect. If there is no dependency
> between object_store and solidfire, why wouldn't merge them separately? I
> ask because the object_store patch is already very large. As a reviewer
> try to comprehend the changes, a series of smaller of patches is easier to
> digest .
>
> Thanks,
> -John
>
> On Jun 11, 2013, at 1:10 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hey John,
> >
> > The SolidFire patch does not depend on the object_store branch, but - as
> > Edison mentioned - it might be easier if we merge the SolidFire branch
> into
> > the object_store branch before object_store goes into master.
> >
> > I'm not sure how the disk_io_throttling fits into this merge strategy.
> > Perhaps Wei can chime in on that.
> >
> >
> > On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> We have a delicate merge dance to perform. The disk_io_throttling,
> >> solidfire, and object_store appear to have a number of overlapping
> >> elements. I understand the dependencies between the patches to be as
> >> follows:
> >>
> >> object_store <- solidfire -> disk_io_throttling
> >>
> >> Am I correct that the device management aspects of SolidFire are
> additive
> >> to the object_store branch or there are circular dependency between the
> >> branches? Once we understand the dependency graph, we can determine the
> >> best approach to land the changes in master.
> >>
> >> Thanks,
> >> -John
> >>
> >>
> >> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Also, if we are good with Edison merging my code into his branch before
> >>> going into master, I am good with that.
> >>>
> >>> We can remove the StoragePoolType.Dynamic code after his merge and we
> can
> >>> deal with Burst IOPS then, as well.
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> Let me make sure I follow where we're going here:
> >>>>
> >>>> 1) There should be NO references to hypervisor code in the storage
> >>>> plug-ins code (this includes the default storage plug-in, which
> >> currently
> >>>> sends several commands to the hypervisor in use (although it does not
> >> know
> >>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>>>
> >>>> 2) managed=true or managed=false can be placed in the url field (if
> not
> >>>> present, we default to false). This info is stored in the
> >>>> storage_pool_details table.
> >>>>
> >>>> 3) When the "attach" command is sent to the hypervisor in question, we
> >>>> pass the managed property along (this takes the place of the
> >>>> StoragePoolType.Dynamic check).
> >>>>
> >>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> managed
> >>>> property. If true for an attach, the necessary hypervisor data
> >> structure is
> >>>> created and the rest of the attach command executes to attach the
> >> volume.
> >>>>
> >>>> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
> the
> >>>> same check is made. If managed, the hypervisor data structure is
> >> removed.
> >>>>
> >>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
> >>>> stored in the volumes and disk_offerings table. If we have some idea,
> >>>> that'd be cool.
> >>>>
> >>>> Thanks!
> >>>>
> >>>>
> >>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> "+1 -- Burst IOPS can be implemented while avoiding implementation
> >>>>> attributes. I always wondered about the details field. I think we
> >> should
> >>>>> beef up the description in the documentation regarding the expected
> >> format
> >>>>> of the field. In 4.1, I noticed that the details are not returned on
> >> the
> >>>>> createStoratePool updateStoragePool, or listStoragePool response.
> Why
> >>>>> don't we return it? It seems like it would be useful for clients to
> be
> >>>>> able to inspect the contents of the details field."
> >>>>>
> >>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>
> >>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
> >>>>> basis. For each Disk Offering created, you have to be able to
> associate
> >>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe it
> >> could
> >>>>> go there?
> >>>>>
> >>>>> I'm also not sure how you would accept the Burst IOPS in the GUI if
> >> it's
> >>>>> not stored like the Min and Max fields are in the DB.
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
So my dependency graph below is incorrect. If there is no dependency between object_store and solidfire, why wouldn't merge them separately? I ask because the object_store patch is already very large. As a reviewer try to comprehend the changes, a series of smaller of patches is easier to digest .
Thanks,
-John
On Jun 11, 2013, at 1:10 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hey John,
>
> The SolidFire patch does not depend on the object_store branch, but - as
> Edison mentioned - it might be easier if we merge the SolidFire branch into
> the object_store branch before object_store goes into master.
>
> I'm not sure how the disk_io_throttling fits into this merge strategy.
> Perhaps Wei can chime in on that.
>
>
> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> We have a delicate merge dance to perform. The disk_io_throttling,
>> solidfire, and object_store appear to have a number of overlapping
>> elements. I understand the dependencies between the patches to be as
>> follows:
>>
>> object_store <- solidfire -> disk_io_throttling
>>
>> Am I correct that the device management aspects of SolidFire are additive
>> to the object_store branch or there are circular dependency between the
>> branches? Once we understand the dependency graph, we can determine the
>> best approach to land the changes in master.
>>
>> Thanks,
>> -John
>>
>>
>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Also, if we are good with Edison merging my code into his branch before
>>> going into master, I am good with that.
>>>
>>> We can remove the StoragePoolType.Dynamic code after his merge and we can
>>> deal with Burst IOPS then, as well.
>>>
>>>
>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> Let me make sure I follow where we're going here:
>>>>
>>>> 1) There should be NO references to hypervisor code in the storage
>>>> plug-ins code (this includes the default storage plug-in, which
>> currently
>>>> sends several commands to the hypervisor in use (although it does not
>> know
>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>
>>>> 2) managed=true or managed=false can be placed in the url field (if not
>>>> present, we default to false). This info is stored in the
>>>> storage_pool_details table.
>>>>
>>>> 3) When the "attach" command is sent to the hypervisor in question, we
>>>> pass the managed property along (this takes the place of the
>>>> StoragePoolType.Dynamic check).
>>>>
>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the managed
>>>> property. If true for an attach, the necessary hypervisor data
>> structure is
>>>> created and the rest of the attach command executes to attach the
>> volume.
>>>>
>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a volume, the
>>>> same check is made. If managed, the hypervisor data structure is
>> removed.
>>>>
>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
>>>> stored in the volumes and disk_offerings table. If we have some idea,
>>>> that'd be cool.
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>>>>> attributes. I always wondered about the details field. I think we
>> should
>>>>> beef up the description in the documentation regarding the expected
>> format
>>>>> of the field. In 4.1, I noticed that the details are not returned on
>> the
>>>>> createStoratePool updateStoragePool, or listStoragePool response. Why
>>>>> don't we return it? It seems like it would be useful for clients to be
>>>>> able to inspect the contents of the details field."
>>>>>
>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>
>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
>>>>> basis. For each Disk Offering created, you have to be able to associate
>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe it
>> could
>>>>> go there?
>>>>>
>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI if
>> it's
>>>>> not stored like the Min and Max fields are in the DB.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I agree with your "None" radio button point, John.
On Wed, Jun 12, 2013 at 4:03 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> I hate to say it, but I believe Storage QoS with a Min and Max will always
> be optimal over hypervisor rate limiting.
>
> The only time you'd want to use hypervisor rate limiting is if storage QoS
> is not available.
>
> We currently have no way to know what storage the root or data disk will
> be deployed to, so we have to present the fields in all situation.
>
> This is because of - in my opinion - the somewhat flawed way CloudStack
> handles storage tags. You are free to enter in any text there and we don't
> know what it maps to when the Compute or Disk Offering dialog is up.
>
>
> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Please see my comments/questions in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> Hi Wei,
>>
>> So, not entirely sure I follow.
>>
>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>
>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>> QoS for the purpose of this e-mail.
>>
>> By default, neither radio button is selected and no QoS fields are
>> visible.
>>
>>
>> I prefer a None option selected by default. I find the no button
>> selected in a Radio button group to be confusing.
>>
>> Also, do we have a mechanism to determine whether or not this radio
>> button group should be displayed at all? If there are no devices or
>> hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>>
>>
>> If the user selects the Storage QoS radio button, then the Custom IOPS
>> checkbox and the Min and Max text fields are made visible.
>>
>> If the user changes his mind and selects the Hypervisor QoS radio button,
>> then the Custom IOPS checkbox and the Min and Max text fields disappear and
>> are replaced by the two Hypervisor QoS text fields.
>>
>> This way, the user can choose neither QoS option or one of them or the
>> other, but never both.
>>
>> On the API side, I was thinking of having logic in place when a request
>> comes in to create a Disk Offering to confirm these fields are the way we
>> want them.
>>
>> Once we know the Disk Offering is in order, a user can create a data disk
>> from it. Since we checked the validity of the Disk Offering when it was
>> created, the VM should never be asked to use Hypervisor QoS when Storage
>> QoS in being utilized.
>>
>>
>> Overall, this model sounds very reasonable. Wondering aloud: Would be
>> possible to have the UI simply ask for a storage QoS, and let the system
>> pick the optional implementation. For example, if a VM is attached to KVM
>> and SolidFire is available, we chose to fulfill the QoS using the storage
>> device. This approach would also allow us to fallback in the event that we
>> can't allocate storage for the QoS, but we have hypervisor capacity. Could
>> we always deterministically determine the optimal implementation of the QoS
>> at time of VM deployment?
>>
>>
>> Thanks!
>>
>>
>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> Mike, John
>>>
>>> The Disk I/O Throttling works like:
>>>
>>> (1) For User VM,
>>> (1.1) root disks: service offering -> default value in global
>>> configuration
>>> (1.2) data disks: disk offering -> default value in global configuration
>>>
>>> (2) For System VM
>>> root disks: service offering
>>>
>>> -Wei
>>>
>>>
>>> 2013/6/12 John Burwell <jb...@basho.com>
>>>
>>> > Mike,
>>> >
>>> > That is one possibility. The other possibility is that hypervisor is
>>> > going to throttle I/O on all disks attached. Therefore, we need
>>> answers to
>>> > the following questions:
>>> >
>>> > 1. Is I/O throttling applied to the root disk or all disks
>>> > attached to the VM?
>>> > 2. If I/O throttling is applied to all disks, how is the
>>> > throttling distributed amongst the disks if only one read/write value
>>> is
>>> > defined?
>>> >
>>> > Thanks,
>>> > -John
>>> >
>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>> > wrote:
>>> >
>>> > > Hey John,
>>> > >
>>> > > Perhaps I don't fully understand how Wei's feature works.
>>> > >
>>> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>>> > >
>>> > > From my thinking, you're root disk could be under Hypervisor QoS, but
>>> > your
>>> > > data disk could be under Storage QoS.
>>> > >
>>> > > Is that incorrect?
>>> > >
>>> > > Thanks
>>> > >
>>> > >
>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>>> > wrote:
>>> > >
>>> > >> Mike,
>>> > >>
>>> > >> As I understand these two patches, the throttled I?O settings are
>>> > applied
>>> > >> from the hypervisor side, and possibly defined in a compute offering
>>> > where
>>> > >> provisioned IOPS are defined on the storage side through disk
>>> > offerings. I
>>> > >> don't see how the management server could enforce this mutual
>>> exclusion
>>> > >> between provisioned IOPS and throttled I/O until a compute and disk
>>> > >> offering were selected. These selections come together when we
>>> deploy a
>>> > >> VM. Based on these assumptions, I would expect to see enforcement
>>> of
>>> > this
>>> > >> rule in UI at the time of VM creation/definition and on the server
>>> > side, as
>>> > >> part of the VM creation. It feels like any attempt to enforce this
>>> rule
>>> > >> when defining offering would be premature, and unnecessarily limit
>>> > >> flexibility. Are my assumptions and understanding correct?
>>> > >>
>>> > >> Thanks,
>>> > >> -John
>>> > >>
>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>>> > mike.tutkowski@solidfire.com>
>>> > >> wrote:
>>> > >>
>>> > >>> Hi John,
>>> > >>>
>>> > >>> So, maybe I'm wrong about this, but what I was thinking is that
>>> we'd
>>> > >> build
>>> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
>>> > >> Compute
>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>>> > >>>
>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
>>> Storage
>>> > >> QoS
>>> > >>> for the purpose of this e-mail.
>>> > >>>
>>> > >>> By default, neither radio button is selected and no QoS fields are
>>> > >> visible.
>>> > >>>
>>> > >>> If the user selects the Storage QoS radio button, then the Custom
>>> IOPS
>>> > >>> checkbox and the Min and Max text fields are made visible.
>>> > >>>
>>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>>> > button,
>>> > >>> then the Custom IOPS checkbox and the Min and Max text fields
>>> disappear
>>> > >> and
>>> > >>> are replaced by the two Hypervisor QoS text fields.
>>> > >>>
>>> > >>> This way, the user can choose neither QoS option or one of them or
>>> the
>>> > >>> other, but never both.
>>> > >>>
>>> > >>> On the API side, I was thinking of having logic in place when a
>>> request
>>> > >>> comes in to create a Disk Offering to confirm these fields are the
>>> way
>>> > we
>>> > >>> want them.
>>> > >>>
>>> > >>> Once we know the Disk Offering is in order, a user can create a
>>> data
>>> > disk
>>> > >>> from it. Since we checked the validity of the Disk Offering when
>>> it was
>>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>>> > Storage
>>> > >>> QoS in being utilized.
>>> > >>>
>>> > >>> Does that make sense or did I miss something?
>>> > >>>
>>> > >>> Thanks
>>> > >>>
>>> > >>>
>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>>> > >> wrote:
>>> > >>>
>>> > >>>> Mike,
>>> > >>>>
>>> > >>>> Looking through the code, I am trying to understand how
>>> > >>>> CreateDiskOfferingCmd would have the context to identify the
>>> conflict.
>>> > >>>> Naively, it seems to me that this rule would need to be enforced
>>> when
>>> > a
>>> > >>>> virtual machine is being deployed. Looking through the code, it
>>> seems
>>> > >> like
>>> > >>>> we should add a private validateStorageQoS method to
>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws
>>> a
>>> > >>>> ResourceAllocationException when the QoS definitions are
>>> inconsistent.
>>> > >> We
>>> > >>>> would then add calls to it from each of the VM creation methods
>>> in the
>>> > >>>> service. Do this type of approach sound reasonable?
>>> > >>>>
>>> > >>>> Thanks,
>>> > >>>> -John
>>> > >>>>
>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>>> > >> mike.tutkowski@solidfire.com>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>> Hi John,
>>> > >>>>>
>>> > >>>>> So, here's what I was planning to do. Of course feel free to
>>> correct
>>> > me
>>> > >>>> on
>>> > >>>>> this approach.
>>> > >>>>>
>>> > >>>>> I think it's OK if Wei merges his code into master and then I can
>>> > draw
>>> > >>>> from
>>> > >>>>> the main repo and merge master into mine locally.
>>> > >>>>>
>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI
>>> code
>>> > to
>>> > >>>> make
>>> > >>>>> it user friendly (toggle between these features on the Add Disk
>>> > >> Offering
>>> > >>>>> window).
>>> > >>>>>
>>> > >>>>> 2) I plan to write validation logic for the create-disk-offering
>>> API
>>> > >>>>> command which throws an exception if the rules are not followed
>>> (this
>>> > >>>>> should never be triggered from the GUI since the GUI will have
>>> > controls
>>> > >>>> in
>>> > >>>>> place to toggle between the one feature and the other).
>>> > >>>>>
>>> > >>>>> I'm not sure about documentation. I haven't had much experience
>>> with
>>> > it
>>> > >>>> on
>>> > >>>>> CloudStack projects yet.
>>> > >>>>>
>>> > >>>>> Thanks!
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <
>>> jburwell@basho.com>
>>> > >>>> wrote:
>>> > >>>>>
>>> > >>>>>> Mike,
>>> > >>>>>>
>>> > >>>>>> Yes, these server-side rails need to be defined and implemented
>>> > before
>>> > >>>>>> either patch can be merged. From my perspective, I would like
>>> to
>>> > see
>>> > >>>> the
>>> > >>>>>> rule implemented in the hypervisor as part of the validation of
>>> the
>>> > >>>> virtual
>>> > >>>>>> machine definition. We also need to make sure that this mutual
>>> > >>>> exclusion
>>> > >>>>>> is documented. Do we usually include this type of documentation
>>> > with
>>> > >>>>>> patches of this nature?
>>> > >>>>>>
>>> > >>>>>> Thanks,
>>> > >>>>>> -John
>>> > >>>>>>
>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>> > >>>> mike.tutkowski@solidfire.com>
>>> > >>>>>> wrote:
>>> > >>>>>>
>>> > >>>>>>> Currently they are not yet implemented.
>>> > >>>>>>>
>>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>>> > usability
>>> > >>>>>>> standpoint, but the API must check for consistency and throw an
>>> > >>>> exception
>>> > >>>>>>> if necessary.
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
>>> jburwell@basho.com
>>> > >
>>> > >>>>>> wrote:
>>> > >>>>>>>
>>> > >>>>>>>> Mike,
>>> > >>>>>>>>
>>> > >>>>>>>> Are the checks only implemented in the UI?
>>> > >>>>>>>>
>>> > >>>>>>>> Thanks,
>>> > >>>>>>>> -John
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>> > >>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>
>>> > >>>>>>>>> Hi John,
>>> > >>>>>>>>>
>>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>>> > >> exclusive.
>>> > >>>> We
>>> > >>>>>>>>> agree with you that only one should be active at a time. We
>>> plan
>>> > to
>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to
>>> turn
>>> > his
>>> > >>>>>>>> feature
>>> > >>>>>>>>> on and mine off and vice versa.
>>> > >>>>>>>>>
>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
>>> > update
>>> > >>>> and
>>> > >>>>>>>>> merge that I will be the person resolving merge conflicts in
>>> the
>>> > >>>>>>>> JavaScript
>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as
>>> opposed
>>> > to
>>> > >>>>>>>> putting
>>> > >>>>>>>>> that work on someone else.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Let me know what you think.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Thanks!
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>>> > jburwell@basho.com
>>> > >>>
>>> > >>>>>>>> wrote:
>>> > >>>>>>>>>
>>> > >>>>>>>>>> Mike,
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>> > >>>> mechanisms
>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the
>>> affected
>>> > FSs
>>> > >>>> been
>>> > >>>>>>>>>> updated with that decision?
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
>>> > >> between
>>> > >>>>>> the
>>> > >>>>>>>>>> patches?
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> Thanks,
>>> > >>>>>>>>>> -John
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>> No problem, John.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up
>>> with a
>>> > >> new
>>> > >>>>>>>> patch
>>> > >>>>>>>>>>> file.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>> checked
>>> > >> in
>>> > >>>>>> and
>>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>> > >> jburwell@basho.com
>>> > >>>>>
>>> > >>>>>>>>>> wrote:
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>> Mike,
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>> > >> offline
>>> > >>>>>> ATM,
>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> Sorry for the oversight,
>>> > >>>>>>>>>>>> -John
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>> this :)
>>> > >> ),
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>> areas
>>> > we
>>> > >>>>>> wanted
>>> > >>>>>>>>>>>>> addressed.
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>> > afternoon,
>>> > >>>> then
>>> > >>>>>>>>>> send
>>> > >>>>>>>>>>>> in
>>> > >>>>>>>>>>>>> another patch.
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> Sure, that sounds good.
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> > >>>> ustcweizhou@gmail.com
>>> > >>>>>>>
>>> > >>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> Hi Mike,
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>>> Java
>>> > >>>> code,
>>> > >>>>>>>>>>>> except
>>> > >>>>>>>>>>>>>>> the cloudstack UI.
>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>> branch
>>> > >> into
>>> > >>>>>>>>>> master
>>> > >>>>>>>>>>>>>>> this
>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> -Wei
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.com
>>> >
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> Hey John,
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>> object_store
>>> > >>>> branch,
>>> > >>>>>>>> but
>>> > >>>>>>>>>> -
>>> > >>>>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>> > >>>> SolidFire
>>> > >>>>>>>>>> branch
>>> > >>>>>>>>>>>>>>> into
>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>> > >> master.
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>> > merge
>>> > >>>>>>>>>> strategy.
>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>> > >>>>>>>> jburwell@basho.com>
>>> > >>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Mike,
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>> > >>>>>>>> disk_io_throttling,
>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>> of
>>> > >>>>>>>> overlapping
>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>> > >> patches
>>> > >>>> to
>>> > >>>>>>>> be
>>> > >>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>>> follows:
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>> > >> SolidFire
>>> > >>>>>> are
>>> > >>>>>>>>>>>>>>> additive
>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>> > dependency
>>> > >>>>>>>> between
>>> > >>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>> we
>>> > can
>>> > >>>>>>>>>> determine
>>> > >>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Thanks,
>>> > >>>>>>>>>>>>>>>>> -John
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> > >>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>> into
>>> > his
>>> > >>>>>> branch
>>> > >>>>>>>>>>>>>>> before
>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>> after his
>>> > >>>> merge
>>> > >>>>>>>> and
>>> > >>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>> can
>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>> code in
>>> > >> the
>>> > >>>>>>>>>> storage
>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>> > plug-in,
>>> > >>>>>> which
>>> > >>>>>>>>>>>>>>>>> currently
>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>> > (although
>>> > >>>> it
>>> > >>>>>>>> does
>>> > >>>>>>>>>>>>>>> not
>>> > >>>>>>>>>>>>>>>>> know
>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>> actually
>>> > in
>>> > >>>>>> use))
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>> the
>>> > url
>>> > >>>>>> field
>>> > >>>>>>>>>> (if
>>> > >>>>>>>>>>>>>>>> not
>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>>> in
>>> > the
>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>> hypervisor
>>> > in
>>> > >>>>>>>>>>>>>>> question, we
>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>> place
>>> > of
>>> > >>>> the
>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>> > checks
>>> > >>>> for
>>> > >>>>>>>> the
>>> > >>>>>>>>>>>>>>>> managed
>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>> > hypervisor
>>> > >>>>>> data
>>> > >>>>>>>>>>>>>>>>> structure is
>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>> executes to
>>> > >>>> attach
>>> > >>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> volume.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>> > >> detach a
>>> > >>>>>>>>>> volume,
>>> > >>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> > >>>> structure
>>> > >>>>>>>> is
>>> > >>>>>>>>>>>>>>>>> removed.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>>> in
>>> > 4.2
>>> > >>>>>>>> unless
>>> > >>>>>>>>>>>>>>> it is
>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>>> we
>>> > >> have
>>> > >>>>>> some
>>> > >>>>>>>>>>>>>>> idea,
>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> Thanks!
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>> avoiding
>>> > >>>>>>>>>> implementation
>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>> > field.
>>> > >> I
>>> > >>>>>>>> think
>>> > >>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>>> should
>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>> regarding
>>> > >> the
>>> > >>>>>>>>>>>>>>> expected
>>> > >>>>>>>>>>>>>>>>> format
>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>> are
>>> > >> not
>>> > >>>>>>>>>>>>>>> returned on
>>> > >>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>> > listStoragePool
>>> > >>>>>>>>>> response.
>>> > >>>>>>>>>>>>>>>> Why
>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>> useful
>>> > >> for
>>> > >>>>>>>>>> clients
>>> > >>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>> be
>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>> field."
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>> here.
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>> > >> Offering-by-Disk
>>> > >>>>>>>>>> Offering
>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>> to be
>>> > >> able
>>> > >>>>>> to
>>> > >>>>>>>>>>>>>>>> associate
>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>> disk_offering_details
>>> > >> table.
>>> > >>>>>>>> Maybe
>>> > >>>>>>>>>>>>>>> it
>>> > >>>>>>>>>>>>>>>>> could
>>> > >>>>>>>>>>>>>>>>>>>> go there?
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>> IOPS
>>> > in
>>> > >>>> the
>>> > >>>>>>>> GUI
>>> > >>>>>>>>>>>>>>> if
>>> > >>>>>>>>>>>>>>>>> it's
>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>>> DB.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>>>>>>> cloud<
>>> > http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>> >
>>> > >>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> --
>>> > >>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> --
>>> > >>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>> *â„¢*
>>> > >>>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>> --
>>> > >>>>>>> *Mike Tutkowski*
>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>> o: 303.746.7302
>>> > >>>>>>> Advancing the way the world uses the
>>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>> *â„¢*
>>> > >>>>>>
>>> > >>>>>>
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> --
>>> > >>>>> *Mike Tutkowski*
>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>> o: 303.746.7302
>>> > >>>>> Advancing the way the world uses the
>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>> *â„¢*
>>> > >>>>
>>> > >>>>
>>> > >>>
>>> > >>>
>>> > >>> --
>>> > >>> *Mike Tutkowski*
>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>> e: mike.tutkowski@solidfire.com
>>> > >>> o: 303.746.7302
>>> > >>> Advancing the way the world uses the
>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>> *â„¢*
>>> > >>
>>> > >>
>>> > >
>>> > >
>>> > > --
>>> > > *Mike Tutkowski*
>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>> > > e: mike.tutkowski@solidfire.com
>>> > > o: 303.746.7302
>>> > > Advancing the way the world uses the
>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > > *â„¢*
>>> >
>>> >
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Alex Huang <Al...@citrix.com>.
Hi All,
I didn't follow this thread as much so maybe it's already been discussed. If so, let me know.
I don't see why these limits should be used as a way to do allocation. CS already supports storage tiering through tags. Why not reuse that concept?
A use case would be.
An admin wants to use SolidFire storage for example and enable the rate limiting feature.
SolidFire's plugin add tags to the storage pools it added to cloudstack.
When the admin creates disk offering, they can either add the tag to the disk offering themselves or call an api command on the solid fire plugin to let solid fire plugin add that tag to the disk offering and set the values for the rate limits.
CloudStack's allocator will use tags to match the storage pools. No changes there so big plus.
Rate limits are carried to the storage provider and hypervisor by CS. You can add integration in the resources to implement the rate limits.
There are several benefits to doing this.
- Utilizes something that already works.
- Less code to maintain (don't need to modify all the allocators and planners)
I think even the values themselves should just be name-value pairs rather than fields in the offering. CS should be responsible for transporting these fields to the interested parties (plugins, and resources) but it itself doesn't understand them.
--Alex
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Wednesday, June 12, 2013 3:31 PM
> To: John Burwell
> Cc: dev@cloudstack.apache.org; Wei Zhou
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> We'd also have to recognize that if the Min value is filled in and Storage QoS
> is not available that Hypervisor QoS (rate limiting here) cannot satisfy that
> constraint.
>
>
> On Wed, Jun 12, 2013 at 4:26 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
> > Yeah, this is interesting.
> >
> > We'll have to wait and see what Wei's thoughts are on this.
> >
> >
> > On Wed, Jun 12, 2013 at 4:17 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Yes. To your point, the appropriate logic would be to check the
> >> Volume allocated by the StorageAllocator. If it doesn't support a
> >> QoS, then the VM allocator would attempt to fulfill the QoS through the
> hypervisor.
> >> Another question would be -- what would be in the behavior in the
> >> event that no resources are available to fulfill the QoS? We could
> >> outright fail or proceed with some kind of warning -- sounds like a
> >> good place for a global setting to configure the policy.
> >>
> >> The other question we haven't answered are usage records. By pushing
> >> the QoS concept out to allocation and making it a more general
> >> concept, it could make usage record capture more straightforward.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 12, 2013, at 6:11 PM, Mike Tutkowski
> >> <mi...@solidfire.com>
> >> wrote:
> >>
> >> I see, John.
> >>
> >> That is an interesting idea.
> >>
> >> We'd also have to change the storage allocator(s) to favor
> >> QoS-supported storage if those fields are filled in.
> >>
> >>
> >> On Wed, Jun 12, 2013 at 4:09 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Mike,
> >>>
> >>> My thought is that we present the min/max IOPS fields for read/write
> >>> operations for all compute and disk offerings. When the VM is
> >>> allocated, we determine the best way to fulfill that QoS. It sounds
> >>> like storage level guarantees would always be preferred. If no
> >>> storage is available to guarantee the QoS, the allocator falls back
> >>> to using hypervisor. This approach only works if summing the
> >>> discrete read/write min/max values to get to min/max total IOPS would
> be acceptable.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>>
> >>> On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>> I hate to say it, but I believe Storage QoS with a Min and Max will
> >>> always be optimal over hypervisor rate limiting.
> >>>
> >>> The only time you'd want to use hypervisor rate limiting is if
> >>> storage QoS is not available.
> >>>
> >>> We currently have no way to know what storage the root or data disk
> >>> will be deployed to, so we have to present the fields in all situation.
> >>>
> >>> This is because of - in my opinion - the somewhat flawed way
> >>> CloudStack handles storage tags. You are free to enter in any text
> >>> there and we don't know what it maps to when the Compute or Disk
> Offering dialog is up.
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell
> <jb...@basho.com>wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Please see my comments/questions in-line below.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>> Hi Wei,
> >>>>
> >>>> So, not entirely sure I follow.
> >>>>
> >>>> Will what I wrote earlier work? Here is a copy of what I wrote:
> >>>>
> >>>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
> >>>> Storage QoS for the purpose of this e-mail.
> >>>>
> >>>> By default, neither radio button is selected and no QoS fields are
> >>>> visible.
> >>>>
> >>>>
> >>>> I prefer a None option selected by default. I find the no button
> >>>> selected in a Radio button group to be confusing.
> >>>>
> >>>> Also, do we have a mechanism to determine whether or not this radio
> >>>> button group should be displayed at all? If there are no devices
> >>>> or hypervisors configured to fulfill the QoS, we shouldn't mislead the
> user ...
> >>>>
> >>>>
> >>>> If the user selects the Storage QoS radio button, then the Custom
> >>>> IOPS checkbox and the Min and Max text fields are made visible.
> >>>>
> >>>> If the user changes his mind and selects the Hypervisor QoS radio
> >>>> button, then the Custom IOPS checkbox and the Min and Max text
> >>>> fields disappear and are replaced by the two Hypervisor QoS text fields.
> >>>>
> >>>> This way, the user can choose neither QoS option or one of them or
> >>>> the other, but never both.
> >>>>
> >>>> On the API side, I was thinking of having logic in place when a
> >>>> request comes in to create a Disk Offering to confirm these fields
> >>>> are the way we want them.
> >>>>
> >>>> Once we know the Disk Offering is in order, a user can create a
> >>>> data disk from it. Since we checked the validity of the Disk
> >>>> Offering when it was created, the VM should never be asked to use
> >>>> Hypervisor QoS when Storage QoS in being utilized.
> >>>>
> >>>>
> >>>> Overall, this model sounds very reasonable. Wondering aloud: Would
> >>>> be possible to have the UI simply ask for a storage QoS, and let
> >>>> the system pick the optional implementation. For example, if a VM
> >>>> is attached to KVM and SolidFire is available, we chose to fulfill
> >>>> the QoS using the storage device. This approach would also allow
> >>>> us to fallback in the event that we can't allocate storage for the
> >>>> QoS, but we have hypervisor capacity. Could we always
> >>>> deterministically determine the optimal implementation of the QoS at
> time of VM deployment?
> >>>>
> >>>>
> >>>> Thanks!
> >>>>
> >>>>
> >>>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU
> <us...@gmail.com>wrote:
> >>>>
> >>>>> Mike, John
> >>>>>
> >>>>> The Disk I/O Throttling works like:
> >>>>>
> >>>>> (1) For User VM,
> >>>>> (1.1) root disks: service offering -> default value in global
> >>>>> configuration
> >>>>> (1.2) data disks: disk offering -> default value in global
> >>>>> configuration
> >>>>>
> >>>>> (2) For System VM
> >>>>> root disks: service offering
> >>>>>
> >>>>> -Wei
> >>>>>
> >>>>>
> >>>>> 2013/6/12 John Burwell <jb...@basho.com>
> >>>>>
> >>>>> > Mike,
> >>>>> >
> >>>>> > That is one possibility. The other possibility is that
> >>>>> > hypervisor is going to throttle I/O on all disks attached.
> >>>>> > Therefore, we need
> >>>>> answers to
> >>>>> > the following questions:
> >>>>> >
> >>>>> > 1. Is I/O throttling applied to the root disk or all
> >>>>> > disks attached to the VM?
> >>>>> > 2. If I/O throttling is applied to all disks, how is the
> >>>>> > throttling distributed amongst the disks if only one read/write
> >>>>> value is
> >>>>> > defined?
> >>>>> >
> >>>>> > Thanks,
> >>>>> > -John
> >>>>> >
> >>>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>> > wrote:
> >>>>> >
> >>>>> > > Hey John,
> >>>>> > >
> >>>>> > > Perhaps I don't fully understand how Wei's feature works.
> >>>>> > >
> >>>>> > > I guess I thought if you choose Hypervisor QoS, you do so on
> >>>>> Compute
> >>>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
> >>>>> > >
> >>>>> > > From my thinking, you're root disk could be under Hypervisor
> >>>>> > > QoS,
> >>>>> but
> >>>>> > your
> >>>>> > > data disk could be under Storage QoS.
> >>>>> > >
> >>>>> > > Is that incorrect?
> >>>>> > >
> >>>>> > > Thanks
> >>>>> > >
> >>>>> > >
> >>>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell
> >>>>> > > <jb...@basho.com>
> >>>>> > wrote:
> >>>>> > >
> >>>>> > >> Mike,
> >>>>> > >>
> >>>>> > >> As I understand these two patches, the throttled I?O settings
> >>>>> > >> are
> >>>>> > applied
> >>>>> > >> from the hypervisor side, and possibly defined in a compute
> >>>>> offering
> >>>>> > where
> >>>>> > >> provisioned IOPS are defined on the storage side through disk
> >>>>> > offerings. I
> >>>>> > >> don't see how the management server could enforce this mutual
> >>>>> exclusion
> >>>>> > >> between provisioned IOPS and throttled I/O until a compute
> >>>>> > >> and
> >>>>> disk
> >>>>> > >> offering were selected. These selections come together when
> >>>>> > >> we
> >>>>> deploy a
> >>>>> > >> VM. Based on these assumptions, I would expect to see
> >>>>> enforcement of
> >>>>> > this
> >>>>> > >> rule in UI at the time of VM creation/definition and on the
> >>>>> > >> server
> >>>>> > side, as
> >>>>> > >> part of the VM creation. It feels like any attempt to
> >>>>> > >> enforce
> >>>>> this rule
> >>>>> > >> when defining offering would be premature, and unnecessarily
> >>>>> > >> limit flexibility. Are my assumptions and understanding correct?
> >>>>> > >>
> >>>>> > >> Thanks,
> >>>>> > >> -John
> >>>>> > >>
> >>>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
> >>>>> > mike.tutkowski@solidfire.com>
> >>>>> > >> wrote:
> >>>>> > >>
> >>>>> > >>> Hi John,
> >>>>> > >>>
> >>>>> > >>> So, maybe I'm wrong about this, but what I was thinking is
> >>>>> > >>> that
> >>>>> we'd
> >>>>> > >> build
> >>>>> > >>> two radio buttons into the Add Disk Offering dialog (we can
> >>>>> ignore
> >>>>> > >> Compute
> >>>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
> >>>>> > >>>
> >>>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and
> >>>>> > >>> 2)
> >>>>> Storage
> >>>>> > >> QoS
> >>>>> > >>> for the purpose of this e-mail.
> >>>>> > >>>
> >>>>> > >>> By default, neither radio button is selected and no QoS
> >>>>> > >>> fields
> >>>>> are
> >>>>> > >> visible.
> >>>>> > >>>
> >>>>> > >>> If the user selects the Storage QoS radio button, then the
> >>>>> Custom IOPS
> >>>>> > >>> checkbox and the Min and Max text fields are made visible.
> >>>>> > >>>
> >>>>> > >>> If the user changes his mind and selects the Hypervisor QoS
> >>>>> > >>> radio
> >>>>> > button,
> >>>>> > >>> then the Custom IOPS checkbox and the Min and Max text
> >>>>> > >>> fields
> >>>>> disappear
> >>>>> > >> and
> >>>>> > >>> are replaced by the two Hypervisor QoS text fields.
> >>>>> > >>>
> >>>>> > >>> This way, the user can choose neither QoS option or one of
> >>>>> > >>> them
> >>>>> or the
> >>>>> > >>> other, but never both.
> >>>>> > >>>
> >>>>> > >>> On the API side, I was thinking of having logic in place
> >>>>> > >>> when a
> >>>>> request
> >>>>> > >>> comes in to create a Disk Offering to confirm these fields
> >>>>> > >>> are
> >>>>> the way
> >>>>> > we
> >>>>> > >>> want them.
> >>>>> > >>>
> >>>>> > >>> Once we know the Disk Offering is in order, a user can
> >>>>> > >>> create a
> >>>>> data
> >>>>> > disk
> >>>>> > >>> from it. Since we checked the validity of the Disk Offering
> >>>>> > >>> when
> >>>>> it was
> >>>>> > >>> created, the VM should never be asked to use Hypervisor QoS
> >>>>> > >>> when
> >>>>> > Storage
> >>>>> > >>> QoS in being utilized.
> >>>>> > >>>
> >>>>> > >>> Does that make sense or did I miss something?
> >>>>> > >>>
> >>>>> > >>> Thanks
> >>>>> > >>>
> >>>>> > >>>
> >>>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>> > >> wrote:
> >>>>> > >>>
> >>>>> > >>>> Mike,
> >>>>> > >>>>
> >>>>> > >>>> Looking through the code, I am trying to understand how
> >>>>> > >>>> CreateDiskOfferingCmd would have the context to identify
> >>>>> > >>>> the
> >>>>> conflict.
> >>>>> > >>>> Naively, it seems to me that this rule would need to be
> >>>>> enforced when
> >>>>> > a
> >>>>> > >>>> virtual machine is being deployed. Looking through the
> >>>>> > >>>> code,
> >>>>> it seems
> >>>>> > >> like
> >>>>> > >>>> we should add a private validateStorageQoS method to
> >>>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition
> and
> >>>>> throws a
> >>>>> > >>>> ResourceAllocationException when the QoS definitions are
> >>>>> inconsistent.
> >>>>> > >> We
> >>>>> > >>>> would then add calls to it from each of the VM creation
> >>>>> > >>>> methods
> >>>>> in the
> >>>>> > >>>> service. Do this type of approach sound reasonable?
> >>>>> > >>>>
> >>>>> > >>>> Thanks,
> >>>>> > >>>> -John
> >>>>> > >>>>
> >>>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
> >>>>> > >> mike.tutkowski@solidfire.com>
> >>>>> > >>>> wrote:
> >>>>> > >>>>
> >>>>> > >>>>> Hi John,
> >>>>> > >>>>>
> >>>>> > >>>>> So, here's what I was planning to do. Of course feel free
> >>>>> > >>>>> to
> >>>>> correct
> >>>>> > me
> >>>>> > >>>> on
> >>>>> > >>>>> this approach.
> >>>>> > >>>>>
> >>>>> > >>>>> I think it's OK if Wei merges his code into master and
> >>>>> > >>>>> then I
> >>>>> can
> >>>>> > draw
> >>>>> > >>>> from
> >>>>> > >>>>> the main repo and merge master into mine locally.
> >>>>> > >>>>>
> >>>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little
> >>>>> > >>>>> GUI
> >>>>> code
> >>>>> > to
> >>>>> > >>>> make
> >>>>> > >>>>> it user friendly (toggle between these features on the Add
> >>>>> > >>>>> Disk
> >>>>> > >> Offering
> >>>>> > >>>>> window).
> >>>>> > >>>>>
> >>>>> > >>>>> 2) I plan to write validation logic for the
> >>>>> create-disk-offering API
> >>>>> > >>>>> command which throws an exception if the rules are not
> >>>>> followed (this
> >>>>> > >>>>> should never be triggered from the GUI since the GUI will
> >>>>> > >>>>> have
> >>>>> > controls
> >>>>> > >>>> in
> >>>>> > >>>>> place to toggle between the one feature and the other).
> >>>>> > >>>>>
> >>>>> > >>>>> I'm not sure about documentation. I haven't had much
> >>>>> experience with
> >>>>> > it
> >>>>> > >>>> on
> >>>>> > >>>>> CloudStack projects yet.
> >>>>> > >>>>>
> >>>>> > >>>>> Thanks!
> >>>>> > >>>>>
> >>>>> > >>>>>
> >>>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>> > >>>> wrote:
> >>>>> > >>>>>
> >>>>> > >>>>>> Mike,
> >>>>> > >>>>>>
> >>>>> > >>>>>> Yes, these server-side rails need to be defined and
> >>>>> implemented
> >>>>> > before
> >>>>> > >>>>>> either patch can be merged. From my perspective, I would
> >>>>> like to
> >>>>> > see
> >>>>> > >>>> the
> >>>>> > >>>>>> rule implemented in the hypervisor as part of the
> >>>>> > >>>>>> validation
> >>>>> of the
> >>>>> > >>>> virtual
> >>>>> > >>>>>> machine definition. We also need to make sure that this
> >>>>> mutual
> >>>>> > >>>> exclusion
> >>>>> > >>>>>> is documented. Do we usually include this type of
> >>>>> documentation
> >>>>> > with
> >>>>> > >>>>>> patches of this nature?
> >>>>> > >>>>>>
> >>>>> > >>>>>> Thanks,
> >>>>> > >>>>>> -John
> >>>>> > >>>>>>
> >>>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
> >>>>> > >>>> mike.tutkowski@solidfire.com>
> >>>>> > >>>>>> wrote:
> >>>>> > >>>>>>
> >>>>> > >>>>>>> Currently they are not yet implemented.
> >>>>> > >>>>>>>
> >>>>> > >>>>>>> We have to make sure they are implemented in the GUI
> >>>>> > >>>>>>> from a
> >>>>> > usability
> >>>>> > >>>>>>> standpoint, but the API must check for consistency and
> >>>>> > >>>>>>> throw
> >>>>> an
> >>>>> > >>>> exception
> >>>>> > >>>>>>> if necessary.
> >>>>> > >>>>>>>
> >>>>> > >>>>>>>
> >>>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
> >>>>> jburwell@basho.com
> >>>>> > >
> >>>>> > >>>>>> wrote:
> >>>>> > >>>>>>>
> >>>>> > >>>>>>>> Mike,
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>> Are the checks only implemented in the UI?
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>> Thanks,
> >>>>> > >>>>>>>> -John
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> >>>>> > >>>>>>>> <mi...@solidfire.com> wrote:
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>>> Hi John,
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>> Wei and I have discussed making the two features
> >>>>> > >>>>>>>>> mutually
> >>>>> > >> exclusive.
> >>>>> > >>>> We
> >>>>> > >>>>>>>>> agree with you that only one should be active at a time.
> >>>>> We plan
> >>>>> > to
> >>>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio
> buttons)
> >>>>> > >>>>>>>>> to
> >>>>> turn
> >>>>> > his
> >>>>> > >>>>>>>> feature
> >>>>> > >>>>>>>>> on and mine off and vice versa.
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>> I was thinking if I wait until he checks in his code,
> >>>>> > >>>>>>>>> then
> >>>>> I
> >>>>> > update
> >>>>> > >>>> and
> >>>>> > >>>>>>>>> merge that I will be the person resolving merge
> >>>>> > >>>>>>>>> conflicts
> >>>>> in the
> >>>>> > >>>>>>>> JavaScript
> >>>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code)
> >>>>> > >>>>>>>>> as
> >>>>> opposed
> >>>>> > to
> >>>>> > >>>>>>>> putting
> >>>>> > >>>>>>>>> that work on someone else.
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>> Let me know what you think.
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>> Thanks!
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
> >>>>> > jburwell@basho.com
> >>>>> > >>>
> >>>>> > >>>>>>>> wrote:
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>>> Mike,
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>> How have Wei and you resolved the issue of
> >>>>> > >>>>>>>>>> conflicting QoS
> >>>>> > >>>> mechanisms
> >>>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the
> >>>>> affected
> >>>>> > FSs
> >>>>> > >>>> been
> >>>>> > >>>>>>>>>> updated with that decision?
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>> In terms of merge timing, can you describe the
> >>>>> dependencies
> >>>>> > >> between
> >>>>> > >>>>>> the
> >>>>> > >>>>>>>>>> patches?
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>> Thanks,
> >>>>> > >>>>>>>>>> -John
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> >>>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>> > >>>>>>>>>>
> >>>>> > >>>>>>>>>>> No problem, John.
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>> I still want to re-review it by myself before coming
> >>>>> > >>>>>>>>>>> up
> >>>>> with a
> >>>>> > >> new
> >>>>> > >>>>>>>> patch
> >>>>> > >>>>>>>>>>> file.
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to
> >>>>> > >>>>>>>>>>> be
> >>>>> checked
> >>>>> > >> in
> >>>>> > >>>>>> and
> >>>>> > >>>>>>>>>>> merge those into mine before generating a new
> patch file?
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>>>> > >> jburwell@basho.com
> >>>>> > >>>>>
> >>>>> > >>>>>>>>>> wrote:
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>>> Mike,
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review.
> >>>>> > >>>>>>>>>>>> I
> >>>>> am
> >>>>> > >> offline
> >>>>> > >>>>>> ATM,
> >>>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review
> Board?
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>> Sorry for the oversight, -John
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>> > >>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
> reading
> >>>>> this :)
> >>>>> > >> ),
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all
> the
> >>>>> areas
> >>>>> > we
> >>>>> > >>>>>> wanted
> >>>>> > >>>>>>>>>>>>> addressed.
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning
> or
> >>>>> > afternoon,
> >>>>> > >>>> then
> >>>>> > >>>>>>>>>> send
> >>>>> > >>>>>>>>>>>> in
> >>>>> > >>>>>>>>>>>>> another patch.
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski
> <
> >>>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>> Sure, that sounds good.
> >>>>> > >>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>> > >>>> ustcweizhou@gmail.com
> >>>>> > >>>>>>>
> >>>>> > >>>>>>>>>>>> wrote:
> >>>>> > >>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>> Hi Mike,
> >>>>> > >>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many
> conflicts
> >>>>> in Java
> >>>>> > >>>> code,
> >>>>> > >>>>>>>>>>>> except
> >>>>> > >>>>>>>>>>>>>>> the cloudstack UI.
> >>>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge
> disk_io_throttling
> >>>>> branch
> >>>>> > >> into
> >>>>> > >>>>>>>>>> master
> >>>>> > >>>>>>>>>>>>>>> this
> >>>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>> > >>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>> -Wei
> >>>>> > >>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>> > >>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>> Hey John,
> >>>>> > >>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>>> object_store
> >>>>> > >>>> branch,
> >>>>> > >>>>>>>> but
> >>>>> > >>>>>>>>>> -
> >>>>> > >>>>>>>>>>>> as
> >>>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we
> merge
> >>>>> the
> >>>>> > >>>> SolidFire
> >>>>> > >>>>>>>>>> branch
> >>>>> > >>>>>>>>>>>>>>> into
> >>>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store
> goes
> >>>>> into
> >>>>> > >> master.
> >>>>> > >>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits
> into
> >>>>> this
> >>>>> > merge
> >>>>> > >>>>>>>>>> strategy.
> >>>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>> > >>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell
> <
> >>>>> > >>>>>>>> jburwell@basho.com>
> >>>>> > >>>>>>>>>>>>>>> wrote:
> >>>>> > >>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>> Mike,
> >>>>> > >>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform.
> The
> >>>>> > >>>>>>>> disk_io_throttling,
> >>>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
> >>>>> number of
> >>>>> > >>>>>>>> overlapping
> >>>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies
> between
> >>>>> the
> >>>>> > >> patches
> >>>>> > >>>> to
> >>>>> > >>>>>>>> be
> >>>>> > >>>>>>>>>> as
> >>>>> > >>>>>>>>>>>>>>>>> follows:
> >>>>> > >>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>> > >>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management
> aspects of
> >>>>> > >> SolidFire
> >>>>> > >>>>>> are
> >>>>> > >>>>>>>>>>>>>>> additive
> >>>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are
> circular
> >>>>> > dependency
> >>>>> > >>>>>>>> between
> >>>>> > >>>>>>>>>>>>>>> the
> >>>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the
> dependency
> >>>>> graph, we
> >>>>> > can
> >>>>> > >>>>>>>>>> determine
> >>>>> > >>>>>>>>>>>>>>> the
> >>>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>> > >>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>> Thanks,
> >>>>> > >>>>>>>>>>>>>>>>> -John
> >>>>> > >>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski
> <
> >>>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>> > >>>>>>>>>>>>>>>>> wrote:
> >>>>> > >>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my
> code
> >>>>> into
> >>>>> > his
> >>>>> > >>>>>> branch
> >>>>> > >>>>>>>>>>>>>>> before
> >>>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>> > >>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>> We can remove the
> StoragePoolType.Dynamic code
> >>>>> after his
> >>>>> > >>>> merge
> >>>>> > >>>>>>>> and
> >>>>> > >>>>>>>>>>>>>>> we
> >>>>> > >>>>>>>>>>>>>>>> can
> >>>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>> > >>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
> Tutkowski <
> >>>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>> > >>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're
> going here:
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to
> hypervisor
> >>>>> code in
> >>>>> > >> the
> >>>>> > >>>>>>>>>> storage
> >>>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
> storage
> >>>>> > plug-in,
> >>>>> > >>>>>> which
> >>>>> > >>>>>>>>>>>>>>>>> currently
> >>>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor
> in use
> >>>>> > (although
> >>>>> > >>>> it
> >>>>> > >>>>>>>> does
> >>>>> > >>>>>>>>>>>>>>> not
> >>>>> > >>>>>>>>>>>>>>>>> know
> >>>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.)
> is
> >>>>> actually
> >>>>> > in
> >>>>> > >>>>>> use))
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can
> be placed
> >>>>> in the
> >>>>> > url
> >>>>> > >>>>>> field
> >>>>> > >>>>>>>>>> (if
> >>>>> > >>>>>>>>>>>>>>>> not
> >>>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is
> >>>>> stored in
> >>>>> > the
> >>>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to
> the
> >>>>> hypervisor
> >>>>> > in
> >>>>> > >>>>>>>>>>>>>>> question, we
> >>>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this
> takes the
> >>>>> place
> >>>>> > of
> >>>>> > >>>> the
> >>>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in
> the hypervisor
> >>>>> > checks
> >>>>> > >>>> for
> >>>>> > >>>>>>>> the
> >>>>> > >>>>>>>>>>>>>>>> managed
> >>>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the
> necessary
> >>>>> > hypervisor
> >>>>> > >>>>>> data
> >>>>> > >>>>>>>>>>>>>>>>> structure is
> >>>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach
> command
> >>>>> executes to
> >>>>> > >>>> attach
> >>>>> > >>>>>>>> the
> >>>>> > >>>>>>>>>>>>>>>>> volume.
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand)
> is invoked
> >>>>> to
> >>>>> > >> detach a
> >>>>> > >>>>>>>>>> volume,
> >>>>> > >>>>>>>>>>>>>>>> the
> >>>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the
> hypervisor
> >>>>> data
> >>>>> > >>>> structure
> >>>>> > >>>>>>>> is
> >>>>> > >>>>>>>>>>>>>>>>> removed.
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support
> Burst
> >>>>> IOPS in
> >>>>> > 4.2
> >>>>> > >>>>>>>> unless
> >>>>> > >>>>>>>>>>>>>>> it is
> >>>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
> table.
> >>>>> If we
> >>>>> > >> have
> >>>>> > >>>>>> some
> >>>>> > >>>>>>>>>>>>>>> idea,
> >>>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> Thanks!
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
> Tutkowski <
> >>>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented
> while
> >>>>> avoiding
> >>>>> > >>>>>>>>>> implementation
> >>>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
> details
> >>>>> > field.
> >>>>> > >> I
> >>>>> > >>>>>>>> think
> >>>>> > >>>>>>>>>>>>>>> we
> >>>>> > >>>>>>>>>>>>>>>>> should
> >>>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the
> documentation
> >>>>> regarding
> >>>>> > >> the
> >>>>> > >>>>>>>>>>>>>>> expected
> >>>>> > >>>>>>>>>>>>>>>>> format
> >>>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
> >>>>> details are
> >>>>> > >> not
> >>>>> > >>>>>>>>>>>>>>> returned on
> >>>>> > >>>>>>>>>>>>>>>>> the
> >>>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >>>>> > listStoragePool
> >>>>> > >>>>>>>>>> response.
> >>>>> > >>>>>>>>>>>>>>>> Why
> >>>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would
> be
> >>>>> useful
> >>>>> > >> for
> >>>>> > >>>>>>>>>> clients
> >>>>> > >>>>>>>>>>>>>>> to
> >>>>> > >>>>>>>>>>>>>>>> be
> >>>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
> >>>>> field."
> >>>>> > >>>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing
> Burst IOPS
> >>>>> here.
> >>>>> > >>>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>>>> > >> Offering-by-Disk
> >>>>> > >>>>>>>>>> Offering
> >>>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
> have
> >>>>> to be
> >>>>> > >> able
> >>>>> > >>>>>> to
> >>>>> > >>>>>>>>>>>>>>>> associate
> >>>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>>> disk_offering_details
> >>>>> > >> table.
> >>>>> > >>>>>>>> Maybe
> >>>>> > >>>>>>>>>>>>>>> it
> >>>>> > >>>>>>>>>>>>>>>>> could
> >>>>> > >>>>>>>>>>>>>>>>>>>> go there?
> >>>>> > >>>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept
> the
> >>>>> Burst IOPS
> >>>>> > in
> >>>>> > >>>> the
> >>>>> > >>>>>>>> GUI
> >>>>> > >>>>>>>>>>>>>>> if
> >>>>> > >>>>>>>>>>>>>>>>> it's
> >>>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields
> are in
> >>>>> the DB.
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>> --
> >>>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>> > >>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> >>>>> >
> >>>>> > >>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>> > >>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>>> --
> >>>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> > >>>>>>>>>>>>>>>>>> cloud<
> >>>>> > http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>>>>>>>>>>>>>>>>> *(tm)*
> >>>>> > >>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>>> --
> >>>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> > >>>>>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>>>>>>>>>>>>>>> *(tm)*
> >>>>> > >>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>> --
> >>>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>>>>>>>>> o: 303.746.7302
> >>>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> > >>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>>>>>>>>>>>>> *(tm)*
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>>
> >>>>> > >>>>>>>>>>>>> --
> >>>>> > >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>>>>>>>> o: 303.746.7302
> >>>>> > >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> > >>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>>>>>>>>>>>> *(tm)*
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>>
> >>>>> > >>>>>>>>>>> --
> >>>>> > >>>>>>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>>>>>> o: 303.746.7302
> >>>>> > >>>>>>>>>>> Advancing the way the world uses the
> >>>>> > >>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>> >
> >>>>> > >>>>>>>>>>> *(tm)*
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>>
> >>>>> > >>>>>>>>> --
> >>>>> > >>>>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>>>> o: 303.746.7302
> >>>>> > >>>>>>>>> Advancing the way the world uses the
> >>>>> > >>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>>>>>>>> *(tm)*
> >>>>> > >>>>>>>>
> >>>>> > >>>>>>>
> >>>>> > >>>>>>>
> >>>>> > >>>>>>>
> >>>>> > >>>>>>> --
> >>>>> > >>>>>>> *Mike Tutkowski*
> >>>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>>>> o: 303.746.7302
> >>>>> > >>>>>>> Advancing the way the world uses the
> >>>>> > >>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>>>>>> *(tm)*
> >>>>> > >>>>>>
> >>>>> > >>>>>>
> >>>>> > >>>>>
> >>>>> > >>>>>
> >>>>> > >>>>> --
> >>>>> > >>>>> *Mike Tutkowski*
> >>>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>>>> e: mike.tutkowski@solidfire.com
> >>>>> > >>>>> o: 303.746.7302
> >>>>> > >>>>> Advancing the way the world uses the
> >>>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>>>> *(tm)*
> >>>>> > >>>>
> >>>>> > >>>>
> >>>>> > >>>
> >>>>> > >>>
> >>>>> > >>> --
> >>>>> > >>> *Mike Tutkowski*
> >>>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > >>> e: mike.tutkowski@solidfire.com
> >>>>> > >>> o: 303.746.7302
> >>>>> > >>> Advancing the way the world uses the
> >>>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> > >>> *(tm)*
> >>>>> > >>
> >>>>> > >>
> >>>>> > >
> >>>>> > >
> >>>>> > > --
> >>>>> > > *Mike Tutkowski*
> >>>>> > > *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > > e: mike.tutkowski@solidfire.com
> >>>>> > > o: 303.746.7302
> >>>>> > > Advancing the way the world uses the
> >>>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> > > *(tm)*
> >>>>> >
> >>>>> >
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *(tm)*
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*
> >>
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
We'd also have to recognize that if the Min value is filled in and Storage
QoS is not available that Hypervisor QoS (rate limiting here) cannot
satisfy that constraint.
On Wed, Jun 12, 2013 at 4:26 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Yeah, this is interesting.
>
> We'll have to wait and see what Wei's thoughts are on this.
>
>
> On Wed, Jun 12, 2013 at 4:17 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Yes. To your point, the appropriate logic would be to check the Volume
>> allocated by the StorageAllocator. If it doesn't support a QoS, then the
>> VM allocator would attempt to fulfill the QoS through the hypervisor.
>> Another question would be -- what would be in the behavior in the event
>> that no resources are available to fulfill the QoS? We could outright fail
>> or proceed with some kind of warning -- sounds like a good place for a
>> global setting to configure the policy.
>>
>> The other question we haven't answered are usage records. By pushing the
>> QoS concept out to allocation and making it a more general concept, it
>> could make usage record capture more straightforward.
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 6:11 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> I see, John.
>>
>> That is an interesting idea.
>>
>> We'd also have to change the storage allocator(s) to favor QoS-supported
>> storage if those fields are filled in.
>>
>>
>> On Wed, Jun 12, 2013 at 4:09 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> My thought is that we present the min/max IOPS fields for read/write
>>> operations for all compute and disk offerings. When the VM is allocated,
>>> we determine the best way to fulfill that QoS. It sounds like storage
>>> level guarantees would always be preferred. If no storage is available to
>>> guarantee the QoS, the allocator falls back to using hypervisor. This
>>> approach only works if summing the discrete read/write min/max values to
>>> get to min/max total IOPS would be acceptable.
>>>
>>> Thanks,
>>> -John
>>>
>>>
>>> On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> I hate to say it, but I believe Storage QoS with a Min and Max will
>>> always be optimal over hypervisor rate limiting.
>>>
>>> The only time you'd want to use hypervisor rate limiting is if storage
>>> QoS is not available.
>>>
>>> We currently have no way to know what storage the root or data disk will
>>> be deployed to, so we have to present the fields in all situation.
>>>
>>> This is because of - in my opinion - the somewhat flawed way CloudStack
>>> handles storage tags. You are free to enter in any text there and we don't
>>> know what it maps to when the Compute or Disk Offering dialog is up.
>>>
>>>
>>> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com>wrote:
>>>
>>>> Mike,
>>>>
>>>> Please see my comments/questions in-line below.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>> Hi Wei,
>>>>
>>>> So, not entirely sure I follow.
>>>>
>>>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>>>
>>>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>>>> QoS for the purpose of this e-mail.
>>>>
>>>> By default, neither radio button is selected and no QoS fields are
>>>> visible.
>>>>
>>>>
>>>> I prefer a None option selected by default. I find the no button
>>>> selected in a Radio button group to be confusing.
>>>>
>>>> Also, do we have a mechanism to determine whether or not this radio
>>>> button group should be displayed at all? If there are no devices or
>>>> hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>>>>
>>>>
>>>> If the user selects the Storage QoS radio button, then the Custom IOPS
>>>> checkbox and the Min and Max text fields are made visible.
>>>>
>>>> If the user changes his mind and selects the Hypervisor QoS radio
>>>> button, then the Custom IOPS checkbox and the Min and Max text fields
>>>> disappear and are replaced by the two Hypervisor QoS text fields.
>>>>
>>>> This way, the user can choose neither QoS option or one of them or the
>>>> other, but never both.
>>>>
>>>> On the API side, I was thinking of having logic in place when a request
>>>> comes in to create a Disk Offering to confirm these fields are the way we
>>>> want them.
>>>>
>>>> Once we know the Disk Offering is in order, a user can create a data
>>>> disk from it. Since we checked the validity of the Disk Offering when it
>>>> was created, the VM should never be asked to use Hypervisor QoS when
>>>> Storage QoS in being utilized.
>>>>
>>>>
>>>> Overall, this model sounds very reasonable. Wondering aloud: Would be
>>>> possible to have the UI simply ask for a storage QoS, and let the system
>>>> pick the optional implementation. For example, if a VM is attached to KVM
>>>> and SolidFire is available, we chose to fulfill the QoS using the storage
>>>> device. This approach would also allow us to fallback in the event that we
>>>> can't allocate storage for the QoS, but we have hypervisor capacity. Could
>>>> we always deterministically determine the optimal implementation of the QoS
>>>> at time of VM deployment?
>>>>
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com>wrote:
>>>>
>>>>> Mike, John
>>>>>
>>>>> The Disk I/O Throttling works like:
>>>>>
>>>>> (1) For User VM,
>>>>> (1.1) root disks: service offering -> default value in global
>>>>> configuration
>>>>> (1.2) data disks: disk offering -> default value in global
>>>>> configuration
>>>>>
>>>>> (2) For System VM
>>>>> root disks: service offering
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>> 2013/6/12 John Burwell <jb...@basho.com>
>>>>>
>>>>> > Mike,
>>>>> >
>>>>> > That is one possibility. The other possibility is that hypervisor is
>>>>> > going to throttle I/O on all disks attached. Therefore, we need
>>>>> answers to
>>>>> > the following questions:
>>>>> >
>>>>> > 1. Is I/O throttling applied to the root disk or all disks
>>>>> > attached to the VM?
>>>>> > 2. If I/O throttling is applied to all disks, how is the
>>>>> > throttling distributed amongst the disks if only one read/write
>>>>> value is
>>>>> > defined?
>>>>> >
>>>>> > Thanks,
>>>>> > -John
>>>>> >
>>>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>> > wrote:
>>>>> >
>>>>> > > Hey John,
>>>>> > >
>>>>> > > Perhaps I don't fully understand how Wei's feature works.
>>>>> > >
>>>>> > > I guess I thought if you choose Hypervisor QoS, you do so on
>>>>> Compute
>>>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>>>>> > >
>>>>> > > From my thinking, you're root disk could be under Hypervisor QoS,
>>>>> but
>>>>> > your
>>>>> > > data disk could be under Storage QoS.
>>>>> > >
>>>>> > > Is that incorrect?
>>>>> > >
>>>>> > > Thanks
>>>>> > >
>>>>> > >
>>>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>>>>> > wrote:
>>>>> > >
>>>>> > >> Mike,
>>>>> > >>
>>>>> > >> As I understand these two patches, the throttled I?O settings are
>>>>> > applied
>>>>> > >> from the hypervisor side, and possibly defined in a compute
>>>>> offering
>>>>> > where
>>>>> > >> provisioned IOPS are defined on the storage side through disk
>>>>> > offerings. I
>>>>> > >> don't see how the management server could enforce this mutual
>>>>> exclusion
>>>>> > >> between provisioned IOPS and throttled I/O until a compute and
>>>>> disk
>>>>> > >> offering were selected. These selections come together when we
>>>>> deploy a
>>>>> > >> VM. Based on these assumptions, I would expect to see
>>>>> enforcement of
>>>>> > this
>>>>> > >> rule in UI at the time of VM creation/definition and on the server
>>>>> > side, as
>>>>> > >> part of the VM creation. It feels like any attempt to enforce
>>>>> this rule
>>>>> > >> when defining offering would be premature, and unnecessarily limit
>>>>> > >> flexibility. Are my assumptions and understanding correct?
>>>>> > >>
>>>>> > >> Thanks,
>>>>> > >> -John
>>>>> > >>
>>>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>>>>> > mike.tutkowski@solidfire.com>
>>>>> > >> wrote:
>>>>> > >>
>>>>> > >>> Hi John,
>>>>> > >>>
>>>>> > >>> So, maybe I'm wrong about this, but what I was thinking is that
>>>>> we'd
>>>>> > >> build
>>>>> > >>> two radio buttons into the Add Disk Offering dialog (we can
>>>>> ignore
>>>>> > >> Compute
>>>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>>>>> > >>>
>>>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
>>>>> Storage
>>>>> > >> QoS
>>>>> > >>> for the purpose of this e-mail.
>>>>> > >>>
>>>>> > >>> By default, neither radio button is selected and no QoS fields
>>>>> are
>>>>> > >> visible.
>>>>> > >>>
>>>>> > >>> If the user selects the Storage QoS radio button, then the
>>>>> Custom IOPS
>>>>> > >>> checkbox and the Min and Max text fields are made visible.
>>>>> > >>>
>>>>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>>>>> > button,
>>>>> > >>> then the Custom IOPS checkbox and the Min and Max text fields
>>>>> disappear
>>>>> > >> and
>>>>> > >>> are replaced by the two Hypervisor QoS text fields.
>>>>> > >>>
>>>>> > >>> This way, the user can choose neither QoS option or one of them
>>>>> or the
>>>>> > >>> other, but never both.
>>>>> > >>>
>>>>> > >>> On the API side, I was thinking of having logic in place when a
>>>>> request
>>>>> > >>> comes in to create a Disk Offering to confirm these fields are
>>>>> the way
>>>>> > we
>>>>> > >>> want them.
>>>>> > >>>
>>>>> > >>> Once we know the Disk Offering is in order, a user can create a
>>>>> data
>>>>> > disk
>>>>> > >>> from it. Since we checked the validity of the Disk Offering when
>>>>> it was
>>>>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>>>>> > Storage
>>>>> > >>> QoS in being utilized.
>>>>> > >>>
>>>>> > >>> Does that make sense or did I miss something?
>>>>> > >>>
>>>>> > >>> Thanks
>>>>> > >>>
>>>>> > >>>
>>>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>> > >> wrote:
>>>>> > >>>
>>>>> > >>>> Mike,
>>>>> > >>>>
>>>>> > >>>> Looking through the code, I am trying to understand how
>>>>> > >>>> CreateDiskOfferingCmd would have the context to identify the
>>>>> conflict.
>>>>> > >>>> Naively, it seems to me that this rule would need to be
>>>>> enforced when
>>>>> > a
>>>>> > >>>> virtual machine is being deployed. Looking through the code,
>>>>> it seems
>>>>> > >> like
>>>>> > >>>> we should add a private validateStorageQoS method to
>>>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and
>>>>> throws a
>>>>> > >>>> ResourceAllocationException when the QoS definitions are
>>>>> inconsistent.
>>>>> > >> We
>>>>> > >>>> would then add calls to it from each of the VM creation methods
>>>>> in the
>>>>> > >>>> service. Do this type of approach sound reasonable?
>>>>> > >>>>
>>>>> > >>>> Thanks,
>>>>> > >>>> -John
>>>>> > >>>>
>>>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>>>>> > >> mike.tutkowski@solidfire.com>
>>>>> > >>>> wrote:
>>>>> > >>>>
>>>>> > >>>>> Hi John,
>>>>> > >>>>>
>>>>> > >>>>> So, here's what I was planning to do. Of course feel free to
>>>>> correct
>>>>> > me
>>>>> > >>>> on
>>>>> > >>>>> this approach.
>>>>> > >>>>>
>>>>> > >>>>> I think it's OK if Wei merges his code into master and then I
>>>>> can
>>>>> > draw
>>>>> > >>>> from
>>>>> > >>>>> the main repo and merge master into mine locally.
>>>>> > >>>>>
>>>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI
>>>>> code
>>>>> > to
>>>>> > >>>> make
>>>>> > >>>>> it user friendly (toggle between these features on the Add Disk
>>>>> > >> Offering
>>>>> > >>>>> window).
>>>>> > >>>>>
>>>>> > >>>>> 2) I plan to write validation logic for the
>>>>> create-disk-offering API
>>>>> > >>>>> command which throws an exception if the rules are not
>>>>> followed (this
>>>>> > >>>>> should never be triggered from the GUI since the GUI will have
>>>>> > controls
>>>>> > >>>> in
>>>>> > >>>>> place to toggle between the one feature and the other).
>>>>> > >>>>>
>>>>> > >>>>> I'm not sure about documentation. I haven't had much
>>>>> experience with
>>>>> > it
>>>>> > >>>> on
>>>>> > >>>>> CloudStack projects yet.
>>>>> > >>>>>
>>>>> > >>>>> Thanks!
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>> > >>>> wrote:
>>>>> > >>>>>
>>>>> > >>>>>> Mike,
>>>>> > >>>>>>
>>>>> > >>>>>> Yes, these server-side rails need to be defined and
>>>>> implemented
>>>>> > before
>>>>> > >>>>>> either patch can be merged. From my perspective, I would
>>>>> like to
>>>>> > see
>>>>> > >>>> the
>>>>> > >>>>>> rule implemented in the hypervisor as part of the validation
>>>>> of the
>>>>> > >>>> virtual
>>>>> > >>>>>> machine definition. We also need to make sure that this
>>>>> mutual
>>>>> > >>>> exclusion
>>>>> > >>>>>> is documented. Do we usually include this type of
>>>>> documentation
>>>>> > with
>>>>> > >>>>>> patches of this nature?
>>>>> > >>>>>>
>>>>> > >>>>>> Thanks,
>>>>> > >>>>>> -John
>>>>> > >>>>>>
>>>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>>>> > >>>> mike.tutkowski@solidfire.com>
>>>>> > >>>>>> wrote:
>>>>> > >>>>>>
>>>>> > >>>>>>> Currently they are not yet implemented.
>>>>> > >>>>>>>
>>>>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>>>>> > usability
>>>>> > >>>>>>> standpoint, but the API must check for consistency and throw
>>>>> an
>>>>> > >>>> exception
>>>>> > >>>>>>> if necessary.
>>>>> > >>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
>>>>> jburwell@basho.com
>>>>> > >
>>>>> > >>>>>> wrote:
>>>>> > >>>>>>>
>>>>> > >>>>>>>> Mike,
>>>>> > >>>>>>>>
>>>>> > >>>>>>>> Are the checks only implemented in the UI?
>>>>> > >>>>>>>>
>>>>> > >>>>>>>> Thanks,
>>>>> > >>>>>>>> -John
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>
>>>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>>>> > >>>>>>>> <mi...@solidfire.com> wrote:
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>> Hi John,
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>>>>> > >> exclusive.
>>>>> > >>>> We
>>>>> > >>>>>>>>> agree with you that only one should be active at a time.
>>>>> We plan
>>>>> > to
>>>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to
>>>>> turn
>>>>> > his
>>>>> > >>>>>>>> feature
>>>>> > >>>>>>>>> on and mine off and vice versa.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then
>>>>> I
>>>>> > update
>>>>> > >>>> and
>>>>> > >>>>>>>>> merge that I will be the person resolving merge conflicts
>>>>> in the
>>>>> > >>>>>>>> JavaScript
>>>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as
>>>>> opposed
>>>>> > to
>>>>> > >>>>>>>> putting
>>>>> > >>>>>>>>> that work on someone else.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Let me know what you think.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Thanks!
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>>>>> > jburwell@basho.com
>>>>> > >>>
>>>>> > >>>>>>>> wrote:
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>> Mike,
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>>>> > >>>> mechanisms
>>>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the
>>>>> affected
>>>>> > FSs
>>>>> > >>>> been
>>>>> > >>>>>>>>>> updated with that decision?
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> In terms of merge timing, can you describe the
>>>>> dependencies
>>>>> > >> between
>>>>> > >>>>>> the
>>>>> > >>>>>>>>>> patches?
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> Thanks,
>>>>> > >>>>>>>>>> -John
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>> No problem, John.
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up
>>>>> with a
>>>>> > >> new
>>>>> > >>>>>>>> patch
>>>>> > >>>>>>>>>>> file.
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>>> checked
>>>>> > >> in
>>>>> > >>>>>> and
>>>>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>> > >> jburwell@basho.com
>>>>> > >>>>>
>>>>> > >>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>> Mike,
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I
>>>>> am
>>>>> > >> offline
>>>>> > >>>>>> ATM,
>>>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> Sorry for the oversight,
>>>>> > >>>>>>>>>>>> -John
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>>> this :)
>>>>> > >> ),
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>>> areas
>>>>> > we
>>>>> > >>>>>> wanted
>>>>> > >>>>>>>>>>>>> addressed.
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>> > afternoon,
>>>>> > >>>> then
>>>>> > >>>>>>>>>> send
>>>>> > >>>>>>>>>>>> in
>>>>> > >>>>>>>>>>>>> another patch.
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>> Sure, that sounds good.
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>> > >>>> ustcweizhou@gmail.com
>>>>> > >>>>>>>
>>>>> > >>>>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> Hi Mike,
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
>>>>> in Java
>>>>> > >>>> code,
>>>>> > >>>>>>>>>>>> except
>>>>> > >>>>>>>>>>>>>>> the cloudstack UI.
>>>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>>> branch
>>>>> > >> into
>>>>> > >>>>>>>>>> master
>>>>> > >>>>>>>>>>>>>>> this
>>>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> -Wei
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> Hey John,
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>> object_store
>>>>> > >>>> branch,
>>>>> > >>>>>>>> but
>>>>> > >>>>>>>>>> -
>>>>> > >>>>>>>>>>>> as
>>>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
>>>>> the
>>>>> > >>>> SolidFire
>>>>> > >>>>>>>>>> branch
>>>>> > >>>>>>>>>>>>>>> into
>>>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes
>>>>> into
>>>>> > >> master.
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>>>>> this
>>>>> > merge
>>>>> > >>>>>>>>>> strategy.
>>>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>> > >>>>>>>> jburwell@basho.com>
>>>>> > >>>>>>>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> Mike,
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>> > >>>>>>>> disk_io_throttling,
>>>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
>>>>> number of
>>>>> > >>>>>>>> overlapping
>>>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between
>>>>> the
>>>>> > >> patches
>>>>> > >>>> to
>>>>> > >>>>>>>> be
>>>>> > >>>>>>>>>> as
>>>>> > >>>>>>>>>>>>>>>>> follows:
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>>> > >> SolidFire
>>>>> > >>>>>> are
>>>>> > >>>>>>>>>>>>>>> additive
>>>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>>> > dependency
>>>>> > >>>>>>>> between
>>>>> > >>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency
>>>>> graph, we
>>>>> > can
>>>>> > >>>>>>>>>> determine
>>>>> > >>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> Thanks,
>>>>> > >>>>>>>>>>>>>>>>> -John
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>> > >>>>>>>>>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>>> into
>>>>> > his
>>>>> > >>>>>> branch
>>>>> > >>>>>>>>>>>>>>> before
>>>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>>> after his
>>>>> > >>>> merge
>>>>> > >>>>>>>> and
>>>>> > >>>>>>>>>>>>>>> we
>>>>> > >>>>>>>>>>>>>>>> can
>>>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>>> code in
>>>>> > >> the
>>>>> > >>>>>>>>>> storage
>>>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>>> > plug-in,
>>>>> > >>>>>> which
>>>>> > >>>>>>>>>>>>>>>>> currently
>>>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>>> > (although
>>>>> > >>>> it
>>>>> > >>>>>>>> does
>>>>> > >>>>>>>>>>>>>>> not
>>>>> > >>>>>>>>>>>>>>>>> know
>>>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>>> actually
>>>>> > in
>>>>> > >>>>>> use))
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed
>>>>> in the
>>>>> > url
>>>>> > >>>>>> field
>>>>> > >>>>>>>>>> (if
>>>>> > >>>>>>>>>>>>>>>> not
>>>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>>>>> stored in
>>>>> > the
>>>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>>> hypervisor
>>>>> > in
>>>>> > >>>>>>>>>>>>>>> question, we
>>>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>>> place
>>>>> > of
>>>>> > >>>> the
>>>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>>> > checks
>>>>> > >>>> for
>>>>> > >>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>> managed
>>>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>>> > hypervisor
>>>>> > >>>>>> data
>>>>> > >>>>>>>>>>>>>>>>> structure is
>>>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>>> executes to
>>>>> > >>>> attach
>>>>> > >>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>> volume.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
>>>>> to
>>>>> > >> detach a
>>>>> > >>>>>>>>>> volume,
>>>>> > >>>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>>>>> data
>>>>> > >>>> structure
>>>>> > >>>>>>>> is
>>>>> > >>>>>>>>>>>>>>>>> removed.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>>>>> IOPS in
>>>>> > 4.2
>>>>> > >>>>>>>> unless
>>>>> > >>>>>>>>>>>>>>> it is
>>>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
>>>>> If we
>>>>> > >> have
>>>>> > >>>>>> some
>>>>> > >>>>>>>>>>>>>>> idea,
>>>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> Thanks!
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>>> avoiding
>>>>> > >>>>>>>>>> implementation
>>>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>>> > field.
>>>>> > >> I
>>>>> > >>>>>>>> think
>>>>> > >>>>>>>>>>>>>>> we
>>>>> > >>>>>>>>>>>>>>>>> should
>>>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>>> regarding
>>>>> > >> the
>>>>> > >>>>>>>>>>>>>>> expected
>>>>> > >>>>>>>>>>>>>>>>> format
>>>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>>>>> details are
>>>>> > >> not
>>>>> > >>>>>>>>>>>>>>> returned on
>>>>> > >>>>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>> > listStoragePool
>>>>> > >>>>>>>>>> response.
>>>>> > >>>>>>>>>>>>>>>> Why
>>>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>>> useful
>>>>> > >> for
>>>>> > >>>>>>>>>> clients
>>>>> > >>>>>>>>>>>>>>> to
>>>>> > >>>>>>>>>>>>>>>> be
>>>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>>> field."
>>>>> > >>>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>>> here.
>>>>> > >>>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>> > >> Offering-by-Disk
>>>>> > >>>>>>>>>> Offering
>>>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>>> to be
>>>>> > >> able
>>>>> > >>>>>> to
>>>>> > >>>>>>>>>>>>>>>> associate
>>>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>> disk_offering_details
>>>>> > >> table.
>>>>> > >>>>>>>> Maybe
>>>>> > >>>>>>>>>>>>>>> it
>>>>> > >>>>>>>>>>>>>>>>> could
>>>>> > >>>>>>>>>>>>>>>>>>>> go there?
>>>>> > >>>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
>>>>> Burst IOPS
>>>>> > in
>>>>> > >>>> the
>>>>> > >>>>>>>> GUI
>>>>> > >>>>>>>>>>>>>>> if
>>>>> > >>>>>>>>>>>>>>>>> it's
>>>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>>>>> the DB.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>> >
>>>>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>>>>>>>>> cloud<
>>>>> > http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> --
>>>>> > >>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>> >
>>>>> > >>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> --
>>>>> > >>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>> *â„¢*
>>>>> > >>>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>> --
>>>>> > >>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>> o: 303.746.7302
>>>>> > >>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>> *â„¢*
>>>>> > >>>>>>
>>>>> > >>>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> --
>>>>> > >>>>> *Mike Tutkowski*
>>>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>> o: 303.746.7302
>>>>> > >>>>> Advancing the way the world uses the
>>>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>> *â„¢*
>>>>> > >>>>
>>>>> > >>>>
>>>>> > >>>
>>>>> > >>>
>>>>> > >>> --
>>>>> > >>> *Mike Tutkowski*
>>>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>> e: mike.tutkowski@solidfire.com
>>>>> > >>> o: 303.746.7302
>>>>> > >>> Advancing the way the world uses the
>>>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>> *â„¢*
>>>>> > >>
>>>>> > >>
>>>>> > >
>>>>> > >
>>>>> > > --
>>>>> > > *Mike Tutkowski*
>>>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > > e: mike.tutkowski@solidfire.com
>>>>> > > o: 303.746.7302
>>>>> > > Advancing the way the world uses the
>>>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > > *â„¢*
>>>>> >
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, this is interesting.
We'll have to wait and see what Wei's thoughts are on this.
On Wed, Jun 12, 2013 at 4:17 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Yes. To your point, the appropriate logic would be to check the Volume
> allocated by the StorageAllocator. If it doesn't support a QoS, then the
> VM allocator would attempt to fulfill the QoS through the hypervisor.
> Another question would be -- what would be in the behavior in the event
> that no resources are available to fulfill the QoS? We could outright fail
> or proceed with some kind of warning -- sounds like a good place for a
> global setting to configure the policy.
>
> The other question we haven't answered are usage records. By pushing the
> QoS concept out to allocation and making it a more general concept, it
> could make usage record capture more straightforward.
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 6:11 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> I see, John.
>
> That is an interesting idea.
>
> We'd also have to change the storage allocator(s) to favor QoS-supported
> storage if those fields are filled in.
>
>
> On Wed, Jun 12, 2013 at 4:09 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> My thought is that we present the min/max IOPS fields for read/write
>> operations for all compute and disk offerings. When the VM is allocated,
>> we determine the best way to fulfill that QoS. It sounds like storage
>> level guarantees would always be preferred. If no storage is available to
>> guarantee the QoS, the allocator falls back to using hypervisor. This
>> approach only works if summing the discrete read/write min/max values to
>> get to min/max total IOPS would be acceptable.
>>
>> Thanks,
>> -John
>>
>>
>> On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> I hate to say it, but I believe Storage QoS with a Min and Max will
>> always be optimal over hypervisor rate limiting.
>>
>> The only time you'd want to use hypervisor rate limiting is if storage
>> QoS is not available.
>>
>> We currently have no way to know what storage the root or data disk will
>> be deployed to, so we have to present the fields in all situation.
>>
>> This is because of - in my opinion - the somewhat flawed way CloudStack
>> handles storage tags. You are free to enter in any text there and we don't
>> know what it maps to when the Compute or Disk Offering dialog is up.
>>
>>
>> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> Please see my comments/questions in-line below.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> Hi Wei,
>>>
>>> So, not entirely sure I follow.
>>>
>>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>>
>>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>>> QoS for the purpose of this e-mail.
>>>
>>> By default, neither radio button is selected and no QoS fields are
>>> visible.
>>>
>>>
>>> I prefer a None option selected by default. I find the no button
>>> selected in a Radio button group to be confusing.
>>>
>>> Also, do we have a mechanism to determine whether or not this radio
>>> button group should be displayed at all? If there are no devices or
>>> hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>>>
>>>
>>> If the user selects the Storage QoS radio button, then the Custom IOPS
>>> checkbox and the Min and Max text fields are made visible.
>>>
>>> If the user changes his mind and selects the Hypervisor QoS radio
>>> button, then the Custom IOPS checkbox and the Min and Max text fields
>>> disappear and are replaced by the two Hypervisor QoS text fields.
>>>
>>> This way, the user can choose neither QoS option or one of them or the
>>> other, but never both.
>>>
>>> On the API side, I was thinking of having logic in place when a request
>>> comes in to create a Disk Offering to confirm these fields are the way we
>>> want them.
>>>
>>> Once we know the Disk Offering is in order, a user can create a data
>>> disk from it. Since we checked the validity of the Disk Offering when it
>>> was created, the VM should never be asked to use Hypervisor QoS when
>>> Storage QoS in being utilized.
>>>
>>>
>>> Overall, this model sounds very reasonable. Wondering aloud: Would be
>>> possible to have the UI simply ask for a storage QoS, and let the system
>>> pick the optional implementation. For example, if a VM is attached to KVM
>>> and SolidFire is available, we chose to fulfill the QoS using the storage
>>> device. This approach would also allow us to fallback in the event that we
>>> can't allocate storage for the QoS, but we have hypervisor capacity. Could
>>> we always deterministically determine the optimal implementation of the QoS
>>> at time of VM deployment?
>>>
>>>
>>> Thanks!
>>>
>>>
>>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
>>>
>>>> Mike, John
>>>>
>>>> The Disk I/O Throttling works like:
>>>>
>>>> (1) For User VM,
>>>> (1.1) root disks: service offering -> default value in global
>>>> configuration
>>>> (1.2) data disks: disk offering -> default value in global configuration
>>>>
>>>> (2) For System VM
>>>> root disks: service offering
>>>>
>>>> -Wei
>>>>
>>>>
>>>> 2013/6/12 John Burwell <jb...@basho.com>
>>>>
>>>> > Mike,
>>>> >
>>>> > That is one possibility. The other possibility is that hypervisor is
>>>> > going to throttle I/O on all disks attached. Therefore, we need
>>>> answers to
>>>> > the following questions:
>>>> >
>>>> > 1. Is I/O throttling applied to the root disk or all disks
>>>> > attached to the VM?
>>>> > 2. If I/O throttling is applied to all disks, how is the
>>>> > throttling distributed amongst the disks if only one read/write value
>>>> is
>>>> > defined?
>>>> >
>>>> > Thanks,
>>>> > -John
>>>> >
>>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>> > wrote:
>>>> >
>>>> > > Hey John,
>>>> > >
>>>> > > Perhaps I don't fully understand how Wei's feature works.
>>>> > >
>>>> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
>>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>>>> > >
>>>> > > From my thinking, you're root disk could be under Hypervisor QoS,
>>>> but
>>>> > your
>>>> > > data disk could be under Storage QoS.
>>>> > >
>>>> > > Is that incorrect?
>>>> > >
>>>> > > Thanks
>>>> > >
>>>> > >
>>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>>>> > wrote:
>>>> > >
>>>> > >> Mike,
>>>> > >>
>>>> > >> As I understand these two patches, the throttled I?O settings are
>>>> > applied
>>>> > >> from the hypervisor side, and possibly defined in a compute
>>>> offering
>>>> > where
>>>> > >> provisioned IOPS are defined on the storage side through disk
>>>> > offerings. I
>>>> > >> don't see how the management server could enforce this mutual
>>>> exclusion
>>>> > >> between provisioned IOPS and throttled I/O until a compute and disk
>>>> > >> offering were selected. These selections come together when we
>>>> deploy a
>>>> > >> VM. Based on these assumptions, I would expect to see enforcement
>>>> of
>>>> > this
>>>> > >> rule in UI at the time of VM creation/definition and on the server
>>>> > side, as
>>>> > >> part of the VM creation. It feels like any attempt to enforce
>>>> this rule
>>>> > >> when defining offering would be premature, and unnecessarily limit
>>>> > >> flexibility. Are my assumptions and understanding correct?
>>>> > >>
>>>> > >> Thanks,
>>>> > >> -John
>>>> > >>
>>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>>>> > mike.tutkowski@solidfire.com>
>>>> > >> wrote:
>>>> > >>
>>>> > >>> Hi John,
>>>> > >>>
>>>> > >>> So, maybe I'm wrong about this, but what I was thinking is that
>>>> we'd
>>>> > >> build
>>>> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
>>>> > >> Compute
>>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>>>> > >>>
>>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
>>>> Storage
>>>> > >> QoS
>>>> > >>> for the purpose of this e-mail.
>>>> > >>>
>>>> > >>> By default, neither radio button is selected and no QoS fields are
>>>> > >> visible.
>>>> > >>>
>>>> > >>> If the user selects the Storage QoS radio button, then the Custom
>>>> IOPS
>>>> > >>> checkbox and the Min and Max text fields are made visible.
>>>> > >>>
>>>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>>>> > button,
>>>> > >>> then the Custom IOPS checkbox and the Min and Max text fields
>>>> disappear
>>>> > >> and
>>>> > >>> are replaced by the two Hypervisor QoS text fields.
>>>> > >>>
>>>> > >>> This way, the user can choose neither QoS option or one of them
>>>> or the
>>>> > >>> other, but never both.
>>>> > >>>
>>>> > >>> On the API side, I was thinking of having logic in place when a
>>>> request
>>>> > >>> comes in to create a Disk Offering to confirm these fields are
>>>> the way
>>>> > we
>>>> > >>> want them.
>>>> > >>>
>>>> > >>> Once we know the Disk Offering is in order, a user can create a
>>>> data
>>>> > disk
>>>> > >>> from it. Since we checked the validity of the Disk Offering when
>>>> it was
>>>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>>>> > Storage
>>>> > >>> QoS in being utilized.
>>>> > >>>
>>>> > >>> Does that make sense or did I miss something?
>>>> > >>>
>>>> > >>> Thanks
>>>> > >>>
>>>> > >>>
>>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jburwell@basho.com
>>>> >
>>>> > >> wrote:
>>>> > >>>
>>>> > >>>> Mike,
>>>> > >>>>
>>>> > >>>> Looking through the code, I am trying to understand how
>>>> > >>>> CreateDiskOfferingCmd would have the context to identify the
>>>> conflict.
>>>> > >>>> Naively, it seems to me that this rule would need to be enforced
>>>> when
>>>> > a
>>>> > >>>> virtual machine is being deployed. Looking through the code, it
>>>> seems
>>>> > >> like
>>>> > >>>> we should add a private validateStorageQoS method to
>>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and
>>>> throws a
>>>> > >>>> ResourceAllocationException when the QoS definitions are
>>>> inconsistent.
>>>> > >> We
>>>> > >>>> would then add calls to it from each of the VM creation methods
>>>> in the
>>>> > >>>> service. Do this type of approach sound reasonable?
>>>> > >>>>
>>>> > >>>> Thanks,
>>>> > >>>> -John
>>>> > >>>>
>>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>>>> > >> mike.tutkowski@solidfire.com>
>>>> > >>>> wrote:
>>>> > >>>>
>>>> > >>>>> Hi John,
>>>> > >>>>>
>>>> > >>>>> So, here's what I was planning to do. Of course feel free to
>>>> correct
>>>> > me
>>>> > >>>> on
>>>> > >>>>> this approach.
>>>> > >>>>>
>>>> > >>>>> I think it's OK if Wei merges his code into master and then I
>>>> can
>>>> > draw
>>>> > >>>> from
>>>> > >>>>> the main repo and merge master into mine locally.
>>>> > >>>>>
>>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI
>>>> code
>>>> > to
>>>> > >>>> make
>>>> > >>>>> it user friendly (toggle between these features on the Add Disk
>>>> > >> Offering
>>>> > >>>>> window).
>>>> > >>>>>
>>>> > >>>>> 2) I plan to write validation logic for the
>>>> create-disk-offering API
>>>> > >>>>> command which throws an exception if the rules are not followed
>>>> (this
>>>> > >>>>> should never be triggered from the GUI since the GUI will have
>>>> > controls
>>>> > >>>> in
>>>> > >>>>> place to toggle between the one feature and the other).
>>>> > >>>>>
>>>> > >>>>> I'm not sure about documentation. I haven't had much experience
>>>> with
>>>> > it
>>>> > >>>> on
>>>> > >>>>> CloudStack projects yet.
>>>> > >>>>>
>>>> > >>>>> Thanks!
>>>> > >>>>>
>>>> > >>>>>
>>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <
>>>> jburwell@basho.com>
>>>> > >>>> wrote:
>>>> > >>>>>
>>>> > >>>>>> Mike,
>>>> > >>>>>>
>>>> > >>>>>> Yes, these server-side rails need to be defined and implemented
>>>> > before
>>>> > >>>>>> either patch can be merged. From my perspective, I would like
>>>> to
>>>> > see
>>>> > >>>> the
>>>> > >>>>>> rule implemented in the hypervisor as part of the validation
>>>> of the
>>>> > >>>> virtual
>>>> > >>>>>> machine definition. We also need to make sure that this mutual
>>>> > >>>> exclusion
>>>> > >>>>>> is documented. Do we usually include this type of
>>>> documentation
>>>> > with
>>>> > >>>>>> patches of this nature?
>>>> > >>>>>>
>>>> > >>>>>> Thanks,
>>>> > >>>>>> -John
>>>> > >>>>>>
>>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>>> > >>>> mike.tutkowski@solidfire.com>
>>>> > >>>>>> wrote:
>>>> > >>>>>>
>>>> > >>>>>>> Currently they are not yet implemented.
>>>> > >>>>>>>
>>>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>>>> > usability
>>>> > >>>>>>> standpoint, but the API must check for consistency and throw
>>>> an
>>>> > >>>> exception
>>>> > >>>>>>> if necessary.
>>>> > >>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
>>>> jburwell@basho.com
>>>> > >
>>>> > >>>>>> wrote:
>>>> > >>>>>>>
>>>> > >>>>>>>> Mike,
>>>> > >>>>>>>>
>>>> > >>>>>>>> Are the checks only implemented in the UI?
>>>> > >>>>>>>>
>>>> > >>>>>>>> Thanks,
>>>> > >>>>>>>> -John
>>>> > >>>>>>>>
>>>> > >>>>>>>>
>>>> > >>>>>>>>
>>>> > >>>>>>>>
>>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>>> > >>>>>>>> <mi...@solidfire.com> wrote:
>>>> > >>>>>>>>
>>>> > >>>>>>>>> Hi John,
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>>>> > >> exclusive.
>>>> > >>>> We
>>>> > >>>>>>>>> agree with you that only one should be active at a time. We
>>>> plan
>>>> > to
>>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to
>>>> turn
>>>> > his
>>>> > >>>>>>>> feature
>>>> > >>>>>>>>> on and mine off and vice versa.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
>>>> > update
>>>> > >>>> and
>>>> > >>>>>>>>> merge that I will be the person resolving merge conflicts
>>>> in the
>>>> > >>>>>>>> JavaScript
>>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as
>>>> opposed
>>>> > to
>>>> > >>>>>>>> putting
>>>> > >>>>>>>>> that work on someone else.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Let me know what you think.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Thanks!
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>>>> > jburwell@basho.com
>>>> > >>>
>>>> > >>>>>>>> wrote:
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>> Mike,
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>>> > >>>> mechanisms
>>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the
>>>> affected
>>>> > FSs
>>>> > >>>> been
>>>> > >>>>>>>>>> updated with that decision?
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
>>>> > >> between
>>>> > >>>>>> the
>>>> > >>>>>>>>>> patches?
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> Thanks,
>>>> > >>>>>>>>>> -John
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>> No problem, John.
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up
>>>> with a
>>>> > >> new
>>>> > >>>>>>>> patch
>>>> > >>>>>>>>>>> file.
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>> checked
>>>> > >> in
>>>> > >>>>>> and
>>>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>> > >> jburwell@basho.com
>>>> > >>>>>
>>>> > >>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>> Mike,
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>>> > >> offline
>>>> > >>>>>> ATM,
>>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> Sorry for the oversight,
>>>> > >>>>>>>>>>>> -John
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>> this :)
>>>> > >> ),
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>> areas
>>>> > we
>>>> > >>>>>> wanted
>>>> > >>>>>>>>>>>>> addressed.
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>> > afternoon,
>>>> > >>>> then
>>>> > >>>>>>>>>> send
>>>> > >>>>>>>>>>>> in
>>>> > >>>>>>>>>>>>> another patch.
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>> Sure, that sounds good.
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>> > >>>> ustcweizhou@gmail.com
>>>> > >>>>>>>
>>>> > >>>>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> Hi Mike,
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
>>>> in Java
>>>> > >>>> code,
>>>> > >>>>>>>>>>>> except
>>>> > >>>>>>>>>>>>>>> the cloudstack UI.
>>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>> branch
>>>> > >> into
>>>> > >>>>>>>>>> master
>>>> > >>>>>>>>>>>>>>> this
>>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> -Wei
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> Hey John,
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>> object_store
>>>> > >>>> branch,
>>>> > >>>>>>>> but
>>>> > >>>>>>>>>> -
>>>> > >>>>>>>>>>>> as
>>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>> > >>>> SolidFire
>>>> > >>>>>>>>>> branch
>>>> > >>>>>>>>>>>>>>> into
>>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>>> > >> master.
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>>>> this
>>>> > merge
>>>> > >>>>>>>>>> strategy.
>>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>> > >>>>>>>> jburwell@basho.com>
>>>> > >>>>>>>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> Mike,
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>> > >>>>>>>> disk_io_throttling,
>>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>>> of
>>>> > >>>>>>>> overlapping
>>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>>> > >> patches
>>>> > >>>> to
>>>> > >>>>>>>> be
>>>> > >>>>>>>>>> as
>>>> > >>>>>>>>>>>>>>>>> follows:
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>> > >> SolidFire
>>>> > >>>>>> are
>>>> > >>>>>>>>>>>>>>> additive
>>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>> > dependency
>>>> > >>>>>>>> between
>>>> > >>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>>> we
>>>> > can
>>>> > >>>>>>>>>> determine
>>>> > >>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> Thanks,
>>>> > >>>>>>>>>>>>>>>>> -John
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> > >>>>>>>>>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>> into
>>>> > his
>>>> > >>>>>> branch
>>>> > >>>>>>>>>>>>>>> before
>>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>> after his
>>>> > >>>> merge
>>>> > >>>>>>>> and
>>>> > >>>>>>>>>>>>>>> we
>>>> > >>>>>>>>>>>>>>>> can
>>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>> code in
>>>> > >> the
>>>> > >>>>>>>>>> storage
>>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>> > plug-in,
>>>> > >>>>>> which
>>>> > >>>>>>>>>>>>>>>>> currently
>>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>> > (although
>>>> > >>>> it
>>>> > >>>>>>>> does
>>>> > >>>>>>>>>>>>>>> not
>>>> > >>>>>>>>>>>>>>>>> know
>>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>> actually
>>>> > in
>>>> > >>>>>> use))
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>>> the
>>>> > url
>>>> > >>>>>> field
>>>> > >>>>>>>>>> (if
>>>> > >>>>>>>>>>>>>>>> not
>>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>>>> stored in
>>>> > the
>>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>> hypervisor
>>>> > in
>>>> > >>>>>>>>>>>>>>> question, we
>>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>> place
>>>> > of
>>>> > >>>> the
>>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>> > checks
>>>> > >>>> for
>>>> > >>>>>>>> the
>>>> > >>>>>>>>>>>>>>>> managed
>>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>> > hypervisor
>>>> > >>>>>> data
>>>> > >>>>>>>>>>>>>>>>> structure is
>>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>> executes to
>>>> > >>>> attach
>>>> > >>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>> volume.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>>> > >> detach a
>>>> > >>>>>>>>>> volume,
>>>> > >>>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>>>> data
>>>> > >>>> structure
>>>> > >>>>>>>> is
>>>> > >>>>>>>>>>>>>>>>> removed.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>>>> IOPS in
>>>> > 4.2
>>>> > >>>>>>>> unless
>>>> > >>>>>>>>>>>>>>> it is
>>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
>>>> If we
>>>> > >> have
>>>> > >>>>>> some
>>>> > >>>>>>>>>>>>>>> idea,
>>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> Thanks!
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>> avoiding
>>>> > >>>>>>>>>> implementation
>>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>> > field.
>>>> > >> I
>>>> > >>>>>>>> think
>>>> > >>>>>>>>>>>>>>> we
>>>> > >>>>>>>>>>>>>>>>> should
>>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>> regarding
>>>> > >> the
>>>> > >>>>>>>>>>>>>>> expected
>>>> > >>>>>>>>>>>>>>>>> format
>>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>>>> details are
>>>> > >> not
>>>> > >>>>>>>>>>>>>>> returned on
>>>> > >>>>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>> > listStoragePool
>>>> > >>>>>>>>>> response.
>>>> > >>>>>>>>>>>>>>>> Why
>>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>> useful
>>>> > >> for
>>>> > >>>>>>>>>> clients
>>>> > >>>>>>>>>>>>>>> to
>>>> > >>>>>>>>>>>>>>>> be
>>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>> field."
>>>> > >>>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>> here.
>>>> > >>>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>> > >> Offering-by-Disk
>>>> > >>>>>>>>>> Offering
>>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>> to be
>>>> > >> able
>>>> > >>>>>> to
>>>> > >>>>>>>>>>>>>>>> associate
>>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>> disk_offering_details
>>>> > >> table.
>>>> > >>>>>>>> Maybe
>>>> > >>>>>>>>>>>>>>> it
>>>> > >>>>>>>>>>>>>>>>> could
>>>> > >>>>>>>>>>>>>>>>>>>> go there?
>>>> > >>>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>>> IOPS
>>>> > in
>>>> > >>>> the
>>>> > >>>>>>>> GUI
>>>> > >>>>>>>>>>>>>>> if
>>>> > >>>>>>>>>>>>>>>>> it's
>>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>>>> the DB.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>>>>>>>>> cloud<
>>>> > http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> --
>>>> > >>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> --
>>>> > >>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>> *â„¢*
>>>> > >>>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>> --
>>>> > >>>>>>> *Mike Tutkowski*
>>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>> o: 303.746.7302
>>>> > >>>>>>> Advancing the way the world uses the
>>>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>> *â„¢*
>>>> > >>>>>>
>>>> > >>>>>>
>>>> > >>>>>
>>>> > >>>>>
>>>> > >>>>> --
>>>> > >>>>> *Mike Tutkowski*
>>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>> o: 303.746.7302
>>>> > >>>>> Advancing the way the world uses the
>>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>> *â„¢*
>>>> > >>>>
>>>> > >>>>
>>>> > >>>
>>>> > >>>
>>>> > >>> --
>>>> > >>> *Mike Tutkowski*
>>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>> e: mike.tutkowski@solidfire.com
>>>> > >>> o: 303.746.7302
>>>> > >>> Advancing the way the world uses the
>>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>> *â„¢*
>>>> > >>
>>>> > >>
>>>> > >
>>>> > >
>>>> > > --
>>>> > > *Mike Tutkowski*
>>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>>> > > e: mike.tutkowski@solidfire.com
>>>> > > o: 303.746.7302
>>>> > > Advancing the way the world uses the
>>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > > *â„¢*
>>>> >
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Yes. To your point, the appropriate logic would be to check the Volume allocated by the StorageAllocator. If it doesn't support a QoS, then the VM allocator would attempt to fulfill the QoS through the hypervisor. Another question would be -- what would be in the behavior in the event that no resources are available to fulfill the QoS? We could outright fail or proceed with some kind of warning -- sounds like a good place for a global setting to configure the policy.
The other question we haven't answered are usage records. By pushing the QoS concept out to allocation and making it a more general concept, it could make usage record capture more straightforward.
Thanks,
-John
On Jun 12, 2013, at 6:11 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> I see, John.
>
> That is an interesting idea.
>
> We'd also have to change the storage allocator(s) to favor QoS-supported storage if those fields are filled in.
>
>
> On Wed, Jun 12, 2013 at 4:09 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> My thought is that we present the min/max IOPS fields for read/write operations for all compute and disk offerings. When the VM is allocated, we determine the best way to fulfill that QoS. It sounds like storage level guarantees would always be preferred. If no storage is available to guarantee the QoS, the allocator falls back to using hypervisor. This approach only works if summing the discrete read/write min/max values to get to min/max total IOPS would be acceptable.
>
> Thanks,
> -John
>
>
> On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>
>> I hate to say it, but I believe Storage QoS with a Min and Max will always be optimal over hypervisor rate limiting.
>>
>> The only time you'd want to use hypervisor rate limiting is if storage QoS is not available.
>>
>> We currently have no way to know what storage the root or data disk will be deployed to, so we have to present the fields in all situation.
>>
>> This is because of - in my opinion - the somewhat flawed way CloudStack handles storage tags. You are free to enter in any text there and we don't know what it maps to when the Compute or Disk Offering dialog is up.
>>
>>
>> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>> Mike,
>>
>> Please see my comments/questions in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>>
>>> Hi Wei,
>>>
>>> So, not entirely sure I follow.
>>>
>>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>>
>>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage QoS for the purpose of this e-mail.
>>>
>>> By default, neither radio button is selected and no QoS fields are visible.
>>
>> I prefer a None option selected by default. I find the no button selected in a Radio button group to be confusing.
>>
>> Also, do we have a mechanism to determine whether or not this radio button group should be displayed at all? If there are no devices or hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>>
>>>
>>> If the user selects the Storage QoS radio button, then the Custom IOPS checkbox and the Min and Max text fields are made visible.
>>>
>>> If the user changes his mind and selects the Hypervisor QoS radio button, then the Custom IOPS checkbox and the Min and Max text fields disappear and are replaced by the two Hypervisor QoS text fields.
>>>
>>> This way, the user can choose neither QoS option or one of them or the other, but never both.
>>>
>>> On the API side, I was thinking of having logic in place when a request comes in to create a Disk Offering to confirm these fields are the way we want them.
>>>
>>> Once we know the Disk Offering is in order, a user can create a data disk from it. Since we checked the validity of the Disk Offering when it was created, the VM should never be asked to use Hypervisor QoS when Storage QoS in being utilized.
>>
>> Overall, this model sounds very reasonable. Wondering aloud: Would be possible to have the UI simply ask for a storage QoS, and let the system pick the optional implementation. For example, if a VM is attached to KVM and SolidFire is available, we chose to fulfill the QoS using the storage device. This approach would also allow us to fallback in the event that we can't allocate storage for the QoS, but we have hypervisor capacity. Could we always deterministically determine the optimal implementation of the QoS at time of VM deployment?
>>
>>>
>>> Thanks!
>>>
>>>
>>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
>>> Mike, John
>>>
>>> The Disk I/O Throttling works like:
>>>
>>> (1) For User VM,
>>> (1.1) root disks: service offering -> default value in global configuration
>>> (1.2) data disks: disk offering -> default value in global configuration
>>>
>>> (2) For System VM
>>> root disks: service offering
>>>
>>> -Wei
>>>
>>>
>>> 2013/6/12 John Burwell <jb...@basho.com>
>>>
>>> > Mike,
>>> >
>>> > That is one possibility. The other possibility is that hypervisor is
>>> > going to throttle I/O on all disks attached. Therefore, we need answers to
>>> > the following questions:
>>> >
>>> > 1. Is I/O throttling applied to the root disk or all disks
>>> > attached to the VM?
>>> > 2. If I/O throttling is applied to all disks, how is the
>>> > throttling distributed amongst the disks if only one read/write value is
>>> > defined?
>>> >
>>> > Thanks,
>>> > -John
>>> >
>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
>>> > wrote:
>>> >
>>> > > Hey John,
>>> > >
>>> > > Perhaps I don't fully understand how Wei's feature works.
>>> > >
>>> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>>> > >
>>> > > From my thinking, you're root disk could be under Hypervisor QoS, but
>>> > your
>>> > > data disk could be under Storage QoS.
>>> > >
>>> > > Is that incorrect?
>>> > >
>>> > > Thanks
>>> > >
>>> > >
>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>>> > wrote:
>>> > >
>>> > >> Mike,
>>> > >>
>>> > >> As I understand these two patches, the throttled I?O settings are
>>> > applied
>>> > >> from the hypervisor side, and possibly defined in a compute offering
>>> > where
>>> > >> provisioned IOPS are defined on the storage side through disk
>>> > offerings. I
>>> > >> don't see how the management server could enforce this mutual exclusion
>>> > >> between provisioned IOPS and throttled I/O until a compute and disk
>>> > >> offering were selected. These selections come together when we deploy a
>>> > >> VM. Based on these assumptions, I would expect to see enforcement of
>>> > this
>>> > >> rule in UI at the time of VM creation/definition and on the server
>>> > side, as
>>> > >> part of the VM creation. It feels like any attempt to enforce this rule
>>> > >> when defining offering would be premature, and unnecessarily limit
>>> > >> flexibility. Are my assumptions and understanding correct?
>>> > >>
>>> > >> Thanks,
>>> > >> -John
>>> > >>
>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>>> > mike.tutkowski@solidfire.com>
>>> > >> wrote:
>>> > >>
>>> > >>> Hi John,
>>> > >>>
>>> > >>> So, maybe I'm wrong about this, but what I was thinking is that we'd
>>> > >> build
>>> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
>>> > >> Compute
>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>>> > >>>
>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>>> > >> QoS
>>> > >>> for the purpose of this e-mail.
>>> > >>>
>>> > >>> By default, neither radio button is selected and no QoS fields are
>>> > >> visible.
>>> > >>>
>>> > >>> If the user selects the Storage QoS radio button, then the Custom IOPS
>>> > >>> checkbox and the Min and Max text fields are made visible.
>>> > >>>
>>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>>> > button,
>>> > >>> then the Custom IOPS checkbox and the Min and Max text fields disappear
>>> > >> and
>>> > >>> are replaced by the two Hypervisor QoS text fields.
>>> > >>>
>>> > >>> This way, the user can choose neither QoS option or one of them or the
>>> > >>> other, but never both.
>>> > >>>
>>> > >>> On the API side, I was thinking of having logic in place when a request
>>> > >>> comes in to create a Disk Offering to confirm these fields are the way
>>> > we
>>> > >>> want them.
>>> > >>>
>>> > >>> Once we know the Disk Offering is in order, a user can create a data
>>> > disk
>>> > >>> from it. Since we checked the validity of the Disk Offering when it was
>>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>>> > Storage
>>> > >>> QoS in being utilized.
>>> > >>>
>>> > >>> Does that make sense or did I miss something?
>>> > >>>
>>> > >>> Thanks
>>> > >>>
>>> > >>>
>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>>> > >> wrote:
>>> > >>>
>>> > >>>> Mike,
>>> > >>>>
>>> > >>>> Looking through the code, I am trying to understand how
>>> > >>>> CreateDiskOfferingCmd would have the context to identify the conflict.
>>> > >>>> Naively, it seems to me that this rule would need to be enforced when
>>> > a
>>> > >>>> virtual machine is being deployed. Looking through the code, it seems
>>> > >> like
>>> > >>>> we should add a private validateStorageQoS method to
>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
>>> > >>>> ResourceAllocationException when the QoS definitions are inconsistent.
>>> > >> We
>>> > >>>> would then add calls to it from each of the VM creation methods in the
>>> > >>>> service. Do this type of approach sound reasonable?
>>> > >>>>
>>> > >>>> Thanks,
>>> > >>>> -John
>>> > >>>>
>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>>> > >> mike.tutkowski@solidfire.com>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>> Hi John,
>>> > >>>>>
>>> > >>>>> So, here's what I was planning to do. Of course feel free to correct
>>> > me
>>> > >>>> on
>>> > >>>>> this approach.
>>> > >>>>>
>>> > >>>>> I think it's OK if Wei merges his code into master and then I can
>>> > draw
>>> > >>>> from
>>> > >>>>> the main repo and merge master into mine locally.
>>> > >>>>>
>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code
>>> > to
>>> > >>>> make
>>> > >>>>> it user friendly (toggle between these features on the Add Disk
>>> > >> Offering
>>> > >>>>> window).
>>> > >>>>>
>>> > >>>>> 2) I plan to write validation logic for the create-disk-offering API
>>> > >>>>> command which throws an exception if the rules are not followed (this
>>> > >>>>> should never be triggered from the GUI since the GUI will have
>>> > controls
>>> > >>>> in
>>> > >>>>> place to toggle between the one feature and the other).
>>> > >>>>>
>>> > >>>>> I'm not sure about documentation. I haven't had much experience with
>>> > it
>>> > >>>> on
>>> > >>>>> CloudStack projects yet.
>>> > >>>>>
>>> > >>>>> Thanks!
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
>>> > >>>> wrote:
>>> > >>>>>
>>> > >>>>>> Mike,
>>> > >>>>>>
>>> > >>>>>> Yes, these server-side rails need to be defined and implemented
>>> > before
>>> > >>>>>> either patch can be merged. From my perspective, I would like to
>>> > see
>>> > >>>> the
>>> > >>>>>> rule implemented in the hypervisor as part of the validation of the
>>> > >>>> virtual
>>> > >>>>>> machine definition. We also need to make sure that this mutual
>>> > >>>> exclusion
>>> > >>>>>> is documented. Do we usually include this type of documentation
>>> > with
>>> > >>>>>> patches of this nature?
>>> > >>>>>>
>>> > >>>>>> Thanks,
>>> > >>>>>> -John
>>> > >>>>>>
>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>> > >>>> mike.tutkowski@solidfire.com>
>>> > >>>>>> wrote:
>>> > >>>>>>
>>> > >>>>>>> Currently they are not yet implemented.
>>> > >>>>>>>
>>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>>> > usability
>>> > >>>>>>> standpoint, but the API must check for consistency and throw an
>>> > >>>> exception
>>> > >>>>>>> if necessary.
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jburwell@basho.com
>>> > >
>>> > >>>>>> wrote:
>>> > >>>>>>>
>>> > >>>>>>>> Mike,
>>> > >>>>>>>>
>>> > >>>>>>>> Are the checks only implemented in the UI?
>>> > >>>>>>>>
>>> > >>>>>>>> Thanks,
>>> > >>>>>>>> -John
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>> > >>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>
>>> > >>>>>>>>> Hi John,
>>> > >>>>>>>>>
>>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>>> > >> exclusive.
>>> > >>>> We
>>> > >>>>>>>>> agree with you that only one should be active at a time. We plan
>>> > to
>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn
>>> > his
>>> > >>>>>>>> feature
>>> > >>>>>>>>> on and mine off and vice versa.
>>> > >>>>>>>>>
>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
>>> > update
>>> > >>>> and
>>> > >>>>>>>>> merge that I will be the person resolving merge conflicts in the
>>> > >>>>>>>> JavaScript
>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed
>>> > to
>>> > >>>>>>>> putting
>>> > >>>>>>>>> that work on someone else.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Let me know what you think.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Thanks!
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>>> > jburwell@basho.com
>>> > >>>
>>> > >>>>>>>> wrote:
>>> > >>>>>>>>>
>>> > >>>>>>>>>> Mike,
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>> > >>>> mechanisms
>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected
>>> > FSs
>>> > >>>> been
>>> > >>>>>>>>>> updated with that decision?
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
>>> > >> between
>>> > >>>>>> the
>>> > >>>>>>>>>> patches?
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> Thanks,
>>> > >>>>>>>>>> -John
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>> No problem, John.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up with a
>>> > >> new
>>> > >>>>>>>> patch
>>> > >>>>>>>>>>> file.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
>>> > >> in
>>> > >>>>>> and
>>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>> > >> jburwell@basho.com
>>> > >>>>>
>>> > >>>>>>>>>> wrote:
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>> Mike,
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>> > >> offline
>>> > >>>>>> ATM,
>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> Sorry for the oversight,
>>> > >>>>>>>>>>>> -John
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
>>> > >> ),
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
>>> > we
>>> > >>>>>> wanted
>>> > >>>>>>>>>>>>> addressed.
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>> > afternoon,
>>> > >>>> then
>>> > >>>>>>>>>> send
>>> > >>>>>>>>>>>> in
>>> > >>>>>>>>>>>>> another patch.
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> Sure, that sounds good.
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> > >>>> ustcweizhou@gmail.com
>>> > >>>>>>>
>>> > >>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> Hi Mike,
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>>> > >>>> code,
>>> > >>>>>>>>>>>> except
>>> > >>>>>>>>>>>>>>> the cloudstack UI.
>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
>>> > >> into
>>> > >>>>>>>>>> master
>>> > >>>>>>>>>>>>>>> this
>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> -Wei
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> Hey John,
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>>> > >>>> branch,
>>> > >>>>>>>> but
>>> > >>>>>>>>>> -
>>> > >>>>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>> > >>>> SolidFire
>>> > >>>>>>>>>> branch
>>> > >>>>>>>>>>>>>>> into
>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>> > >> master.
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>> > merge
>>> > >>>>>>>>>> strategy.
>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>> > >>>>>>>> jburwell@basho.com>
>>> > >>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Mike,
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>> > >>>>>>>> disk_io_throttling,
>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>> > >>>>>>>> overlapping
>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>> > >> patches
>>> > >>>> to
>>> > >>>>>>>> be
>>> > >>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>>> follows:
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>> > >> SolidFire
>>> > >>>>>> are
>>> > >>>>>>>>>>>>>>> additive
>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>> > dependency
>>> > >>>>>>>> between
>>> > >>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
>>> > can
>>> > >>>>>>>>>> determine
>>> > >>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Thanks,
>>> > >>>>>>>>>>>>>>>>> -John
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> > >>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
>>> > his
>>> > >>>>>> branch
>>> > >>>>>>>>>>>>>>> before
>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>> > >>>> merge
>>> > >>>>>>>> and
>>> > >>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>> can
>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
>>> > >> the
>>> > >>>>>>>>>> storage
>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>> > plug-in,
>>> > >>>>>> which
>>> > >>>>>>>>>>>>>>>>> currently
>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>> > (although
>>> > >>>> it
>>> > >>>>>>>> does
>>> > >>>>>>>>>>>>>>> not
>>> > >>>>>>>>>>>>>>>>> know
>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
>>> > in
>>> > >>>>>> use))
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
>>> > url
>>> > >>>>>> field
>>> > >>>>>>>>>> (if
>>> > >>>>>>>>>>>>>>>> not
>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
>>> > the
>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
>>> > in
>>> > >>>>>>>>>>>>>>> question, we
>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
>>> > of
>>> > >>>> the
>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>> > checks
>>> > >>>> for
>>> > >>>>>>>> the
>>> > >>>>>>>>>>>>>>>> managed
>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>> > hypervisor
>>> > >>>>>> data
>>> > >>>>>>>>>>>>>>>>> structure is
>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>>> > >>>> attach
>>> > >>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> volume.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>> > >> detach a
>>> > >>>>>>>>>> volume,
>>> > >>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> > >>>> structure
>>> > >>>>>>>> is
>>> > >>>>>>>>>>>>>>>>> removed.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
>>> > 4.2
>>> > >>>>>>>> unless
>>> > >>>>>>>>>>>>>>> it is
>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
>>> > >> have
>>> > >>>>>> some
>>> > >>>>>>>>>>>>>>> idea,
>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> Thanks!
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>> > >>>>>>>>>> implementation
>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>> > field.
>>> > >> I
>>> > >>>>>>>> think
>>> > >>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>>> should
>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
>>> > >> the
>>> > >>>>>>>>>>>>>>> expected
>>> > >>>>>>>>>>>>>>>>> format
>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
>>> > >> not
>>> > >>>>>>>>>>>>>>> returned on
>>> > >>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>> > listStoragePool
>>> > >>>>>>>>>> response.
>>> > >>>>>>>>>>>>>>>> Why
>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
>>> > >> for
>>> > >>>>>>>>>> clients
>>> > >>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>> be
>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>> > >> Offering-by-Disk
>>> > >>>>>>>>>> Offering
>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>>> > >> able
>>> > >>>>>> to
>>> > >>>>>>>>>>>>>>>> associate
>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>>> > >> table.
>>> > >>>>>>>> Maybe
>>> > >>>>>>>>>>>>>>> it
>>> > >>>>>>>>>>>>>>>>> could
>>> > >>>>>>>>>>>>>>>>>>>> go there?
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
>>> > in
>>> > >>>> the
>>> > >>>>>>>> GUI
>>> > >>>>>>>>>>>>>>> if
>>> > >>>>>>>>>>>>>>>>> it's
>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>>>>>>> cloud<
>>> > http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> --
>>> > >>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> --
>>> > >>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>> *â„¢*
>>> > >>>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>> --
>>> > >>>>>>> *Mike Tutkowski*
>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>> o: 303.746.7302
>>> > >>>>>>> Advancing the way the world uses the
>>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>> *â„¢*
>>> > >>>>>>
>>> > >>>>>>
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> --
>>> > >>>>> *Mike Tutkowski*
>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>> o: 303.746.7302
>>> > >>>>> Advancing the way the world uses the
>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>> *â„¢*
>>> > >>>>
>>> > >>>>
>>> > >>>
>>> > >>>
>>> > >>> --
>>> > >>> *Mike Tutkowski*
>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>> e: mike.tutkowski@solidfire.com
>>> > >>> o: 303.746.7302
>>> > >>> Advancing the way the world uses the
>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>> *â„¢*
>>> > >>
>>> > >>
>>> > >
>>> > >
>>> > > --
>>> > > *Mike Tutkowski*
>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>> > > e: mike.tutkowski@solidfire.com
>>> > > o: 303.746.7302
>>> > > Advancing the way the world uses the
>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > > *â„¢*
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Mike Tutkowski
>>> Senior CloudStack Developer, SolidFire Inc.
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloudâ„¢
>>
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloudâ„¢
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
It looks like Wei has four new fields:
Max Read IOPS, Max Write IOPS, Max Read BPS, and Max Write BPS
I have two:
Max IOPS and Min IOPS
Mine can be set by the admin or the admin can choose to have the end user
fill in these values.
On Wed, Jun 12, 2013 at 4:13 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> As far as compressing read and write IOPS fields into one value (one for
> Min and one for Max), that is Wei's area as my feature does not distinguish
> between read and write IOPS.
>
>
> On Wed, Jun 12, 2013 at 4:11 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> I see, John.
>>
>> That is an interesting idea.
>>
>> We'd also have to change the storage allocator(s) to favor QoS-supported
>> storage if those fields are filled in.
>>
>>
>> On Wed, Jun 12, 2013 at 4:09 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> My thought is that we present the min/max IOPS fields for read/write
>>> operations for all compute and disk offerings. When the VM is allocated,
>>> we determine the best way to fulfill that QoS. It sounds like storage
>>> level guarantees would always be preferred. If no storage is available to
>>> guarantee the QoS, the allocator falls back to using hypervisor. This
>>> approach only works if summing the discrete read/write min/max values to
>>> get to min/max total IOPS would be acceptable.
>>>
>>> Thanks,
>>> -John
>>>
>>>
>>> On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> I hate to say it, but I believe Storage QoS with a Min and Max will
>>> always be optimal over hypervisor rate limiting.
>>>
>>> The only time you'd want to use hypervisor rate limiting is if storage
>>> QoS is not available.
>>>
>>> We currently have no way to know what storage the root or data disk will
>>> be deployed to, so we have to present the fields in all situation.
>>>
>>> This is because of - in my opinion - the somewhat flawed way CloudStack
>>> handles storage tags. You are free to enter in any text there and we don't
>>> know what it maps to when the Compute or Disk Offering dialog is up.
>>>
>>>
>>> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com>wrote:
>>>
>>>> Mike,
>>>>
>>>> Please see my comments/questions in-line below.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>> Hi Wei,
>>>>
>>>> So, not entirely sure I follow.
>>>>
>>>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>>>
>>>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>>>> QoS for the purpose of this e-mail.
>>>>
>>>> By default, neither radio button is selected and no QoS fields are
>>>> visible.
>>>>
>>>>
>>>> I prefer a None option selected by default. I find the no button
>>>> selected in a Radio button group to be confusing.
>>>>
>>>> Also, do we have a mechanism to determine whether or not this radio
>>>> button group should be displayed at all? If there are no devices or
>>>> hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>>>>
>>>>
>>>> If the user selects the Storage QoS radio button, then the Custom IOPS
>>>> checkbox and the Min and Max text fields are made visible.
>>>>
>>>> If the user changes his mind and selects the Hypervisor QoS radio
>>>> button, then the Custom IOPS checkbox and the Min and Max text fields
>>>> disappear and are replaced by the two Hypervisor QoS text fields.
>>>>
>>>> This way, the user can choose neither QoS option or one of them or the
>>>> other, but never both.
>>>>
>>>> On the API side, I was thinking of having logic in place when a request
>>>> comes in to create a Disk Offering to confirm these fields are the way we
>>>> want them.
>>>>
>>>> Once we know the Disk Offering is in order, a user can create a data
>>>> disk from it. Since we checked the validity of the Disk Offering when it
>>>> was created, the VM should never be asked to use Hypervisor QoS when
>>>> Storage QoS in being utilized.
>>>>
>>>>
>>>> Overall, this model sounds very reasonable. Wondering aloud: Would be
>>>> possible to have the UI simply ask for a storage QoS, and let the system
>>>> pick the optional implementation. For example, if a VM is attached to KVM
>>>> and SolidFire is available, we chose to fulfill the QoS using the storage
>>>> device. This approach would also allow us to fallback in the event that we
>>>> can't allocate storage for the QoS, but we have hypervisor capacity. Could
>>>> we always deterministically determine the optimal implementation of the QoS
>>>> at time of VM deployment?
>>>>
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com>wrote:
>>>>
>>>>> Mike, John
>>>>>
>>>>> The Disk I/O Throttling works like:
>>>>>
>>>>> (1) For User VM,
>>>>> (1.1) root disks: service offering -> default value in global
>>>>> configuration
>>>>> (1.2) data disks: disk offering -> default value in global
>>>>> configuration
>>>>>
>>>>> (2) For System VM
>>>>> root disks: service offering
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>> 2013/6/12 John Burwell <jb...@basho.com>
>>>>>
>>>>> > Mike,
>>>>> >
>>>>> > That is one possibility. The other possibility is that hypervisor is
>>>>> > going to throttle I/O on all disks attached. Therefore, we need
>>>>> answers to
>>>>> > the following questions:
>>>>> >
>>>>> > 1. Is I/O throttling applied to the root disk or all disks
>>>>> > attached to the VM?
>>>>> > 2. If I/O throttling is applied to all disks, how is the
>>>>> > throttling distributed amongst the disks if only one read/write
>>>>> value is
>>>>> > defined?
>>>>> >
>>>>> > Thanks,
>>>>> > -John
>>>>> >
>>>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>> > wrote:
>>>>> >
>>>>> > > Hey John,
>>>>> > >
>>>>> > > Perhaps I don't fully understand how Wei's feature works.
>>>>> > >
>>>>> > > I guess I thought if you choose Hypervisor QoS, you do so on
>>>>> Compute
>>>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>>>>> > >
>>>>> > > From my thinking, you're root disk could be under Hypervisor QoS,
>>>>> but
>>>>> > your
>>>>> > > data disk could be under Storage QoS.
>>>>> > >
>>>>> > > Is that incorrect?
>>>>> > >
>>>>> > > Thanks
>>>>> > >
>>>>> > >
>>>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>>>>> > wrote:
>>>>> > >
>>>>> > >> Mike,
>>>>> > >>
>>>>> > >> As I understand these two patches, the throttled I?O settings are
>>>>> > applied
>>>>> > >> from the hypervisor side, and possibly defined in a compute
>>>>> offering
>>>>> > where
>>>>> > >> provisioned IOPS are defined on the storage side through disk
>>>>> > offerings. I
>>>>> > >> don't see how the management server could enforce this mutual
>>>>> exclusion
>>>>> > >> between provisioned IOPS and throttled I/O until a compute and
>>>>> disk
>>>>> > >> offering were selected. These selections come together when we
>>>>> deploy a
>>>>> > >> VM. Based on these assumptions, I would expect to see
>>>>> enforcement of
>>>>> > this
>>>>> > >> rule in UI at the time of VM creation/definition and on the server
>>>>> > side, as
>>>>> > >> part of the VM creation. It feels like any attempt to enforce
>>>>> this rule
>>>>> > >> when defining offering would be premature, and unnecessarily limit
>>>>> > >> flexibility. Are my assumptions and understanding correct?
>>>>> > >>
>>>>> > >> Thanks,
>>>>> > >> -John
>>>>> > >>
>>>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>>>>> > mike.tutkowski@solidfire.com>
>>>>> > >> wrote:
>>>>> > >>
>>>>> > >>> Hi John,
>>>>> > >>>
>>>>> > >>> So, maybe I'm wrong about this, but what I was thinking is that
>>>>> we'd
>>>>> > >> build
>>>>> > >>> two radio buttons into the Add Disk Offering dialog (we can
>>>>> ignore
>>>>> > >> Compute
>>>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>>>>> > >>>
>>>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
>>>>> Storage
>>>>> > >> QoS
>>>>> > >>> for the purpose of this e-mail.
>>>>> > >>>
>>>>> > >>> By default, neither radio button is selected and no QoS fields
>>>>> are
>>>>> > >> visible.
>>>>> > >>>
>>>>> > >>> If the user selects the Storage QoS radio button, then the
>>>>> Custom IOPS
>>>>> > >>> checkbox and the Min and Max text fields are made visible.
>>>>> > >>>
>>>>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>>>>> > button,
>>>>> > >>> then the Custom IOPS checkbox and the Min and Max text fields
>>>>> disappear
>>>>> > >> and
>>>>> > >>> are replaced by the two Hypervisor QoS text fields.
>>>>> > >>>
>>>>> > >>> This way, the user can choose neither QoS option or one of them
>>>>> or the
>>>>> > >>> other, but never both.
>>>>> > >>>
>>>>> > >>> On the API side, I was thinking of having logic in place when a
>>>>> request
>>>>> > >>> comes in to create a Disk Offering to confirm these fields are
>>>>> the way
>>>>> > we
>>>>> > >>> want them.
>>>>> > >>>
>>>>> > >>> Once we know the Disk Offering is in order, a user can create a
>>>>> data
>>>>> > disk
>>>>> > >>> from it. Since we checked the validity of the Disk Offering when
>>>>> it was
>>>>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>>>>> > Storage
>>>>> > >>> QoS in being utilized.
>>>>> > >>>
>>>>> > >>> Does that make sense or did I miss something?
>>>>> > >>>
>>>>> > >>> Thanks
>>>>> > >>>
>>>>> > >>>
>>>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>> > >> wrote:
>>>>> > >>>
>>>>> > >>>> Mike,
>>>>> > >>>>
>>>>> > >>>> Looking through the code, I am trying to understand how
>>>>> > >>>> CreateDiskOfferingCmd would have the context to identify the
>>>>> conflict.
>>>>> > >>>> Naively, it seems to me that this rule would need to be
>>>>> enforced when
>>>>> > a
>>>>> > >>>> virtual machine is being deployed. Looking through the code,
>>>>> it seems
>>>>> > >> like
>>>>> > >>>> we should add a private validateStorageQoS method to
>>>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and
>>>>> throws a
>>>>> > >>>> ResourceAllocationException when the QoS definitions are
>>>>> inconsistent.
>>>>> > >> We
>>>>> > >>>> would then add calls to it from each of the VM creation methods
>>>>> in the
>>>>> > >>>> service. Do this type of approach sound reasonable?
>>>>> > >>>>
>>>>> > >>>> Thanks,
>>>>> > >>>> -John
>>>>> > >>>>
>>>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>>>>> > >> mike.tutkowski@solidfire.com>
>>>>> > >>>> wrote:
>>>>> > >>>>
>>>>> > >>>>> Hi John,
>>>>> > >>>>>
>>>>> > >>>>> So, here's what I was planning to do. Of course feel free to
>>>>> correct
>>>>> > me
>>>>> > >>>> on
>>>>> > >>>>> this approach.
>>>>> > >>>>>
>>>>> > >>>>> I think it's OK if Wei merges his code into master and then I
>>>>> can
>>>>> > draw
>>>>> > >>>> from
>>>>> > >>>>> the main repo and merge master into mine locally.
>>>>> > >>>>>
>>>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI
>>>>> code
>>>>> > to
>>>>> > >>>> make
>>>>> > >>>>> it user friendly (toggle between these features on the Add Disk
>>>>> > >> Offering
>>>>> > >>>>> window).
>>>>> > >>>>>
>>>>> > >>>>> 2) I plan to write validation logic for the
>>>>> create-disk-offering API
>>>>> > >>>>> command which throws an exception if the rules are not
>>>>> followed (this
>>>>> > >>>>> should never be triggered from the GUI since the GUI will have
>>>>> > controls
>>>>> > >>>> in
>>>>> > >>>>> place to toggle between the one feature and the other).
>>>>> > >>>>>
>>>>> > >>>>> I'm not sure about documentation. I haven't had much
>>>>> experience with
>>>>> > it
>>>>> > >>>> on
>>>>> > >>>>> CloudStack projects yet.
>>>>> > >>>>>
>>>>> > >>>>> Thanks!
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>> > >>>> wrote:
>>>>> > >>>>>
>>>>> > >>>>>> Mike,
>>>>> > >>>>>>
>>>>> > >>>>>> Yes, these server-side rails need to be defined and
>>>>> implemented
>>>>> > before
>>>>> > >>>>>> either patch can be merged. From my perspective, I would
>>>>> like to
>>>>> > see
>>>>> > >>>> the
>>>>> > >>>>>> rule implemented in the hypervisor as part of the validation
>>>>> of the
>>>>> > >>>> virtual
>>>>> > >>>>>> machine definition. We also need to make sure that this
>>>>> mutual
>>>>> > >>>> exclusion
>>>>> > >>>>>> is documented. Do we usually include this type of
>>>>> documentation
>>>>> > with
>>>>> > >>>>>> patches of this nature?
>>>>> > >>>>>>
>>>>> > >>>>>> Thanks,
>>>>> > >>>>>> -John
>>>>> > >>>>>>
>>>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>>>> > >>>> mike.tutkowski@solidfire.com>
>>>>> > >>>>>> wrote:
>>>>> > >>>>>>
>>>>> > >>>>>>> Currently they are not yet implemented.
>>>>> > >>>>>>>
>>>>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>>>>> > usability
>>>>> > >>>>>>> standpoint, but the API must check for consistency and throw
>>>>> an
>>>>> > >>>> exception
>>>>> > >>>>>>> if necessary.
>>>>> > >>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
>>>>> jburwell@basho.com
>>>>> > >
>>>>> > >>>>>> wrote:
>>>>> > >>>>>>>
>>>>> > >>>>>>>> Mike,
>>>>> > >>>>>>>>
>>>>> > >>>>>>>> Are the checks only implemented in the UI?
>>>>> > >>>>>>>>
>>>>> > >>>>>>>> Thanks,
>>>>> > >>>>>>>> -John
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>
>>>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>>>> > >>>>>>>> <mi...@solidfire.com> wrote:
>>>>> > >>>>>>>>
>>>>> > >>>>>>>>> Hi John,
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>>>>> > >> exclusive.
>>>>> > >>>> We
>>>>> > >>>>>>>>> agree with you that only one should be active at a time.
>>>>> We plan
>>>>> > to
>>>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to
>>>>> turn
>>>>> > his
>>>>> > >>>>>>>> feature
>>>>> > >>>>>>>>> on and mine off and vice versa.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then
>>>>> I
>>>>> > update
>>>>> > >>>> and
>>>>> > >>>>>>>>> merge that I will be the person resolving merge conflicts
>>>>> in the
>>>>> > >>>>>>>> JavaScript
>>>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as
>>>>> opposed
>>>>> > to
>>>>> > >>>>>>>> putting
>>>>> > >>>>>>>>> that work on someone else.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Let me know what you think.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> Thanks!
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>>>>> > jburwell@basho.com
>>>>> > >>>
>>>>> > >>>>>>>> wrote:
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>> Mike,
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>>>> > >>>> mechanisms
>>>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the
>>>>> affected
>>>>> > FSs
>>>>> > >>>> been
>>>>> > >>>>>>>>>> updated with that decision?
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> In terms of merge timing, can you describe the
>>>>> dependencies
>>>>> > >> between
>>>>> > >>>>>> the
>>>>> > >>>>>>>>>> patches?
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> Thanks,
>>>>> > >>>>>>>>>> -John
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>> > >>>>>>>>>>
>>>>> > >>>>>>>>>>> No problem, John.
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up
>>>>> with a
>>>>> > >> new
>>>>> > >>>>>>>> patch
>>>>> > >>>>>>>>>>> file.
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>>> checked
>>>>> > >> in
>>>>> > >>>>>> and
>>>>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>> > >> jburwell@basho.com
>>>>> > >>>>>
>>>>> > >>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>> Mike,
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I
>>>>> am
>>>>> > >> offline
>>>>> > >>>>>> ATM,
>>>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> Sorry for the oversight,
>>>>> > >>>>>>>>>>>> -John
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>>> this :)
>>>>> > >> ),
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>>> areas
>>>>> > we
>>>>> > >>>>>> wanted
>>>>> > >>>>>>>>>>>>> addressed.
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>> > afternoon,
>>>>> > >>>> then
>>>>> > >>>>>>>>>> send
>>>>> > >>>>>>>>>>>> in
>>>>> > >>>>>>>>>>>>> another patch.
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>> Sure, that sounds good.
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>> > >>>> ustcweizhou@gmail.com
>>>>> > >>>>>>>
>>>>> > >>>>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> Hi Mike,
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
>>>>> in Java
>>>>> > >>>> code,
>>>>> > >>>>>>>>>>>> except
>>>>> > >>>>>>>>>>>>>>> the cloudstack UI.
>>>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>>> branch
>>>>> > >> into
>>>>> > >>>>>>>>>> master
>>>>> > >>>>>>>>>>>>>>> this
>>>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> -Wei
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>> > >>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> Hey John,
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>> object_store
>>>>> > >>>> branch,
>>>>> > >>>>>>>> but
>>>>> > >>>>>>>>>> -
>>>>> > >>>>>>>>>>>> as
>>>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
>>>>> the
>>>>> > >>>> SolidFire
>>>>> > >>>>>>>>>> branch
>>>>> > >>>>>>>>>>>>>>> into
>>>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes
>>>>> into
>>>>> > >> master.
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>>>>> this
>>>>> > merge
>>>>> > >>>>>>>>>> strategy.
>>>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>> > >>>>>>>> jburwell@basho.com>
>>>>> > >>>>>>>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> Mike,
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>> > >>>>>>>> disk_io_throttling,
>>>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
>>>>> number of
>>>>> > >>>>>>>> overlapping
>>>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between
>>>>> the
>>>>> > >> patches
>>>>> > >>>> to
>>>>> > >>>>>>>> be
>>>>> > >>>>>>>>>> as
>>>>> > >>>>>>>>>>>>>>>>> follows:
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>>> > >> SolidFire
>>>>> > >>>>>> are
>>>>> > >>>>>>>>>>>>>>> additive
>>>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>>> > dependency
>>>>> > >>>>>>>> between
>>>>> > >>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency
>>>>> graph, we
>>>>> > can
>>>>> > >>>>>>>>>> determine
>>>>> > >>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> Thanks,
>>>>> > >>>>>>>>>>>>>>>>> -John
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>> > >>>>>>>>>>>>>>>>> wrote:
>>>>> > >>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>>> into
>>>>> > his
>>>>> > >>>>>> branch
>>>>> > >>>>>>>>>>>>>>> before
>>>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>>> after his
>>>>> > >>>> merge
>>>>> > >>>>>>>> and
>>>>> > >>>>>>>>>>>>>>> we
>>>>> > >>>>>>>>>>>>>>>> can
>>>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>>> code in
>>>>> > >> the
>>>>> > >>>>>>>>>> storage
>>>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>>> > plug-in,
>>>>> > >>>>>> which
>>>>> > >>>>>>>>>>>>>>>>> currently
>>>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>>> > (although
>>>>> > >>>> it
>>>>> > >>>>>>>> does
>>>>> > >>>>>>>>>>>>>>> not
>>>>> > >>>>>>>>>>>>>>>>> know
>>>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>>> actually
>>>>> > in
>>>>> > >>>>>> use))
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed
>>>>> in the
>>>>> > url
>>>>> > >>>>>> field
>>>>> > >>>>>>>>>> (if
>>>>> > >>>>>>>>>>>>>>>> not
>>>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>>>>> stored in
>>>>> > the
>>>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>>> hypervisor
>>>>> > in
>>>>> > >>>>>>>>>>>>>>> question, we
>>>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>>> place
>>>>> > of
>>>>> > >>>> the
>>>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>>> > checks
>>>>> > >>>> for
>>>>> > >>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>> managed
>>>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>>> > hypervisor
>>>>> > >>>>>> data
>>>>> > >>>>>>>>>>>>>>>>> structure is
>>>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>>> executes to
>>>>> > >>>> attach
>>>>> > >>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>> volume.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
>>>>> to
>>>>> > >> detach a
>>>>> > >>>>>>>>>> volume,
>>>>> > >>>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>>>>> data
>>>>> > >>>> structure
>>>>> > >>>>>>>> is
>>>>> > >>>>>>>>>>>>>>>>> removed.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>>>>> IOPS in
>>>>> > 4.2
>>>>> > >>>>>>>> unless
>>>>> > >>>>>>>>>>>>>>> it is
>>>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
>>>>> If we
>>>>> > >> have
>>>>> > >>>>>> some
>>>>> > >>>>>>>>>>>>>>> idea,
>>>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> Thanks!
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>>> avoiding
>>>>> > >>>>>>>>>> implementation
>>>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>>> > field.
>>>>> > >> I
>>>>> > >>>>>>>> think
>>>>> > >>>>>>>>>>>>>>> we
>>>>> > >>>>>>>>>>>>>>>>> should
>>>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>>> regarding
>>>>> > >> the
>>>>> > >>>>>>>>>>>>>>> expected
>>>>> > >>>>>>>>>>>>>>>>> format
>>>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>>>>> details are
>>>>> > >> not
>>>>> > >>>>>>>>>>>>>>> returned on
>>>>> > >>>>>>>>>>>>>>>>> the
>>>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>> > listStoragePool
>>>>> > >>>>>>>>>> response.
>>>>> > >>>>>>>>>>>>>>>> Why
>>>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>>> useful
>>>>> > >> for
>>>>> > >>>>>>>>>> clients
>>>>> > >>>>>>>>>>>>>>> to
>>>>> > >>>>>>>>>>>>>>>> be
>>>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>>> field."
>>>>> > >>>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>>> here.
>>>>> > >>>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>> > >> Offering-by-Disk
>>>>> > >>>>>>>>>> Offering
>>>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>>> to be
>>>>> > >> able
>>>>> > >>>>>> to
>>>>> > >>>>>>>>>>>>>>>> associate
>>>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>> disk_offering_details
>>>>> > >> table.
>>>>> > >>>>>>>> Maybe
>>>>> > >>>>>>>>>>>>>>> it
>>>>> > >>>>>>>>>>>>>>>>> could
>>>>> > >>>>>>>>>>>>>>>>>>>> go there?
>>>>> > >>>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
>>>>> Burst IOPS
>>>>> > in
>>>>> > >>>> the
>>>>> > >>>>>>>> GUI
>>>>> > >>>>>>>>>>>>>>> if
>>>>> > >>>>>>>>>>>>>>>>> it's
>>>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>>>>> the DB.
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>> >
>>>>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>>>>>>>>> cloud<
>>>>> > http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>>
>>>>> > >>>>>>>>>>>>> --
>>>>> > >>>>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>>
>>>>> > >>>>>>>>>>> --
>>>>> > >>>>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>> >
>>>>> > >>>>>>>>>>> *â„¢*
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>>
>>>>> > >>>>>>>>> --
>>>>> > >>>>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>>>> o: 303.746.7302
>>>>> > >>>>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>>>> *â„¢*
>>>>> > >>>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>>
>>>>> > >>>>>>> --
>>>>> > >>>>>>> *Mike Tutkowski*
>>>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>>>> o: 303.746.7302
>>>>> > >>>>>>> Advancing the way the world uses the
>>>>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>>>> *â„¢*
>>>>> > >>>>>>
>>>>> > >>>>>>
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> --
>>>>> > >>>>> *Mike Tutkowski*
>>>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>>>> e: mike.tutkowski@solidfire.com
>>>>> > >>>>> o: 303.746.7302
>>>>> > >>>>> Advancing the way the world uses the
>>>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>>>> *â„¢*
>>>>> > >>>>
>>>>> > >>>>
>>>>> > >>>
>>>>> > >>>
>>>>> > >>> --
>>>>> > >>> *Mike Tutkowski*
>>>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > >>> e: mike.tutkowski@solidfire.com
>>>>> > >>> o: 303.746.7302
>>>>> > >>> Advancing the way the world uses the
>>>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > >>> *â„¢*
>>>>> > >>
>>>>> > >>
>>>>> > >
>>>>> > >
>>>>> > > --
>>>>> > > *Mike Tutkowski*
>>>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > > e: mike.tutkowski@solidfire.com
>>>>> > > o: 303.746.7302
>>>>> > > Advancing the way the world uses the
>>>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > > *â„¢*
>>>>> >
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
As far as compressing read and write IOPS fields into one value (one for
Min and one for Max), that is Wei's area as my feature does not distinguish
between read and write IOPS.
On Wed, Jun 12, 2013 at 4:11 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> I see, John.
>
> That is an interesting idea.
>
> We'd also have to change the storage allocator(s) to favor QoS-supported
> storage if those fields are filled in.
>
>
> On Wed, Jun 12, 2013 at 4:09 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> My thought is that we present the min/max IOPS fields for read/write
>> operations for all compute and disk offerings. When the VM is allocated,
>> we determine the best way to fulfill that QoS. It sounds like storage
>> level guarantees would always be preferred. If no storage is available to
>> guarantee the QoS, the allocator falls back to using hypervisor. This
>> approach only works if summing the discrete read/write min/max values to
>> get to min/max total IOPS would be acceptable.
>>
>> Thanks,
>> -John
>>
>>
>> On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> I hate to say it, but I believe Storage QoS with a Min and Max will
>> always be optimal over hypervisor rate limiting.
>>
>> The only time you'd want to use hypervisor rate limiting is if storage
>> QoS is not available.
>>
>> We currently have no way to know what storage the root or data disk will
>> be deployed to, so we have to present the fields in all situation.
>>
>> This is because of - in my opinion - the somewhat flawed way CloudStack
>> handles storage tags. You are free to enter in any text there and we don't
>> know what it maps to when the Compute or Disk Offering dialog is up.
>>
>>
>> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> Please see my comments/questions in-line below.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> Hi Wei,
>>>
>>> So, not entirely sure I follow.
>>>
>>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>>
>>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>>> QoS for the purpose of this e-mail.
>>>
>>> By default, neither radio button is selected and no QoS fields are
>>> visible.
>>>
>>>
>>> I prefer a None option selected by default. I find the no button
>>> selected in a Radio button group to be confusing.
>>>
>>> Also, do we have a mechanism to determine whether or not this radio
>>> button group should be displayed at all? If there are no devices or
>>> hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>>>
>>>
>>> If the user selects the Storage QoS radio button, then the Custom IOPS
>>> checkbox and the Min and Max text fields are made visible.
>>>
>>> If the user changes his mind and selects the Hypervisor QoS radio
>>> button, then the Custom IOPS checkbox and the Min and Max text fields
>>> disappear and are replaced by the two Hypervisor QoS text fields.
>>>
>>> This way, the user can choose neither QoS option or one of them or the
>>> other, but never both.
>>>
>>> On the API side, I was thinking of having logic in place when a request
>>> comes in to create a Disk Offering to confirm these fields are the way we
>>> want them.
>>>
>>> Once we know the Disk Offering is in order, a user can create a data
>>> disk from it. Since we checked the validity of the Disk Offering when it
>>> was created, the VM should never be asked to use Hypervisor QoS when
>>> Storage QoS in being utilized.
>>>
>>>
>>> Overall, this model sounds very reasonable. Wondering aloud: Would be
>>> possible to have the UI simply ask for a storage QoS, and let the system
>>> pick the optional implementation. For example, if a VM is attached to KVM
>>> and SolidFire is available, we chose to fulfill the QoS using the storage
>>> device. This approach would also allow us to fallback in the event that we
>>> can't allocate storage for the QoS, but we have hypervisor capacity. Could
>>> we always deterministically determine the optimal implementation of the QoS
>>> at time of VM deployment?
>>>
>>>
>>> Thanks!
>>>
>>>
>>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
>>>
>>>> Mike, John
>>>>
>>>> The Disk I/O Throttling works like:
>>>>
>>>> (1) For User VM,
>>>> (1.1) root disks: service offering -> default value in global
>>>> configuration
>>>> (1.2) data disks: disk offering -> default value in global configuration
>>>>
>>>> (2) For System VM
>>>> root disks: service offering
>>>>
>>>> -Wei
>>>>
>>>>
>>>> 2013/6/12 John Burwell <jb...@basho.com>
>>>>
>>>> > Mike,
>>>> >
>>>> > That is one possibility. The other possibility is that hypervisor is
>>>> > going to throttle I/O on all disks attached. Therefore, we need
>>>> answers to
>>>> > the following questions:
>>>> >
>>>> > 1. Is I/O throttling applied to the root disk or all disks
>>>> > attached to the VM?
>>>> > 2. If I/O throttling is applied to all disks, how is the
>>>> > throttling distributed amongst the disks if only one read/write value
>>>> is
>>>> > defined?
>>>> >
>>>> > Thanks,
>>>> > -John
>>>> >
>>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>> > wrote:
>>>> >
>>>> > > Hey John,
>>>> > >
>>>> > > Perhaps I don't fully understand how Wei's feature works.
>>>> > >
>>>> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
>>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>>>> > >
>>>> > > From my thinking, you're root disk could be under Hypervisor QoS,
>>>> but
>>>> > your
>>>> > > data disk could be under Storage QoS.
>>>> > >
>>>> > > Is that incorrect?
>>>> > >
>>>> > > Thanks
>>>> > >
>>>> > >
>>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>>>> > wrote:
>>>> > >
>>>> > >> Mike,
>>>> > >>
>>>> > >> As I understand these two patches, the throttled I?O settings are
>>>> > applied
>>>> > >> from the hypervisor side, and possibly defined in a compute
>>>> offering
>>>> > where
>>>> > >> provisioned IOPS are defined on the storage side through disk
>>>> > offerings. I
>>>> > >> don't see how the management server could enforce this mutual
>>>> exclusion
>>>> > >> between provisioned IOPS and throttled I/O until a compute and disk
>>>> > >> offering were selected. These selections come together when we
>>>> deploy a
>>>> > >> VM. Based on these assumptions, I would expect to see enforcement
>>>> of
>>>> > this
>>>> > >> rule in UI at the time of VM creation/definition and on the server
>>>> > side, as
>>>> > >> part of the VM creation. It feels like any attempt to enforce
>>>> this rule
>>>> > >> when defining offering would be premature, and unnecessarily limit
>>>> > >> flexibility. Are my assumptions and understanding correct?
>>>> > >>
>>>> > >> Thanks,
>>>> > >> -John
>>>> > >>
>>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>>>> > mike.tutkowski@solidfire.com>
>>>> > >> wrote:
>>>> > >>
>>>> > >>> Hi John,
>>>> > >>>
>>>> > >>> So, maybe I'm wrong about this, but what I was thinking is that
>>>> we'd
>>>> > >> build
>>>> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
>>>> > >> Compute
>>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>>>> > >>>
>>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
>>>> Storage
>>>> > >> QoS
>>>> > >>> for the purpose of this e-mail.
>>>> > >>>
>>>> > >>> By default, neither radio button is selected and no QoS fields are
>>>> > >> visible.
>>>> > >>>
>>>> > >>> If the user selects the Storage QoS radio button, then the Custom
>>>> IOPS
>>>> > >>> checkbox and the Min and Max text fields are made visible.
>>>> > >>>
>>>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>>>> > button,
>>>> > >>> then the Custom IOPS checkbox and the Min and Max text fields
>>>> disappear
>>>> > >> and
>>>> > >>> are replaced by the two Hypervisor QoS text fields.
>>>> > >>>
>>>> > >>> This way, the user can choose neither QoS option or one of them
>>>> or the
>>>> > >>> other, but never both.
>>>> > >>>
>>>> > >>> On the API side, I was thinking of having logic in place when a
>>>> request
>>>> > >>> comes in to create a Disk Offering to confirm these fields are
>>>> the way
>>>> > we
>>>> > >>> want them.
>>>> > >>>
>>>> > >>> Once we know the Disk Offering is in order, a user can create a
>>>> data
>>>> > disk
>>>> > >>> from it. Since we checked the validity of the Disk Offering when
>>>> it was
>>>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>>>> > Storage
>>>> > >>> QoS in being utilized.
>>>> > >>>
>>>> > >>> Does that make sense or did I miss something?
>>>> > >>>
>>>> > >>> Thanks
>>>> > >>>
>>>> > >>>
>>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jburwell@basho.com
>>>> >
>>>> > >> wrote:
>>>> > >>>
>>>> > >>>> Mike,
>>>> > >>>>
>>>> > >>>> Looking through the code, I am trying to understand how
>>>> > >>>> CreateDiskOfferingCmd would have the context to identify the
>>>> conflict.
>>>> > >>>> Naively, it seems to me that this rule would need to be enforced
>>>> when
>>>> > a
>>>> > >>>> virtual machine is being deployed. Looking through the code, it
>>>> seems
>>>> > >> like
>>>> > >>>> we should add a private validateStorageQoS method to
>>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and
>>>> throws a
>>>> > >>>> ResourceAllocationException when the QoS definitions are
>>>> inconsistent.
>>>> > >> We
>>>> > >>>> would then add calls to it from each of the VM creation methods
>>>> in the
>>>> > >>>> service. Do this type of approach sound reasonable?
>>>> > >>>>
>>>> > >>>> Thanks,
>>>> > >>>> -John
>>>> > >>>>
>>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>>>> > >> mike.tutkowski@solidfire.com>
>>>> > >>>> wrote:
>>>> > >>>>
>>>> > >>>>> Hi John,
>>>> > >>>>>
>>>> > >>>>> So, here's what I was planning to do. Of course feel free to
>>>> correct
>>>> > me
>>>> > >>>> on
>>>> > >>>>> this approach.
>>>> > >>>>>
>>>> > >>>>> I think it's OK if Wei merges his code into master and then I
>>>> can
>>>> > draw
>>>> > >>>> from
>>>> > >>>>> the main repo and merge master into mine locally.
>>>> > >>>>>
>>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI
>>>> code
>>>> > to
>>>> > >>>> make
>>>> > >>>>> it user friendly (toggle between these features on the Add Disk
>>>> > >> Offering
>>>> > >>>>> window).
>>>> > >>>>>
>>>> > >>>>> 2) I plan to write validation logic for the
>>>> create-disk-offering API
>>>> > >>>>> command which throws an exception if the rules are not followed
>>>> (this
>>>> > >>>>> should never be triggered from the GUI since the GUI will have
>>>> > controls
>>>> > >>>> in
>>>> > >>>>> place to toggle between the one feature and the other).
>>>> > >>>>>
>>>> > >>>>> I'm not sure about documentation. I haven't had much experience
>>>> with
>>>> > it
>>>> > >>>> on
>>>> > >>>>> CloudStack projects yet.
>>>> > >>>>>
>>>> > >>>>> Thanks!
>>>> > >>>>>
>>>> > >>>>>
>>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <
>>>> jburwell@basho.com>
>>>> > >>>> wrote:
>>>> > >>>>>
>>>> > >>>>>> Mike,
>>>> > >>>>>>
>>>> > >>>>>> Yes, these server-side rails need to be defined and implemented
>>>> > before
>>>> > >>>>>> either patch can be merged. From my perspective, I would like
>>>> to
>>>> > see
>>>> > >>>> the
>>>> > >>>>>> rule implemented in the hypervisor as part of the validation
>>>> of the
>>>> > >>>> virtual
>>>> > >>>>>> machine definition. We also need to make sure that this mutual
>>>> > >>>> exclusion
>>>> > >>>>>> is documented. Do we usually include this type of
>>>> documentation
>>>> > with
>>>> > >>>>>> patches of this nature?
>>>> > >>>>>>
>>>> > >>>>>> Thanks,
>>>> > >>>>>> -John
>>>> > >>>>>>
>>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>>> > >>>> mike.tutkowski@solidfire.com>
>>>> > >>>>>> wrote:
>>>> > >>>>>>
>>>> > >>>>>>> Currently they are not yet implemented.
>>>> > >>>>>>>
>>>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>>>> > usability
>>>> > >>>>>>> standpoint, but the API must check for consistency and throw
>>>> an
>>>> > >>>> exception
>>>> > >>>>>>> if necessary.
>>>> > >>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
>>>> jburwell@basho.com
>>>> > >
>>>> > >>>>>> wrote:
>>>> > >>>>>>>
>>>> > >>>>>>>> Mike,
>>>> > >>>>>>>>
>>>> > >>>>>>>> Are the checks only implemented in the UI?
>>>> > >>>>>>>>
>>>> > >>>>>>>> Thanks,
>>>> > >>>>>>>> -John
>>>> > >>>>>>>>
>>>> > >>>>>>>>
>>>> > >>>>>>>>
>>>> > >>>>>>>>
>>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>>> > >>>>>>>> <mi...@solidfire.com> wrote:
>>>> > >>>>>>>>
>>>> > >>>>>>>>> Hi John,
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>>>> > >> exclusive.
>>>> > >>>> We
>>>> > >>>>>>>>> agree with you that only one should be active at a time. We
>>>> plan
>>>> > to
>>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to
>>>> turn
>>>> > his
>>>> > >>>>>>>> feature
>>>> > >>>>>>>>> on and mine off and vice versa.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
>>>> > update
>>>> > >>>> and
>>>> > >>>>>>>>> merge that I will be the person resolving merge conflicts
>>>> in the
>>>> > >>>>>>>> JavaScript
>>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as
>>>> opposed
>>>> > to
>>>> > >>>>>>>> putting
>>>> > >>>>>>>>> that work on someone else.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Let me know what you think.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> Thanks!
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>>>> > jburwell@basho.com
>>>> > >>>
>>>> > >>>>>>>> wrote:
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>> Mike,
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>>> > >>>> mechanisms
>>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the
>>>> affected
>>>> > FSs
>>>> > >>>> been
>>>> > >>>>>>>>>> updated with that decision?
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
>>>> > >> between
>>>> > >>>>>> the
>>>> > >>>>>>>>>> patches?
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> Thanks,
>>>> > >>>>>>>>>> -John
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>>>> > >>>>>>>>>>
>>>> > >>>>>>>>>>> No problem, John.
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up
>>>> with a
>>>> > >> new
>>>> > >>>>>>>> patch
>>>> > >>>>>>>>>>> file.
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>> checked
>>>> > >> in
>>>> > >>>>>> and
>>>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>> > >> jburwell@basho.com
>>>> > >>>>>
>>>> > >>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>> Mike,
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>>> > >> offline
>>>> > >>>>>> ATM,
>>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> Sorry for the oversight,
>>>> > >>>>>>>>>>>> -John
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>> this :)
>>>> > >> ),
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>> areas
>>>> > we
>>>> > >>>>>> wanted
>>>> > >>>>>>>>>>>>> addressed.
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>> > afternoon,
>>>> > >>>> then
>>>> > >>>>>>>>>> send
>>>> > >>>>>>>>>>>> in
>>>> > >>>>>>>>>>>>> another patch.
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>> Sure, that sounds good.
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>> > >>>> ustcweizhou@gmail.com
>>>> > >>>>>>>
>>>> > >>>>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> Hi Mike,
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
>>>> in Java
>>>> > >>>> code,
>>>> > >>>>>>>>>>>> except
>>>> > >>>>>>>>>>>>>>> the cloudstack UI.
>>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>> branch
>>>> > >> into
>>>> > >>>>>>>>>> master
>>>> > >>>>>>>>>>>>>>> this
>>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> -Wei
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>> > >>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> Hey John,
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>> object_store
>>>> > >>>> branch,
>>>> > >>>>>>>> but
>>>> > >>>>>>>>>> -
>>>> > >>>>>>>>>>>> as
>>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>> > >>>> SolidFire
>>>> > >>>>>>>>>> branch
>>>> > >>>>>>>>>>>>>>> into
>>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>>> > >> master.
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>>>> this
>>>> > merge
>>>> > >>>>>>>>>> strategy.
>>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>> > >>>>>>>> jburwell@basho.com>
>>>> > >>>>>>>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> Mike,
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>> > >>>>>>>> disk_io_throttling,
>>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>>> of
>>>> > >>>>>>>> overlapping
>>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>>> > >> patches
>>>> > >>>> to
>>>> > >>>>>>>> be
>>>> > >>>>>>>>>> as
>>>> > >>>>>>>>>>>>>>>>> follows:
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>> > >> SolidFire
>>>> > >>>>>> are
>>>> > >>>>>>>>>>>>>>> additive
>>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>> > dependency
>>>> > >>>>>>>> between
>>>> > >>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>>> we
>>>> > can
>>>> > >>>>>>>>>> determine
>>>> > >>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> Thanks,
>>>> > >>>>>>>>>>>>>>>>> -John
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> > >>>>>>>>>>>>>>>>> wrote:
>>>> > >>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>> into
>>>> > his
>>>> > >>>>>> branch
>>>> > >>>>>>>>>>>>>>> before
>>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>> after his
>>>> > >>>> merge
>>>> > >>>>>>>> and
>>>> > >>>>>>>>>>>>>>> we
>>>> > >>>>>>>>>>>>>>>> can
>>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>> code in
>>>> > >> the
>>>> > >>>>>>>>>> storage
>>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>> > plug-in,
>>>> > >>>>>> which
>>>> > >>>>>>>>>>>>>>>>> currently
>>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>> > (although
>>>> > >>>> it
>>>> > >>>>>>>> does
>>>> > >>>>>>>>>>>>>>> not
>>>> > >>>>>>>>>>>>>>>>> know
>>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>> actually
>>>> > in
>>>> > >>>>>> use))
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>>> the
>>>> > url
>>>> > >>>>>> field
>>>> > >>>>>>>>>> (if
>>>> > >>>>>>>>>>>>>>>> not
>>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>>>> stored in
>>>> > the
>>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>> hypervisor
>>>> > in
>>>> > >>>>>>>>>>>>>>> question, we
>>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>> place
>>>> > of
>>>> > >>>> the
>>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>> > checks
>>>> > >>>> for
>>>> > >>>>>>>> the
>>>> > >>>>>>>>>>>>>>>> managed
>>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>> > hypervisor
>>>> > >>>>>> data
>>>> > >>>>>>>>>>>>>>>>> structure is
>>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>> executes to
>>>> > >>>> attach
>>>> > >>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>> volume.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>>> > >> detach a
>>>> > >>>>>>>>>> volume,
>>>> > >>>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>>>> data
>>>> > >>>> structure
>>>> > >>>>>>>> is
>>>> > >>>>>>>>>>>>>>>>> removed.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>>>> IOPS in
>>>> > 4.2
>>>> > >>>>>>>> unless
>>>> > >>>>>>>>>>>>>>> it is
>>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
>>>> If we
>>>> > >> have
>>>> > >>>>>> some
>>>> > >>>>>>>>>>>>>>> idea,
>>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> Thanks!
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>> avoiding
>>>> > >>>>>>>>>> implementation
>>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>> > field.
>>>> > >> I
>>>> > >>>>>>>> think
>>>> > >>>>>>>>>>>>>>> we
>>>> > >>>>>>>>>>>>>>>>> should
>>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>> regarding
>>>> > >> the
>>>> > >>>>>>>>>>>>>>> expected
>>>> > >>>>>>>>>>>>>>>>> format
>>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>>>> details are
>>>> > >> not
>>>> > >>>>>>>>>>>>>>> returned on
>>>> > >>>>>>>>>>>>>>>>> the
>>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>> > listStoragePool
>>>> > >>>>>>>>>> response.
>>>> > >>>>>>>>>>>>>>>> Why
>>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>> useful
>>>> > >> for
>>>> > >>>>>>>>>> clients
>>>> > >>>>>>>>>>>>>>> to
>>>> > >>>>>>>>>>>>>>>> be
>>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>> field."
>>>> > >>>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>> here.
>>>> > >>>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>> > >> Offering-by-Disk
>>>> > >>>>>>>>>> Offering
>>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>> to be
>>>> > >> able
>>>> > >>>>>> to
>>>> > >>>>>>>>>>>>>>>> associate
>>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>> disk_offering_details
>>>> > >> table.
>>>> > >>>>>>>> Maybe
>>>> > >>>>>>>>>>>>>>> it
>>>> > >>>>>>>>>>>>>>>>> could
>>>> > >>>>>>>>>>>>>>>>>>>> go there?
>>>> > >>>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>>> IOPS
>>>> > in
>>>> > >>>> the
>>>> > >>>>>>>> GUI
>>>> > >>>>>>>>>>>>>>> if
>>>> > >>>>>>>>>>>>>>>>> it's
>>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>>>> the DB.
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>>>>>>>>> cloud<
>>>> > http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>>
>>>> > >>>>>>>>>>>>> --
>>>> > >>>>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>>
>>>> > >>>>>>>>>>> --
>>>> > >>>>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>>>> *â„¢*
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>
>>>> > >>>>>>>>>
>>>> > >>>>>>>>> --
>>>> > >>>>>>>>> *Mike Tutkowski*
>>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>>>> o: 303.746.7302
>>>> > >>>>>>>>> Advancing the way the world uses the
>>>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>>>> *â„¢*
>>>> > >>>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>>
>>>> > >>>>>>> --
>>>> > >>>>>>> *Mike Tutkowski*
>>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>>>> o: 303.746.7302
>>>> > >>>>>>> Advancing the way the world uses the
>>>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>>>> *â„¢*
>>>> > >>>>>>
>>>> > >>>>>>
>>>> > >>>>>
>>>> > >>>>>
>>>> > >>>>> --
>>>> > >>>>> *Mike Tutkowski*
>>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>>>> e: mike.tutkowski@solidfire.com
>>>> > >>>>> o: 303.746.7302
>>>> > >>>>> Advancing the way the world uses the
>>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>>>> *â„¢*
>>>> > >>>>
>>>> > >>>>
>>>> > >>>
>>>> > >>>
>>>> > >>> --
>>>> > >>> *Mike Tutkowski*
>>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> > >>> e: mike.tutkowski@solidfire.com
>>>> > >>> o: 303.746.7302
>>>> > >>> Advancing the way the world uses the
>>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > >>> *â„¢*
>>>> > >>
>>>> > >>
>>>> > >
>>>> > >
>>>> > > --
>>>> > > *Mike Tutkowski*
>>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>>> > > e: mike.tutkowski@solidfire.com
>>>> > > o: 303.746.7302
>>>> > > Advancing the way the world uses the
>>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > > *â„¢*
>>>> >
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I see, John.
That is an interesting idea.
We'd also have to change the storage allocator(s) to favor QoS-supported
storage if those fields are filled in.
On Wed, Jun 12, 2013 at 4:09 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> My thought is that we present the min/max IOPS fields for read/write
> operations for all compute and disk offerings. When the VM is allocated,
> we determine the best way to fulfill that QoS. It sounds like storage
> level guarantees would always be preferred. If no storage is available to
> guarantee the QoS, the allocator falls back to using hypervisor. This
> approach only works if summing the discrete read/write min/max values to
> get to min/max total IOPS would be acceptable.
>
> Thanks,
> -John
>
>
> On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> I hate to say it, but I believe Storage QoS with a Min and Max will always
> be optimal over hypervisor rate limiting.
>
> The only time you'd want to use hypervisor rate limiting is if storage QoS
> is not available.
>
> We currently have no way to know what storage the root or data disk will
> be deployed to, so we have to present the fields in all situation.
>
> This is because of - in my opinion - the somewhat flawed way CloudStack
> handles storage tags. You are free to enter in any text there and we don't
> know what it maps to when the Compute or Disk Offering dialog is up.
>
>
> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Please see my comments/questions in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> Hi Wei,
>>
>> So, not entirely sure I follow.
>>
>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>
>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>> QoS for the purpose of this e-mail.
>>
>> By default, neither radio button is selected and no QoS fields are
>> visible.
>>
>>
>> I prefer a None option selected by default. I find the no button
>> selected in a Radio button group to be confusing.
>>
>> Also, do we have a mechanism to determine whether or not this radio
>> button group should be displayed at all? If there are no devices or
>> hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>>
>>
>> If the user selects the Storage QoS radio button, then the Custom IOPS
>> checkbox and the Min and Max text fields are made visible.
>>
>> If the user changes his mind and selects the Hypervisor QoS radio button,
>> then the Custom IOPS checkbox and the Min and Max text fields disappear and
>> are replaced by the two Hypervisor QoS text fields.
>>
>> This way, the user can choose neither QoS option or one of them or the
>> other, but never both.
>>
>> On the API side, I was thinking of having logic in place when a request
>> comes in to create a Disk Offering to confirm these fields are the way we
>> want them.
>>
>> Once we know the Disk Offering is in order, a user can create a data disk
>> from it. Since we checked the validity of the Disk Offering when it was
>> created, the VM should never be asked to use Hypervisor QoS when Storage
>> QoS in being utilized.
>>
>>
>> Overall, this model sounds very reasonable. Wondering aloud: Would be
>> possible to have the UI simply ask for a storage QoS, and let the system
>> pick the optional implementation. For example, if a VM is attached to KVM
>> and SolidFire is available, we chose to fulfill the QoS using the storage
>> device. This approach would also allow us to fallback in the event that we
>> can't allocate storage for the QoS, but we have hypervisor capacity. Could
>> we always deterministically determine the optimal implementation of the QoS
>> at time of VM deployment?
>>
>>
>> Thanks!
>>
>>
>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> Mike, John
>>>
>>> The Disk I/O Throttling works like:
>>>
>>> (1) For User VM,
>>> (1.1) root disks: service offering -> default value in global
>>> configuration
>>> (1.2) data disks: disk offering -> default value in global configuration
>>>
>>> (2) For System VM
>>> root disks: service offering
>>>
>>> -Wei
>>>
>>>
>>> 2013/6/12 John Burwell <jb...@basho.com>
>>>
>>> > Mike,
>>> >
>>> > That is one possibility. The other possibility is that hypervisor is
>>> > going to throttle I/O on all disks attached. Therefore, we need
>>> answers to
>>> > the following questions:
>>> >
>>> > 1. Is I/O throttling applied to the root disk or all disks
>>> > attached to the VM?
>>> > 2. If I/O throttling is applied to all disks, how is the
>>> > throttling distributed amongst the disks if only one read/write value
>>> is
>>> > defined?
>>> >
>>> > Thanks,
>>> > -John
>>> >
>>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>> > wrote:
>>> >
>>> > > Hey John,
>>> > >
>>> > > Perhaps I don't fully understand how Wei's feature works.
>>> > >
>>> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
>>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>>> > >
>>> > > From my thinking, you're root disk could be under Hypervisor QoS, but
>>> > your
>>> > > data disk could be under Storage QoS.
>>> > >
>>> > > Is that incorrect?
>>> > >
>>> > > Thanks
>>> > >
>>> > >
>>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>>> > wrote:
>>> > >
>>> > >> Mike,
>>> > >>
>>> > >> As I understand these two patches, the throttled I?O settings are
>>> > applied
>>> > >> from the hypervisor side, and possibly defined in a compute offering
>>> > where
>>> > >> provisioned IOPS are defined on the storage side through disk
>>> > offerings. I
>>> > >> don't see how the management server could enforce this mutual
>>> exclusion
>>> > >> between provisioned IOPS and throttled I/O until a compute and disk
>>> > >> offering were selected. These selections come together when we
>>> deploy a
>>> > >> VM. Based on these assumptions, I would expect to see enforcement
>>> of
>>> > this
>>> > >> rule in UI at the time of VM creation/definition and on the server
>>> > side, as
>>> > >> part of the VM creation. It feels like any attempt to enforce this
>>> rule
>>> > >> when defining offering would be premature, and unnecessarily limit
>>> > >> flexibility. Are my assumptions and understanding correct?
>>> > >>
>>> > >> Thanks,
>>> > >> -John
>>> > >>
>>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>>> > mike.tutkowski@solidfire.com>
>>> > >> wrote:
>>> > >>
>>> > >>> Hi John,
>>> > >>>
>>> > >>> So, maybe I'm wrong about this, but what I was thinking is that
>>> we'd
>>> > >> build
>>> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
>>> > >> Compute
>>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>>> > >>>
>>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
>>> Storage
>>> > >> QoS
>>> > >>> for the purpose of this e-mail.
>>> > >>>
>>> > >>> By default, neither radio button is selected and no QoS fields are
>>> > >> visible.
>>> > >>>
>>> > >>> If the user selects the Storage QoS radio button, then the Custom
>>> IOPS
>>> > >>> checkbox and the Min and Max text fields are made visible.
>>> > >>>
>>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>>> > button,
>>> > >>> then the Custom IOPS checkbox and the Min and Max text fields
>>> disappear
>>> > >> and
>>> > >>> are replaced by the two Hypervisor QoS text fields.
>>> > >>>
>>> > >>> This way, the user can choose neither QoS option or one of them or
>>> the
>>> > >>> other, but never both.
>>> > >>>
>>> > >>> On the API side, I was thinking of having logic in place when a
>>> request
>>> > >>> comes in to create a Disk Offering to confirm these fields are the
>>> way
>>> > we
>>> > >>> want them.
>>> > >>>
>>> > >>> Once we know the Disk Offering is in order, a user can create a
>>> data
>>> > disk
>>> > >>> from it. Since we checked the validity of the Disk Offering when
>>> it was
>>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>>> > Storage
>>> > >>> QoS in being utilized.
>>> > >>>
>>> > >>> Does that make sense or did I miss something?
>>> > >>>
>>> > >>> Thanks
>>> > >>>
>>> > >>>
>>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>>> > >> wrote:
>>> > >>>
>>> > >>>> Mike,
>>> > >>>>
>>> > >>>> Looking through the code, I am trying to understand how
>>> > >>>> CreateDiskOfferingCmd would have the context to identify the
>>> conflict.
>>> > >>>> Naively, it seems to me that this rule would need to be enforced
>>> when
>>> > a
>>> > >>>> virtual machine is being deployed. Looking through the code, it
>>> seems
>>> > >> like
>>> > >>>> we should add a private validateStorageQoS method to
>>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws
>>> a
>>> > >>>> ResourceAllocationException when the QoS definitions are
>>> inconsistent.
>>> > >> We
>>> > >>>> would then add calls to it from each of the VM creation methods
>>> in the
>>> > >>>> service. Do this type of approach sound reasonable?
>>> > >>>>
>>> > >>>> Thanks,
>>> > >>>> -John
>>> > >>>>
>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>>> > >> mike.tutkowski@solidfire.com>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>> Hi John,
>>> > >>>>>
>>> > >>>>> So, here's what I was planning to do. Of course feel free to
>>> correct
>>> > me
>>> > >>>> on
>>> > >>>>> this approach.
>>> > >>>>>
>>> > >>>>> I think it's OK if Wei merges his code into master and then I can
>>> > draw
>>> > >>>> from
>>> > >>>>> the main repo and merge master into mine locally.
>>> > >>>>>
>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI
>>> code
>>> > to
>>> > >>>> make
>>> > >>>>> it user friendly (toggle between these features on the Add Disk
>>> > >> Offering
>>> > >>>>> window).
>>> > >>>>>
>>> > >>>>> 2) I plan to write validation logic for the create-disk-offering
>>> API
>>> > >>>>> command which throws an exception if the rules are not followed
>>> (this
>>> > >>>>> should never be triggered from the GUI since the GUI will have
>>> > controls
>>> > >>>> in
>>> > >>>>> place to toggle between the one feature and the other).
>>> > >>>>>
>>> > >>>>> I'm not sure about documentation. I haven't had much experience
>>> with
>>> > it
>>> > >>>> on
>>> > >>>>> CloudStack projects yet.
>>> > >>>>>
>>> > >>>>> Thanks!
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <
>>> jburwell@basho.com>
>>> > >>>> wrote:
>>> > >>>>>
>>> > >>>>>> Mike,
>>> > >>>>>>
>>> > >>>>>> Yes, these server-side rails need to be defined and implemented
>>> > before
>>> > >>>>>> either patch can be merged. From my perspective, I would like
>>> to
>>> > see
>>> > >>>> the
>>> > >>>>>> rule implemented in the hypervisor as part of the validation of
>>> the
>>> > >>>> virtual
>>> > >>>>>> machine definition. We also need to make sure that this mutual
>>> > >>>> exclusion
>>> > >>>>>> is documented. Do we usually include this type of documentation
>>> > with
>>> > >>>>>> patches of this nature?
>>> > >>>>>>
>>> > >>>>>> Thanks,
>>> > >>>>>> -John
>>> > >>>>>>
>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>> > >>>> mike.tutkowski@solidfire.com>
>>> > >>>>>> wrote:
>>> > >>>>>>
>>> > >>>>>>> Currently they are not yet implemented.
>>> > >>>>>>>
>>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>>> > usability
>>> > >>>>>>> standpoint, but the API must check for consistency and throw an
>>> > >>>> exception
>>> > >>>>>>> if necessary.
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
>>> jburwell@basho.com
>>> > >
>>> > >>>>>> wrote:
>>> > >>>>>>>
>>> > >>>>>>>> Mike,
>>> > >>>>>>>>
>>> > >>>>>>>> Are the checks only implemented in the UI?
>>> > >>>>>>>>
>>> > >>>>>>>> Thanks,
>>> > >>>>>>>> -John
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>> > >>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>
>>> > >>>>>>>>> Hi John,
>>> > >>>>>>>>>
>>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>>> > >> exclusive.
>>> > >>>> We
>>> > >>>>>>>>> agree with you that only one should be active at a time. We
>>> plan
>>> > to
>>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to
>>> turn
>>> > his
>>> > >>>>>>>> feature
>>> > >>>>>>>>> on and mine off and vice versa.
>>> > >>>>>>>>>
>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
>>> > update
>>> > >>>> and
>>> > >>>>>>>>> merge that I will be the person resolving merge conflicts in
>>> the
>>> > >>>>>>>> JavaScript
>>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as
>>> opposed
>>> > to
>>> > >>>>>>>> putting
>>> > >>>>>>>>> that work on someone else.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Let me know what you think.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>> > >>>>>>>>>
>>> > >>>>>>>>> Thanks!
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>>> > jburwell@basho.com
>>> > >>>
>>> > >>>>>>>> wrote:
>>> > >>>>>>>>>
>>> > >>>>>>>>>> Mike,
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>> > >>>> mechanisms
>>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the
>>> affected
>>> > FSs
>>> > >>>> been
>>> > >>>>>>>>>> updated with that decision?
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
>>> > >> between
>>> > >>>>>> the
>>> > >>>>>>>>>> patches?
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> Thanks,
>>> > >>>>>>>>>> -John
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>> No problem, John.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up
>>> with a
>>> > >> new
>>> > >>>>>>>> patch
>>> > >>>>>>>>>>> file.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>> checked
>>> > >> in
>>> > >>>>>> and
>>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>> > >> jburwell@basho.com
>>> > >>>>>
>>> > >>>>>>>>>> wrote:
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>> Mike,
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>> > >> offline
>>> > >>>>>> ATM,
>>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> Sorry for the oversight,
>>> > >>>>>>>>>>>> -John
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>> this :)
>>> > >> ),
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>> areas
>>> > we
>>> > >>>>>> wanted
>>> > >>>>>>>>>>>>> addressed.
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>> > afternoon,
>>> > >>>> then
>>> > >>>>>>>>>> send
>>> > >>>>>>>>>>>> in
>>> > >>>>>>>>>>>>> another patch.
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> Sure, that sounds good.
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> > >>>> ustcweizhou@gmail.com
>>> > >>>>>>>
>>> > >>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> Hi Mike,
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>>> Java
>>> > >>>> code,
>>> > >>>>>>>>>>>> except
>>> > >>>>>>>>>>>>>>> the cloudstack UI.
>>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>> branch
>>> > >> into
>>> > >>>>>>>>>> master
>>> > >>>>>>>>>>>>>>> this
>>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> -Wei
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.com
>>> >
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> Hey John,
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>> object_store
>>> > >>>> branch,
>>> > >>>>>>>> but
>>> > >>>>>>>>>> -
>>> > >>>>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>> > >>>> SolidFire
>>> > >>>>>>>>>> branch
>>> > >>>>>>>>>>>>>>> into
>>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>> > >> master.
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>> > merge
>>> > >>>>>>>>>> strategy.
>>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>> > >>>>>>>> jburwell@basho.com>
>>> > >>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Mike,
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>> > >>>>>>>> disk_io_throttling,
>>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>> of
>>> > >>>>>>>> overlapping
>>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>> > >> patches
>>> > >>>> to
>>> > >>>>>>>> be
>>> > >>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>>> follows:
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>> > >> SolidFire
>>> > >>>>>> are
>>> > >>>>>>>>>>>>>>> additive
>>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>> > dependency
>>> > >>>>>>>> between
>>> > >>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>> we
>>> > can
>>> > >>>>>>>>>> determine
>>> > >>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> Thanks,
>>> > >>>>>>>>>>>>>>>>> -John
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> > >>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>> into
>>> > his
>>> > >>>>>> branch
>>> > >>>>>>>>>>>>>>> before
>>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>> after his
>>> > >>>> merge
>>> > >>>>>>>> and
>>> > >>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>> can
>>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>> code in
>>> > >> the
>>> > >>>>>>>>>> storage
>>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>> > plug-in,
>>> > >>>>>> which
>>> > >>>>>>>>>>>>>>>>> currently
>>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>> > (although
>>> > >>>> it
>>> > >>>>>>>> does
>>> > >>>>>>>>>>>>>>> not
>>> > >>>>>>>>>>>>>>>>> know
>>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>> actually
>>> > in
>>> > >>>>>> use))
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>> the
>>> > url
>>> > >>>>>> field
>>> > >>>>>>>>>> (if
>>> > >>>>>>>>>>>>>>>> not
>>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>>> in
>>> > the
>>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>> hypervisor
>>> > in
>>> > >>>>>>>>>>>>>>> question, we
>>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>> place
>>> > of
>>> > >>>> the
>>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>> > checks
>>> > >>>> for
>>> > >>>>>>>> the
>>> > >>>>>>>>>>>>>>>> managed
>>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>> > hypervisor
>>> > >>>>>> data
>>> > >>>>>>>>>>>>>>>>> structure is
>>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>> executes to
>>> > >>>> attach
>>> > >>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> volume.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>> > >> detach a
>>> > >>>>>>>>>> volume,
>>> > >>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> > >>>> structure
>>> > >>>>>>>> is
>>> > >>>>>>>>>>>>>>>>> removed.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>>> in
>>> > 4.2
>>> > >>>>>>>> unless
>>> > >>>>>>>>>>>>>>> it is
>>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>>> we
>>> > >> have
>>> > >>>>>> some
>>> > >>>>>>>>>>>>>>> idea,
>>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> Thanks!
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>> avoiding
>>> > >>>>>>>>>> implementation
>>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>> > field.
>>> > >> I
>>> > >>>>>>>> think
>>> > >>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>>> should
>>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>> regarding
>>> > >> the
>>> > >>>>>>>>>>>>>>> expected
>>> > >>>>>>>>>>>>>>>>> format
>>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>> are
>>> > >> not
>>> > >>>>>>>>>>>>>>> returned on
>>> > >>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>> > listStoragePool
>>> > >>>>>>>>>> response.
>>> > >>>>>>>>>>>>>>>> Why
>>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>> useful
>>> > >> for
>>> > >>>>>>>>>> clients
>>> > >>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>> be
>>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>> field."
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>> here.
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>> > >> Offering-by-Disk
>>> > >>>>>>>>>> Offering
>>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>> to be
>>> > >> able
>>> > >>>>>> to
>>> > >>>>>>>>>>>>>>>> associate
>>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>> disk_offering_details
>>> > >> table.
>>> > >>>>>>>> Maybe
>>> > >>>>>>>>>>>>>>> it
>>> > >>>>>>>>>>>>>>>>> could
>>> > >>>>>>>>>>>>>>>>>>>> go there?
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>> IOPS
>>> > in
>>> > >>>> the
>>> > >>>>>>>> GUI
>>> > >>>>>>>>>>>>>>> if
>>> > >>>>>>>>>>>>>>>>> it's
>>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>>> DB.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>>>>>>> cloud<
>>> > http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>> >
>>> > >>>>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> --
>>> > >>>>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>>>> *â„¢*
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> --
>>> > >>>>>>>>> *Mike Tutkowski*
>>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>>>> o: 303.746.7302
>>> > >>>>>>>>> Advancing the way the world uses the
>>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>>>> *â„¢*
>>> > >>>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>>
>>> > >>>>>>> --
>>> > >>>>>>> *Mike Tutkowski*
>>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>>>> o: 303.746.7302
>>> > >>>>>>> Advancing the way the world uses the
>>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>>>> *â„¢*
>>> > >>>>>>
>>> > >>>>>>
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> --
>>> > >>>>> *Mike Tutkowski*
>>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>>>> e: mike.tutkowski@solidfire.com
>>> > >>>>> o: 303.746.7302
>>> > >>>>> Advancing the way the world uses the
>>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>>>> *â„¢*
>>> > >>>>
>>> > >>>>
>>> > >>>
>>> > >>>
>>> > >>> --
>>> > >>> *Mike Tutkowski*
>>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >>> e: mike.tutkowski@solidfire.com
>>> > >>> o: 303.746.7302
>>> > >>> Advancing the way the world uses the
>>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >>> *â„¢*
>>> > >>
>>> > >>
>>> > >
>>> > >
>>> > > --
>>> > > *Mike Tutkowski*
>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>> > > e: mike.tutkowski@solidfire.com
>>> > > o: 303.746.7302
>>> > > Advancing the way the world uses the
>>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > > *â„¢*
>>> >
>>> >
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
My thought is that we present the min/max IOPS fields for read/write operations for all compute and disk offerings. When the VM is allocated, we determine the best way to fulfill that QoS. It sounds like storage level guarantees would always be preferred. If no storage is available to guarantee the QoS, the allocator falls back to using hypervisor. This approach only works if summing the discrete read/write min/max values to get to min/max total IOPS would be acceptable.
Thanks,
-John
On Jun 12, 2013, at 6:03 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> I hate to say it, but I believe Storage QoS with a Min and Max will always be optimal over hypervisor rate limiting.
>
> The only time you'd want to use hypervisor rate limiting is if storage QoS is not available.
>
> We currently have no way to know what storage the root or data disk will be deployed to, so we have to present the fields in all situation.
>
> This is because of - in my opinion - the somewhat flawed way CloudStack handles storage tags. You are free to enter in any text there and we don't know what it maps to when the Compute or Disk Offering dialog is up.
>
>
> On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my comments/questions in-line below.
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>
>> Hi Wei,
>>
>> So, not entirely sure I follow.
>>
>> Will what I wrote earlier work? Here is a copy of what I wrote:
>>
>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage QoS for the purpose of this e-mail.
>>
>> By default, neither radio button is selected and no QoS fields are visible.
>
> I prefer a None option selected by default. I find the no button selected in a Radio button group to be confusing.
>
> Also, do we have a mechanism to determine whether or not this radio button group should be displayed at all? If there are no devices or hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>
>>
>> If the user selects the Storage QoS radio button, then the Custom IOPS checkbox and the Min and Max text fields are made visible.
>>
>> If the user changes his mind and selects the Hypervisor QoS radio button, then the Custom IOPS checkbox and the Min and Max text fields disappear and are replaced by the two Hypervisor QoS text fields.
>>
>> This way, the user can choose neither QoS option or one of them or the other, but never both.
>>
>> On the API side, I was thinking of having logic in place when a request comes in to create a Disk Offering to confirm these fields are the way we want them.
>>
>> Once we know the Disk Offering is in order, a user can create a data disk from it. Since we checked the validity of the Disk Offering when it was created, the VM should never be asked to use Hypervisor QoS when Storage QoS in being utilized.
>
> Overall, this model sounds very reasonable. Wondering aloud: Would be possible to have the UI simply ask for a storage QoS, and let the system pick the optional implementation. For example, if a VM is attached to KVM and SolidFire is available, we chose to fulfill the QoS using the storage device. This approach would also allow us to fallback in the event that we can't allocate storage for the QoS, but we have hypervisor capacity. Could we always deterministically determine the optimal implementation of the QoS at time of VM deployment?
>
>>
>> Thanks!
>>
>>
>> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
>> Mike, John
>>
>> The Disk I/O Throttling works like:
>>
>> (1) For User VM,
>> (1.1) root disks: service offering -> default value in global configuration
>> (1.2) data disks: disk offering -> default value in global configuration
>>
>> (2) For System VM
>> root disks: service offering
>>
>> -Wei
>>
>>
>> 2013/6/12 John Burwell <jb...@basho.com>
>>
>> > Mike,
>> >
>> > That is one possibility. The other possibility is that hypervisor is
>> > going to throttle I/O on all disks attached. Therefore, we need answers to
>> > the following questions:
>> >
>> > 1. Is I/O throttling applied to the root disk or all disks
>> > attached to the VM?
>> > 2. If I/O throttling is applied to all disks, how is the
>> > throttling distributed amongst the disks if only one read/write value is
>> > defined?
>> >
>> > Thanks,
>> > -John
>> >
>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
>> > wrote:
>> >
>> > > Hey John,
>> > >
>> > > Perhaps I don't fully understand how Wei's feature works.
>> > >
>> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>> > >
>> > > From my thinking, you're root disk could be under Hypervisor QoS, but
>> > your
>> > > data disk could be under Storage QoS.
>> > >
>> > > Is that incorrect?
>> > >
>> > > Thanks
>> > >
>> > >
>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>> > wrote:
>> > >
>> > >> Mike,
>> > >>
>> > >> As I understand these two patches, the throttled I?O settings are
>> > applied
>> > >> from the hypervisor side, and possibly defined in a compute offering
>> > where
>> > >> provisioned IOPS are defined on the storage side through disk
>> > offerings. I
>> > >> don't see how the management server could enforce this mutual exclusion
>> > >> between provisioned IOPS and throttled I/O until a compute and disk
>> > >> offering were selected. These selections come together when we deploy a
>> > >> VM. Based on these assumptions, I would expect to see enforcement of
>> > this
>> > >> rule in UI at the time of VM creation/definition and on the server
>> > side, as
>> > >> part of the VM creation. It feels like any attempt to enforce this rule
>> > >> when defining offering would be premature, and unnecessarily limit
>> > >> flexibility. Are my assumptions and understanding correct?
>> > >>
>> > >> Thanks,
>> > >> -John
>> > >>
>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com>
>> > >> wrote:
>> > >>
>> > >>> Hi John,
>> > >>>
>> > >>> So, maybe I'm wrong about this, but what I was thinking is that we'd
>> > >> build
>> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
>> > >> Compute
>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>> > >>>
>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>> > >> QoS
>> > >>> for the purpose of this e-mail.
>> > >>>
>> > >>> By default, neither radio button is selected and no QoS fields are
>> > >> visible.
>> > >>>
>> > >>> If the user selects the Storage QoS radio button, then the Custom IOPS
>> > >>> checkbox and the Min and Max text fields are made visible.
>> > >>>
>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>> > button,
>> > >>> then the Custom IOPS checkbox and the Min and Max text fields disappear
>> > >> and
>> > >>> are replaced by the two Hypervisor QoS text fields.
>> > >>>
>> > >>> This way, the user can choose neither QoS option or one of them or the
>> > >>> other, but never both.
>> > >>>
>> > >>> On the API side, I was thinking of having logic in place when a request
>> > >>> comes in to create a Disk Offering to confirm these fields are the way
>> > we
>> > >>> want them.
>> > >>>
>> > >>> Once we know the Disk Offering is in order, a user can create a data
>> > disk
>> > >>> from it. Since we checked the validity of the Disk Offering when it was
>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>> > Storage
>> > >>> QoS in being utilized.
>> > >>>
>> > >>> Does that make sense or did I miss something?
>> > >>>
>> > >>> Thanks
>> > >>>
>> > >>>
>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>> > >> wrote:
>> > >>>
>> > >>>> Mike,
>> > >>>>
>> > >>>> Looking through the code, I am trying to understand how
>> > >>>> CreateDiskOfferingCmd would have the context to identify the conflict.
>> > >>>> Naively, it seems to me that this rule would need to be enforced when
>> > a
>> > >>>> virtual machine is being deployed. Looking through the code, it seems
>> > >> like
>> > >>>> we should add a private validateStorageQoS method to
>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
>> > >>>> ResourceAllocationException when the QoS definitions are inconsistent.
>> > >> We
>> > >>>> would then add calls to it from each of the VM creation methods in the
>> > >>>> service. Do this type of approach sound reasonable?
>> > >>>>
>> > >>>> Thanks,
>> > >>>> -John
>> > >>>>
>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>> > >> mike.tutkowski@solidfire.com>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> Hi John,
>> > >>>>>
>> > >>>>> So, here's what I was planning to do. Of course feel free to correct
>> > me
>> > >>>> on
>> > >>>>> this approach.
>> > >>>>>
>> > >>>>> I think it's OK if Wei merges his code into master and then I can
>> > draw
>> > >>>> from
>> > >>>>> the main repo and merge master into mine locally.
>> > >>>>>
>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code
>> > to
>> > >>>> make
>> > >>>>> it user friendly (toggle between these features on the Add Disk
>> > >> Offering
>> > >>>>> window).
>> > >>>>>
>> > >>>>> 2) I plan to write validation logic for the create-disk-offering API
>> > >>>>> command which throws an exception if the rules are not followed (this
>> > >>>>> should never be triggered from the GUI since the GUI will have
>> > controls
>> > >>>> in
>> > >>>>> place to toggle between the one feature and the other).
>> > >>>>>
>> > >>>>> I'm not sure about documentation. I haven't had much experience with
>> > it
>> > >>>> on
>> > >>>>> CloudStack projects yet.
>> > >>>>>
>> > >>>>> Thanks!
>> > >>>>>
>> > >>>>>
>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
>> > >>>> wrote:
>> > >>>>>
>> > >>>>>> Mike,
>> > >>>>>>
>> > >>>>>> Yes, these server-side rails need to be defined and implemented
>> > before
>> > >>>>>> either patch can be merged. From my perspective, I would like to
>> > see
>> > >>>> the
>> > >>>>>> rule implemented in the hypervisor as part of the validation of the
>> > >>>> virtual
>> > >>>>>> machine definition. We also need to make sure that this mutual
>> > >>>> exclusion
>> > >>>>>> is documented. Do we usually include this type of documentation
>> > with
>> > >>>>>> patches of this nature?
>> > >>>>>>
>> > >>>>>> Thanks,
>> > >>>>>> -John
>> > >>>>>>
>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>> > >>>> mike.tutkowski@solidfire.com>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Currently they are not yet implemented.
>> > >>>>>>>
>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>> > usability
>> > >>>>>>> standpoint, but the API must check for consistency and throw an
>> > >>>> exception
>> > >>>>>>> if necessary.
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jburwell@basho.com
>> > >
>> > >>>>>> wrote:
>> > >>>>>>>
>> > >>>>>>>> Mike,
>> > >>>>>>>>
>> > >>>>>>>> Are the checks only implemented in the UI?
>> > >>>>>>>>
>> > >>>>>>>> Thanks,
>> > >>>>>>>> -John
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>> > >>>>>>>> <mi...@solidfire.com> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Hi John,
>> > >>>>>>>>>
>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>> > >> exclusive.
>> > >>>> We
>> > >>>>>>>>> agree with you that only one should be active at a time. We plan
>> > to
>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn
>> > his
>> > >>>>>>>> feature
>> > >>>>>>>>> on and mine off and vice versa.
>> > >>>>>>>>>
>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
>> > update
>> > >>>> and
>> > >>>>>>>>> merge that I will be the person resolving merge conflicts in the
>> > >>>>>>>> JavaScript
>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed
>> > to
>> > >>>>>>>> putting
>> > >>>>>>>>> that work on someone else.
>> > >>>>>>>>>
>> > >>>>>>>>> Let me know what you think.
>> > >>>>>>>>>
>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>> > >>>>>>>>>
>> > >>>>>>>>> Thanks!
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>> > jburwell@basho.com
>> > >>>
>> > >>>>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> Mike,
>> > >>>>>>>>>>
>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>> > >>>> mechanisms
>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected
>> > FSs
>> > >>>> been
>> > >>>>>>>>>> updated with that decision?
>> > >>>>>>>>>>
>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
>> > >> between
>> > >>>>>> the
>> > >>>>>>>>>> patches?
>> > >>>>>>>>>>
>> > >>>>>>>>>> Thanks,
>> > >>>>>>>>>> -John
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> No problem, John.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> I still want to re-review it by myself before coming up with a
>> > >> new
>> > >>>>>>>> patch
>> > >>>>>>>>>>> file.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
>> > >> in
>> > >>>>>> and
>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> > >> jburwell@basho.com
>> > >>>>>
>> > >>>>>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> Mike,
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>> > >> offline
>> > >>>>>> ATM,
>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Sorry for the oversight,
>> > >>>>>>>>>>>> -John
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
>> > >> ),
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
>> > we
>> > >>>>>> wanted
>> > >>>>>>>>>>>>> addressed.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>> > afternoon,
>> > >>>> then
>> > >>>>>>>>>> send
>> > >>>>>>>>>>>> in
>> > >>>>>>>>>>>>> another patch.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Sure, that sounds good.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> > >>>> ustcweizhou@gmail.com
>> > >>>>>>>
>> > >>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Hi Mike,
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>> > >>>> code,
>> > >>>>>>>>>>>> except
>> > >>>>>>>>>>>>>>> the cloudstack UI.
>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
>> > >> into
>> > >>>>>>>>>> master
>> > >>>>>>>>>>>>>>> this
>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> -Wei
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Hey John,
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>> > >>>> branch,
>> > >>>>>>>> but
>> > >>>>>>>>>> -
>> > >>>>>>>>>>>> as
>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>> > >>>> SolidFire
>> > >>>>>>>>>> branch
>> > >>>>>>>>>>>>>>> into
>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>> > >> master.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>> > merge
>> > >>>>>>>>>> strategy.
>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> > >>>>>>>> jburwell@basho.com>
>> > >>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Mike,
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>> > >>>>>>>> disk_io_throttling,
>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>> > >>>>>>>> overlapping
>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>> > >> patches
>> > >>>> to
>> > >>>>>>>> be
>> > >>>>>>>>>> as
>> > >>>>>>>>>>>>>>>>> follows:
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>> > >> SolidFire
>> > >>>>>> are
>> > >>>>>>>>>>>>>>> additive
>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>> > dependency
>> > >>>>>>>> between
>> > >>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
>> > can
>> > >>>>>>>>>> determine
>> > >>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>>>>>>> -John
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > >>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
>> > his
>> > >>>>>> branch
>> > >>>>>>>>>>>>>>> before
>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>> > >>>> merge
>> > >>>>>>>> and
>> > >>>>>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>> can
>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
>> > >> the
>> > >>>>>>>>>> storage
>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>> > plug-in,
>> > >>>>>> which
>> > >>>>>>>>>>>>>>>>> currently
>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>> > (although
>> > >>>> it
>> > >>>>>>>> does
>> > >>>>>>>>>>>>>>> not
>> > >>>>>>>>>>>>>>>>> know
>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
>> > in
>> > >>>>>> use))
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
>> > url
>> > >>>>>> field
>> > >>>>>>>>>> (if
>> > >>>>>>>>>>>>>>>> not
>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
>> > the
>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
>> > in
>> > >>>>>>>>>>>>>>> question, we
>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
>> > of
>> > >>>> the
>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>> > checks
>> > >>>> for
>> > >>>>>>>> the
>> > >>>>>>>>>>>>>>>> managed
>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>> > hypervisor
>> > >>>>>> data
>> > >>>>>>>>>>>>>>>>> structure is
>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>> > >>>> attach
>> > >>>>>>>> the
>> > >>>>>>>>>>>>>>>>> volume.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>> > >> detach a
>> > >>>>>>>>>> volume,
>> > >>>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>> > >>>> structure
>> > >>>>>>>> is
>> > >>>>>>>>>>>>>>>>> removed.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
>> > 4.2
>> > >>>>>>>> unless
>> > >>>>>>>>>>>>>>> it is
>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
>> > >> have
>> > >>>>>> some
>> > >>>>>>>>>>>>>>> idea,
>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> Thanks!
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>> > >>>>>>>>>> implementation
>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>> > field.
>> > >> I
>> > >>>>>>>> think
>> > >>>>>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>>> should
>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
>> > >> the
>> > >>>>>>>>>>>>>>> expected
>> > >>>>>>>>>>>>>>>>> format
>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
>> > >> not
>> > >>>>>>>>>>>>>>> returned on
>> > >>>>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>> > listStoragePool
>> > >>>>>>>>>> response.
>> > >>>>>>>>>>>>>>>> Why
>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
>> > >> for
>> > >>>>>>>>>> clients
>> > >>>>>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>> be
>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> > >> Offering-by-Disk
>> > >>>>>>>>>> Offering
>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>> > >> able
>> > >>>>>> to
>> > >>>>>>>>>>>>>>>> associate
>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>> > >> table.
>> > >>>>>>>> Maybe
>> > >>>>>>>>>>>>>>> it
>> > >>>>>>>>>>>>>>>>> could
>> > >>>>>>>>>>>>>>>>>>>> go there?
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
>> > in
>> > >>>> the
>> > >>>>>>>> GUI
>> > >>>>>>>>>>>>>>> if
>> > >>>>>>>>>>>>>>>>> it's
>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>>>>>>>>> cloud<
>> > http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> --
>> > >>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> --
>> > >>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>> *â„¢*
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> --
>> > >>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>> o: 303.746.7302
>> > >>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>> *â„¢*
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> --
>> > >>>>>>> *Mike Tutkowski*
>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>> o: 303.746.7302
>> > >>>>>>> Advancing the way the world uses the
>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>> *â„¢*
>> > >>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> --
>> > >>>>> *Mike Tutkowski*
>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>> e: mike.tutkowski@solidfire.com
>> > >>>>> o: 303.746.7302
>> > >>>>> Advancing the way the world uses the
>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>> *â„¢*
>> > >>>>
>> > >>>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> *Mike Tutkowski*
>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>> e: mike.tutkowski@solidfire.com
>> > >>> o: 303.746.7302
>> > >>> Advancing the way the world uses the
>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>> *â„¢*
>> > >>
>> > >>
>> > >
>> > >
>> > > --
>> > > *Mike Tutkowski*
>> > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > e: mike.tutkowski@solidfire.com
>> > > o: 303.746.7302
>> > > Advancing the way the world uses the
>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > *â„¢*
>> >
>> >
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloudâ„¢
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I hate to say it, but I believe Storage QoS with a Min and Max will always
be optimal over hypervisor rate limiting.
The only time you'd want to use hypervisor rate limiting is if storage QoS
is not available.
We currently have no way to know what storage the root or data disk will be
deployed to, so we have to present the fields in all situation.
This is because of - in my opinion - the somewhat flawed way CloudStack
handles storage tags. You are free to enter in any text there and we don't
know what it maps to when the Compute or Disk Offering dialog is up.
On Wed, Jun 12, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my comments/questions in-line below.
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> Hi Wei,
>
> So, not entirely sure I follow.
>
> Will what I wrote earlier work? Here is a copy of what I wrote:
>
> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage QoS
> for the purpose of this e-mail.
>
> By default, neither radio button is selected and no QoS fields are visible.
>
>
> I prefer a None option selected by default. I find the no button selected
> in a Radio button group to be confusing.
>
> Also, do we have a mechanism to determine whether or not this radio button
> group should be displayed at all? If there are no devices or hypervisors
> configured to fulfill the QoS, we shouldn't mislead the user ...
>
>
> If the user selects the Storage QoS radio button, then the Custom IOPS
> checkbox and the Min and Max text fields are made visible.
>
> If the user changes his mind and selects the Hypervisor QoS radio button,
> then the Custom IOPS checkbox and the Min and Max text fields disappear and
> are replaced by the two Hypervisor QoS text fields.
>
> This way, the user can choose neither QoS option or one of them or the
> other, but never both.
>
> On the API side, I was thinking of having logic in place when a request
> comes in to create a Disk Offering to confirm these fields are the way we
> want them.
>
> Once we know the Disk Offering is in order, a user can create a data disk
> from it. Since we checked the validity of the Disk Offering when it was
> created, the VM should never be asked to use Hypervisor QoS when Storage
> QoS in being utilized.
>
>
> Overall, this model sounds very reasonable. Wondering aloud: Would be
> possible to have the UI simply ask for a storage QoS, and let the system
> pick the optional implementation. For example, if a VM is attached to KVM
> and SolidFire is available, we chose to fulfill the QoS using the storage
> device. This approach would also allow us to fallback in the event that we
> can't allocate storage for the QoS, but we have hypervisor capacity. Could
> we always deterministically determine the optimal implementation of the QoS
> at time of VM deployment?
>
>
> Thanks!
>
>
> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
>
>> Mike, John
>>
>> The Disk I/O Throttling works like:
>>
>> (1) For User VM,
>> (1.1) root disks: service offering -> default value in global
>> configuration
>> (1.2) data disks: disk offering -> default value in global configuration
>>
>> (2) For System VM
>> root disks: service offering
>>
>> -Wei
>>
>>
>> 2013/6/12 John Burwell <jb...@basho.com>
>>
>> > Mike,
>> >
>> > That is one possibility. The other possibility is that hypervisor is
>> > going to throttle I/O on all disks attached. Therefore, we need
>> answers to
>> > the following questions:
>> >
>> > 1. Is I/O throttling applied to the root disk or all disks
>> > attached to the VM?
>> > 2. If I/O throttling is applied to all disks, how is the
>> > throttling distributed amongst the disks if only one read/write value is
>> > defined?
>> >
>> > Thanks,
>> > -John
>> >
>> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> > wrote:
>> >
>> > > Hey John,
>> > >
>> > > Perhaps I don't fully understand how Wei's feature works.
>> > >
>> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
>> > > Offerings (root disks) and/or Disk Offerings (data disks).
>> > >
>> > > From my thinking, you're root disk could be under Hypervisor QoS, but
>> > your
>> > > data disk could be under Storage QoS.
>> > >
>> > > Is that incorrect?
>> > >
>> > > Thanks
>> > >
>> > >
>> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
>> > wrote:
>> > >
>> > >> Mike,
>> > >>
>> > >> As I understand these two patches, the throttled I?O settings are
>> > applied
>> > >> from the hypervisor side, and possibly defined in a compute offering
>> > where
>> > >> provisioned IOPS are defined on the storage side through disk
>> > offerings. I
>> > >> don't see how the management server could enforce this mutual
>> exclusion
>> > >> between provisioned IOPS and throttled I/O until a compute and disk
>> > >> offering were selected. These selections come together when we
>> deploy a
>> > >> VM. Based on these assumptions, I would expect to see enforcement of
>> > this
>> > >> rule in UI at the time of VM creation/definition and on the server
>> > side, as
>> > >> part of the VM creation. It feels like any attempt to enforce this
>> rule
>> > >> when defining offering would be premature, and unnecessarily limit
>> > >> flexibility. Are my assumptions and understanding correct?
>> > >>
>> > >> Thanks,
>> > >> -John
>> > >>
>> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com>
>> > >> wrote:
>> > >>
>> > >>> Hi John,
>> > >>>
>> > >>> So, maybe I'm wrong about this, but what I was thinking is that we'd
>> > >> build
>> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
>> > >> Compute
>> > >>> Offerings for 4.2 since my feature doesn't yet support them).
>> > >>>
>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
>> Storage
>> > >> QoS
>> > >>> for the purpose of this e-mail.
>> > >>>
>> > >>> By default, neither radio button is selected and no QoS fields are
>> > >> visible.
>> > >>>
>> > >>> If the user selects the Storage QoS radio button, then the Custom
>> IOPS
>> > >>> checkbox and the Min and Max text fields are made visible.
>> > >>>
>> > >>> If the user changes his mind and selects the Hypervisor QoS radio
>> > button,
>> > >>> then the Custom IOPS checkbox and the Min and Max text fields
>> disappear
>> > >> and
>> > >>> are replaced by the two Hypervisor QoS text fields.
>> > >>>
>> > >>> This way, the user can choose neither QoS option or one of them or
>> the
>> > >>> other, but never both.
>> > >>>
>> > >>> On the API side, I was thinking of having logic in place when a
>> request
>> > >>> comes in to create a Disk Offering to confirm these fields are the
>> way
>> > we
>> > >>> want them.
>> > >>>
>> > >>> Once we know the Disk Offering is in order, a user can create a data
>> > disk
>> > >>> from it. Since we checked the validity of the Disk Offering when it
>> was
>> > >>> created, the VM should never be asked to use Hypervisor QoS when
>> > Storage
>> > >>> QoS in being utilized.
>> > >>>
>> > >>> Does that make sense or did I miss something?
>> > >>>
>> > >>> Thanks
>> > >>>
>> > >>>
>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>> > >> wrote:
>> > >>>
>> > >>>> Mike,
>> > >>>>
>> > >>>> Looking through the code, I am trying to understand how
>> > >>>> CreateDiskOfferingCmd would have the context to identify the
>> conflict.
>> > >>>> Naively, it seems to me that this rule would need to be enforced
>> when
>> > a
>> > >>>> virtual machine is being deployed. Looking through the code, it
>> seems
>> > >> like
>> > >>>> we should add a private validateStorageQoS method to
>> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
>> > >>>> ResourceAllocationException when the QoS definitions are
>> inconsistent.
>> > >> We
>> > >>>> would then add calls to it from each of the VM creation methods in
>> the
>> > >>>> service. Do this type of approach sound reasonable?
>> > >>>>
>> > >>>> Thanks,
>> > >>>> -John
>> > >>>>
>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>> > >> mike.tutkowski@solidfire.com>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> Hi John,
>> > >>>>>
>> > >>>>> So, here's what I was planning to do. Of course feel free to
>> correct
>> > me
>> > >>>> on
>> > >>>>> this approach.
>> > >>>>>
>> > >>>>> I think it's OK if Wei merges his code into master and then I can
>> > draw
>> > >>>> from
>> > >>>>> the main repo and merge master into mine locally.
>> > >>>>>
>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI
>> code
>> > to
>> > >>>> make
>> > >>>>> it user friendly (toggle between these features on the Add Disk
>> > >> Offering
>> > >>>>> window).
>> > >>>>>
>> > >>>>> 2) I plan to write validation logic for the create-disk-offering
>> API
>> > >>>>> command which throws an exception if the rules are not followed
>> (this
>> > >>>>> should never be triggered from the GUI since the GUI will have
>> > controls
>> > >>>> in
>> > >>>>> place to toggle between the one feature and the other).
>> > >>>>>
>> > >>>>> I'm not sure about documentation. I haven't had much experience
>> with
>> > it
>> > >>>> on
>> > >>>>> CloudStack projects yet.
>> > >>>>>
>> > >>>>> Thanks!
>> > >>>>>
>> > >>>>>
>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jburwell@basho.com
>> >
>> > >>>> wrote:
>> > >>>>>
>> > >>>>>> Mike,
>> > >>>>>>
>> > >>>>>> Yes, these server-side rails need to be defined and implemented
>> > before
>> > >>>>>> either patch can be merged. From my perspective, I would like to
>> > see
>> > >>>> the
>> > >>>>>> rule implemented in the hypervisor as part of the validation of
>> the
>> > >>>> virtual
>> > >>>>>> machine definition. We also need to make sure that this mutual
>> > >>>> exclusion
>> > >>>>>> is documented. Do we usually include this type of documentation
>> > with
>> > >>>>>> patches of this nature?
>> > >>>>>>
>> > >>>>>> Thanks,
>> > >>>>>> -John
>> > >>>>>>
>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>> > >>>> mike.tutkowski@solidfire.com>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Currently they are not yet implemented.
>> > >>>>>>>
>> > >>>>>>> We have to make sure they are implemented in the GUI from a
>> > usability
>> > >>>>>>> standpoint, but the API must check for consistency and throw an
>> > >>>> exception
>> > >>>>>>> if necessary.
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
>> jburwell@basho.com
>> > >
>> > >>>>>> wrote:
>> > >>>>>>>
>> > >>>>>>>> Mike,
>> > >>>>>>>>
>> > >>>>>>>> Are the checks only implemented in the UI?
>> > >>>>>>>>
>> > >>>>>>>> Thanks,
>> > >>>>>>>> -John
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>> > >>>>>>>> <mi...@solidfire.com> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Hi John,
>> > >>>>>>>>>
>> > >>>>>>>>> Wei and I have discussed making the two features mutually
>> > >> exclusive.
>> > >>>> We
>> > >>>>>>>>> agree with you that only one should be active at a time. We
>> plan
>> > to
>> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn
>> > his
>> > >>>>>>>> feature
>> > >>>>>>>>> on and mine off and vice versa.
>> > >>>>>>>>>
>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
>> > update
>> > >>>> and
>> > >>>>>>>>> merge that I will be the person resolving merge conflicts in
>> the
>> > >>>>>>>> JavaScript
>> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as
>> opposed
>> > to
>> > >>>>>>>> putting
>> > >>>>>>>>> that work on someone else.
>> > >>>>>>>>>
>> > >>>>>>>>> Let me know what you think.
>> > >>>>>>>>>
>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>> > >>>>>>>>>
>> > >>>>>>>>> Thanks!
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
>> > jburwell@basho.com
>> > >>>
>> > >>>>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> Mike,
>> > >>>>>>>>>>
>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>> > >>>> mechanisms
>> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected
>> > FSs
>> > >>>> been
>> > >>>>>>>>>> updated with that decision?
>> > >>>>>>>>>>
>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
>> > >> between
>> > >>>>>> the
>> > >>>>>>>>>> patches?
>> > >>>>>>>>>>
>> > >>>>>>>>>> Thanks,
>> > >>>>>>>>>> -John
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>> > >>>>>>>>>> <mi...@solidfire.com> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> No problem, John.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> I still want to re-review it by myself before coming up
>> with a
>> > >> new
>> > >>>>>>>> patch
>> > >>>>>>>>>>> file.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>> checked
>> > >> in
>> > >>>>>> and
>> > >>>>>>>>>>> merge those into mine before generating a new patch file?
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> > >> jburwell@basho.com
>> > >>>>>
>> > >>>>>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> Mike,
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>> > >> offline
>> > >>>>>> ATM,
>> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Sorry for the oversight,
>> > >>>>>>>>>>>> -John
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>> this :)
>> > >> ),
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>> areas
>> > we
>> > >>>>>> wanted
>> > >>>>>>>>>>>>> addressed.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>> > afternoon,
>> > >>>> then
>> > >>>>>>>>>> send
>> > >>>>>>>>>>>> in
>> > >>>>>>>>>>>>> another patch.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Sure, that sounds good.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> > >>>> ustcweizhou@gmail.com
>> > >>>>>>>
>> > >>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Hi Mike,
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>> Java
>> > >>>> code,
>> > >>>>>>>>>>>> except
>> > >>>>>>>>>>>>>>> the cloudstack UI.
>> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>> branch
>> > >> into
>> > >>>>>>>>>> master
>> > >>>>>>>>>>>>>>> this
>> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> -Wei
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Hey John,
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>> > >>>> branch,
>> > >>>>>>>> but
>> > >>>>>>>>>> -
>> > >>>>>>>>>>>> as
>> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>> > >>>> SolidFire
>> > >>>>>>>>>> branch
>> > >>>>>>>>>>>>>>> into
>> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>> > >> master.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>> > merge
>> > >>>>>>>>>> strategy.
>> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> > >>>>>>>> jburwell@basho.com>
>> > >>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Mike,
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>> > >>>>>>>> disk_io_throttling,
>> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>> > >>>>>>>> overlapping
>> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>> > >> patches
>> > >>>> to
>> > >>>>>>>> be
>> > >>>>>>>>>> as
>> > >>>>>>>>>>>>>>>>> follows:
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>> > >> SolidFire
>> > >>>>>> are
>> > >>>>>>>>>>>>>>> additive
>> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>> > dependency
>> > >>>>>>>> between
>> > >>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
>> > can
>> > >>>>>>>>>> determine
>> > >>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>>>>>>> -John
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > >>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
>> > his
>> > >>>>>> branch
>> > >>>>>>>>>>>>>>> before
>> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after
>> his
>> > >>>> merge
>> > >>>>>>>> and
>> > >>>>>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>> can
>> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code
>> in
>> > >> the
>> > >>>>>>>>>> storage
>> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>> > plug-in,
>> > >>>>>> which
>> > >>>>>>>>>>>>>>>>> currently
>> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>> > (although
>> > >>>> it
>> > >>>>>>>> does
>> > >>>>>>>>>>>>>>> not
>> > >>>>>>>>>>>>>>>>> know
>> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>> actually
>> > in
>> > >>>>>> use))
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>> the
>> > url
>> > >>>>>> field
>> > >>>>>>>>>> (if
>> > >>>>>>>>>>>>>>>> not
>> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>> in
>> > the
>> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>> hypervisor
>> > in
>> > >>>>>>>>>>>>>>> question, we
>> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>> place
>> > of
>> > >>>> the
>> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>> > checks
>> > >>>> for
>> > >>>>>>>> the
>> > >>>>>>>>>>>>>>>> managed
>> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>> > hypervisor
>> > >>>>>> data
>> > >>>>>>>>>>>>>>>>> structure is
>> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes
>> to
>> > >>>> attach
>> > >>>>>>>> the
>> > >>>>>>>>>>>>>>>>> volume.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>> > >> detach a
>> > >>>>>>>>>> volume,
>> > >>>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>> > >>>> structure
>> > >>>>>>>> is
>> > >>>>>>>>>>>>>>>>> removed.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>> in
>> > 4.2
>> > >>>>>>>> unless
>> > >>>>>>>>>>>>>>> it is
>> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>> we
>> > >> have
>> > >>>>>> some
>> > >>>>>>>>>>>>>>> idea,
>> > >>>>>>>>>>>>>>>>>>> that'd be cool.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> Thanks!
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>> > >>>>>>>>>> implementation
>> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>> > field.
>> > >> I
>> > >>>>>>>> think
>> > >>>>>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>>> should
>> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>> regarding
>> > >> the
>> > >>>>>>>>>>>>>>> expected
>> > >>>>>>>>>>>>>>>>> format
>> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>> are
>> > >> not
>> > >>>>>>>>>>>>>>> returned on
>> > >>>>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>> > listStoragePool
>> > >>>>>>>>>> response.
>> > >>>>>>>>>>>>>>>> Why
>> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>> useful
>> > >> for
>> > >>>>>>>>>> clients
>> > >>>>>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>> be
>> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>> here.
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> > >> Offering-by-Disk
>> > >>>>>>>>>> Offering
>> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to
>> be
>> > >> able
>> > >>>>>> to
>> > >>>>>>>>>>>>>>>> associate
>> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>> > >> table.
>> > >>>>>>>> Maybe
>> > >>>>>>>>>>>>>>> it
>> > >>>>>>>>>>>>>>>>> could
>> > >>>>>>>>>>>>>>>>>>>> go there?
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>> IOPS
>> > in
>> > >>>> the
>> > >>>>>>>> GUI
>> > >>>>>>>>>>>>>>> if
>> > >>>>>>>>>>>>>>>>> it's
>> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>> DB.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>>>>>>>>> cloud<
>> > http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> --
>> > >>>>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>>>> *â„¢*
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> --
>> > >>>>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>>>> o: 303.746.7302
>> > >>>>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>>>> *â„¢*
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> --
>> > >>>>>>>>> *Mike Tutkowski*
>> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>>>> o: 303.746.7302
>> > >>>>>>>>> Advancing the way the world uses the
>> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>>>> *â„¢*
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> --
>> > >>>>>>> *Mike Tutkowski*
>> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>>>> e: mike.tutkowski@solidfire.com
>> > >>>>>>> o: 303.746.7302
>> > >>>>>>> Advancing the way the world uses the
>> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>>>> *â„¢*
>> > >>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> --
>> > >>>>> *Mike Tutkowski*
>> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>>>> e: mike.tutkowski@solidfire.com
>> > >>>>> o: 303.746.7302
>> > >>>>> Advancing the way the world uses the
>> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>>>> *â„¢*
>> > >>>>
>> > >>>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> *Mike Tutkowski*
>> > >>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >>> e: mike.tutkowski@solidfire.com
>> > >>> o: 303.746.7302
>> > >>> Advancing the way the world uses the
>> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >>> *â„¢*
>> > >>
>> > >>
>> > >
>> > >
>> > > --
>> > > *Mike Tutkowski*
>> > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > e: mike.tutkowski@solidfire.com
>> > > o: 303.746.7302
>> > > Advancing the way the world uses the
>> > > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > *â„¢*
>> >
>> >
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Please see my comments/questions in-line below.
Thanks,
-John
On Jun 12, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi Wei,
>
> So, not entirely sure I follow.
>
> Will what I wrote earlier work? Here is a copy of what I wrote:
>
> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage QoS for the purpose of this e-mail.
>
> By default, neither radio button is selected and no QoS fields are visible.
I prefer a None option selected by default. I find the no button selected in a Radio button group to be confusing.
Also, do we have a mechanism to determine whether or not this radio button group should be displayed at all? If there are no devices or hypervisors configured to fulfill the QoS, we shouldn't mislead the user ...
>
> If the user selects the Storage QoS radio button, then the Custom IOPS checkbox and the Min and Max text fields are made visible.
>
> If the user changes his mind and selects the Hypervisor QoS radio button, then the Custom IOPS checkbox and the Min and Max text fields disappear and are replaced by the two Hypervisor QoS text fields.
>
> This way, the user can choose neither QoS option or one of them or the other, but never both.
>
> On the API side, I was thinking of having logic in place when a request comes in to create a Disk Offering to confirm these fields are the way we want them.
>
> Once we know the Disk Offering is in order, a user can create a data disk from it. Since we checked the validity of the Disk Offering when it was created, the VM should never be asked to use Hypervisor QoS when Storage QoS in being utilized.
Overall, this model sounds very reasonable. Wondering aloud: Would be possible to have the UI simply ask for a storage QoS, and let the system pick the optional implementation. For example, if a VM is attached to KVM and SolidFire is available, we chose to fulfill the QoS using the storage device. This approach would also allow us to fallback in the event that we can't allocate storage for the QoS, but we have hypervisor capacity. Could we always deterministically determine the optimal implementation of the QoS at time of VM deployment?
>
> Thanks!
>
>
> On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
> Mike, John
>
> The Disk I/O Throttling works like:
>
> (1) For User VM,
> (1.1) root disks: service offering -> default value in global configuration
> (1.2) data disks: disk offering -> default value in global configuration
>
> (2) For System VM
> root disks: service offering
>
> -Wei
>
>
> 2013/6/12 John Burwell <jb...@basho.com>
>
> > Mike,
> >
> > That is one possibility. The other possibility is that hypervisor is
> > going to throttle I/O on all disks attached. Therefore, we need answers to
> > the following questions:
> >
> > 1. Is I/O throttling applied to the root disk or all disks
> > attached to the VM?
> > 2. If I/O throttling is applied to all disks, how is the
> > throttling distributed amongst the disks if only one read/write value is
> > defined?
> >
> > Thanks,
> > -John
> >
> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
> > wrote:
> >
> > > Hey John,
> > >
> > > Perhaps I don't fully understand how Wei's feature works.
> > >
> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
> > > Offerings (root disks) and/or Disk Offerings (data disks).
> > >
> > > From my thinking, you're root disk could be under Hypervisor QoS, but
> > your
> > > data disk could be under Storage QoS.
> > >
> > > Is that incorrect?
> > >
> > > Thanks
> > >
> > >
> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
> > wrote:
> > >
> > >> Mike,
> > >>
> > >> As I understand these two patches, the throttled I?O settings are
> > applied
> > >> from the hypervisor side, and possibly defined in a compute offering
> > where
> > >> provisioned IOPS are defined on the storage side through disk
> > offerings. I
> > >> don't see how the management server could enforce this mutual exclusion
> > >> between provisioned IOPS and throttled I/O until a compute and disk
> > >> offering were selected. These selections come together when we deploy a
> > >> VM. Based on these assumptions, I would expect to see enforcement of
> > this
> > >> rule in UI at the time of VM creation/definition and on the server
> > side, as
> > >> part of the VM creation. It feels like any attempt to enforce this rule
> > >> when defining offering would be premature, and unnecessarily limit
> > >> flexibility. Are my assumptions and understanding correct?
> > >>
> > >> Thanks,
> > >> -John
> > >>
> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > >> wrote:
> > >>
> > >>> Hi John,
> > >>>
> > >>> So, maybe I'm wrong about this, but what I was thinking is that we'd
> > >> build
> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
> > >> Compute
> > >>> Offerings for 4.2 since my feature doesn't yet support them).
> > >>>
> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
> > >> QoS
> > >>> for the purpose of this e-mail.
> > >>>
> > >>> By default, neither radio button is selected and no QoS fields are
> > >> visible.
> > >>>
> > >>> If the user selects the Storage QoS radio button, then the Custom IOPS
> > >>> checkbox and the Min and Max text fields are made visible.
> > >>>
> > >>> If the user changes his mind and selects the Hypervisor QoS radio
> > button,
> > >>> then the Custom IOPS checkbox and the Min and Max text fields disappear
> > >> and
> > >>> are replaced by the two Hypervisor QoS text fields.
> > >>>
> > >>> This way, the user can choose neither QoS option or one of them or the
> > >>> other, but never both.
> > >>>
> > >>> On the API side, I was thinking of having logic in place when a request
> > >>> comes in to create a Disk Offering to confirm these fields are the way
> > we
> > >>> want them.
> > >>>
> > >>> Once we know the Disk Offering is in order, a user can create a data
> > disk
> > >>> from it. Since we checked the validity of the Disk Offering when it was
> > >>> created, the VM should never be asked to use Hypervisor QoS when
> > Storage
> > >>> QoS in being utilized.
> > >>>
> > >>> Does that make sense or did I miss something?
> > >>>
> > >>> Thanks
> > >>>
> > >>>
> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> > >> wrote:
> > >>>
> > >>>> Mike,
> > >>>>
> > >>>> Looking through the code, I am trying to understand how
> > >>>> CreateDiskOfferingCmd would have the context to identify the conflict.
> > >>>> Naively, it seems to me that this rule would need to be enforced when
> > a
> > >>>> virtual machine is being deployed. Looking through the code, it seems
> > >> like
> > >>>> we should add a private validateStorageQoS method to
> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
> > >>>> ResourceAllocationException when the QoS definitions are inconsistent.
> > >> We
> > >>>> would then add calls to it from each of the VM creation methods in the
> > >>>> service. Do this type of approach sound reasonable?
> > >>>>
> > >>>> Thanks,
> > >>>> -John
> > >>>>
> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Hi John,
> > >>>>>
> > >>>>> So, here's what I was planning to do. Of course feel free to correct
> > me
> > >>>> on
> > >>>>> this approach.
> > >>>>>
> > >>>>> I think it's OK if Wei merges his code into master and then I can
> > draw
> > >>>> from
> > >>>>> the main repo and merge master into mine locally.
> > >>>>>
> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code
> > to
> > >>>> make
> > >>>>> it user friendly (toggle between these features on the Add Disk
> > >> Offering
> > >>>>> window).
> > >>>>>
> > >>>>> 2) I plan to write validation logic for the create-disk-offering API
> > >>>>> command which throws an exception if the rules are not followed (this
> > >>>>> should never be triggered from the GUI since the GUI will have
> > controls
> > >>>> in
> > >>>>> place to toggle between the one feature and the other).
> > >>>>>
> > >>>>> I'm not sure about documentation. I haven't had much experience with
> > it
> > >>>> on
> > >>>>> CloudStack projects yet.
> > >>>>>
> > >>>>> Thanks!
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
> > >>>> wrote:
> > >>>>>
> > >>>>>> Mike,
> > >>>>>>
> > >>>>>> Yes, these server-side rails need to be defined and implemented
> > before
> > >>>>>> either patch can be merged. From my perspective, I would like to
> > see
> > >>>> the
> > >>>>>> rule implemented in the hypervisor as part of the validation of the
> > >>>> virtual
> > >>>>>> machine definition. We also need to make sure that this mutual
> > >>>> exclusion
> > >>>>>> is documented. Do we usually include this type of documentation
> > with
> > >>>>>> patches of this nature?
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> -John
> > >>>>>>
> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
> > >>>> mike.tutkowski@solidfire.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Currently they are not yet implemented.
> > >>>>>>>
> > >>>>>>> We have to make sure they are implemented in the GUI from a
> > usability
> > >>>>>>> standpoint, but the API must check for consistency and throw an
> > >>>> exception
> > >>>>>>> if necessary.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jburwell@basho.com
> > >
> > >>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Mike,
> > >>>>>>>>
> > >>>>>>>> Are the checks only implemented in the UI?
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> -John
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> > >>>>>>>> <mi...@solidfire.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi John,
> > >>>>>>>>>
> > >>>>>>>>> Wei and I have discussed making the two features mutually
> > >> exclusive.
> > >>>> We
> > >>>>>>>>> agree with you that only one should be active at a time. We plan
> > to
> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn
> > his
> > >>>>>>>> feature
> > >>>>>>>>> on and mine off and vice versa.
> > >>>>>>>>>
> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
> > update
> > >>>> and
> > >>>>>>>>> merge that I will be the person resolving merge conflicts in the
> > >>>>>>>> JavaScript
> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed
> > to
> > >>>>>>>> putting
> > >>>>>>>>> that work on someone else.
> > >>>>>>>>>
> > >>>>>>>>> Let me know what you think.
> > >>>>>>>>>
> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
> > >>>>>>>>>
> > >>>>>>>>> Thanks!
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
> > jburwell@basho.com
> > >>>
> > >>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Mike,
> > >>>>>>>>>>
> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
> > >>>> mechanisms
> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected
> > FSs
> > >>>> been
> > >>>>>>>>>> updated with that decision?
> > >>>>>>>>>>
> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
> > >> between
> > >>>>>> the
> > >>>>>>>>>> patches?
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> -John
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> > >>>>>>>>>> <mi...@solidfire.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> No problem, John.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I still want to re-review it by myself before coming up with a
> > >> new
> > >>>>>>>> patch
> > >>>>>>>>>>> file.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
> > >> in
> > >>>>>> and
> > >>>>>>>>>>> merge those into mine before generating a new patch file?
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> > >> jburwell@basho.com
> > >>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> > >> offline
> > >>>>>> ATM,
> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Sorry for the oversight,
> > >>>>>>>>>>>> -John
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
> > >> ),
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
> > we
> > >>>>>> wanted
> > >>>>>>>>>>>>> addressed.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> > afternoon,
> > >>>> then
> > >>>>>>>>>> send
> > >>>>>>>>>>>> in
> > >>>>>>>>>>>>> another patch.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Sure, that sounds good.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> > >>>> ustcweizhou@gmail.com
> > >>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi Mike,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
> > >>>> code,
> > >>>>>>>>>>>> except
> > >>>>>>>>>>>>>>> the cloudstack UI.
> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
> > >> into
> > >>>>>>>>>> master
> > >>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hey John,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> > >>>> branch,
> > >>>>>>>> but
> > >>>>>>>>>> -
> > >>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> > >>>> SolidFire
> > >>>>>>>>>> branch
> > >>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
> > >> master.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
> > merge
> > >>>>>>>>>> strategy.
> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> > >>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> > >>>>>>>> disk_io_throttling,
> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> > >>>>>>>> overlapping
> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> > >> patches
> > >>>> to
> > >>>>>>>> be
> > >>>>>>>>>> as
> > >>>>>>>>>>>>>>>>> follows:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> > >> SolidFire
> > >>>>>> are
> > >>>>>>>>>>>>>>> additive
> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> > dependency
> > >>>>>>>> between
> > >>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
> > can
> > >>>>>>>>>> determine
> > >>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
> > his
> > >>>>>> branch
> > >>>>>>>>>>>>>>> before
> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
> > >>>> merge
> > >>>>>>>> and
> > >>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
> > >> the
> > >>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> > plug-in,
> > >>>>>> which
> > >>>>>>>>>>>>>>>>> currently
> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> > (although
> > >>>> it
> > >>>>>>>> does
> > >>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>> know
> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
> > in
> > >>>>>> use))
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
> > url
> > >>>>>> field
> > >>>>>>>>>> (if
> > >>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
> > the
> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
> > in
> > >>>>>>>>>>>>>>> question, we
> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
> > of
> > >>>> the
> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> > checks
> > >>>> for
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> > hypervisor
> > >>>>>> data
> > >>>>>>>>>>>>>>>>> structure is
> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
> > >>>> attach
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>>> volume.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> > >> detach a
> > >>>>>>>>>> volume,
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> > >>>> structure
> > >>>>>>>> is
> > >>>>>>>>>>>>>>>>> removed.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
> > 4.2
> > >>>>>>>> unless
> > >>>>>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> > >> have
> > >>>>>> some
> > >>>>>>>>>>>>>>> idea,
> > >>>>>>>>>>>>>>>>>>> that'd be cool.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> > >>>>>>>>>> implementation
> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> > field.
> > >> I
> > >>>>>>>> think
> > >>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
> > >> the
> > >>>>>>>>>>>>>>> expected
> > >>>>>>>>>>>>>>>>> format
> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
> > >> not
> > >>>>>>>>>>>>>>> returned on
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> > listStoragePool
> > >>>>>>>>>> response.
> > >>>>>>>>>>>>>>>> Why
> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
> > >> for
> > >>>>>>>>>> clients
> > >>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> > >> Offering-by-Disk
> > >>>>>>>>>> Offering
> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
> > >> able
> > >>>>>> to
> > >>>>>>>>>>>>>>>> associate
> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> > >> table.
> > >>>>>>>> Maybe
> > >>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>> could
> > >>>>>>>>>>>>>>>>>>>> go there?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
> > in
> > >>>> the
> > >>>>>>>> GUI
> > >>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>> cloud<
> > http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>> *â„¢*
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>> o: 303.746.7302
> > >>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>> *â„¢*
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> *Mike Tutkowski*
> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>> o: 303.746.7302
> > >>>>>>> Advancing the way the world uses the
> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>> *â„¢*
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> *Mike Tutkowski*
> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>> e: mike.tutkowski@solidfire.com
> > >>>>> o: 303.746.7302
> > >>>>> Advancing the way the world uses the
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>> *â„¢*
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> *Mike Tutkowski*
> > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>> e: mike.tutkowski@solidfire.com
> > >>> o: 303.746.7302
> > >>> Advancing the way the world uses the
> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>> *â„¢*
> > >>
> > >>
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *â„¢*
> >
> >
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi Wei,
So, not entirely sure I follow.
Will what I wrote earlier work? Here is a copy of what I wrote:
Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage QoS
for the purpose of this e-mail.
By default, neither radio button is selected and no QoS fields are visible.
If the user selects the Storage QoS radio button, then the Custom IOPS
checkbox and the Min and Max text fields are made visible.
If the user changes his mind and selects the Hypervisor QoS radio button,
then the Custom IOPS checkbox and the Min and Max text fields disappear and
are replaced by the two Hypervisor QoS text fields.
This way, the user can choose neither QoS option or one of them or the
other, but never both.
On the API side, I was thinking of having logic in place when a request
comes in to create a Disk Offering to confirm these fields are the way we
want them.
Once we know the Disk Offering is in order, a user can create a data disk
from it. Since we checked the validity of the Disk Offering when it was
created, the VM should never be asked to use Hypervisor QoS when Storage
QoS in being utilized.
Thanks!
On Wed, Jun 12, 2013 at 3:33 PM, Wei ZHOU <us...@gmail.com> wrote:
> Mike, John
>
> The Disk I/O Throttling works like:
>
> (1) For User VM,
> (1.1) root disks: service offering -> default value in global configuration
> (1.2) data disks: disk offering -> default value in global configuration
>
> (2) For System VM
> root disks: service offering
>
> -Wei
>
>
> 2013/6/12 John Burwell <jb...@basho.com>
>
> > Mike,
> >
> > That is one possibility. The other possibility is that hypervisor is
> > going to throttle I/O on all disks attached. Therefore, we need answers
> to
> > the following questions:
> >
> > 1. Is I/O throttling applied to the root disk or all disks
> > attached to the VM?
> > 2. If I/O throttling is applied to all disks, how is the
> > throttling distributed amongst the disks if only one read/write value is
> > defined?
> >
> > Thanks,
> > -John
> >
> > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> > wrote:
> >
> > > Hey John,
> > >
> > > Perhaps I don't fully understand how Wei's feature works.
> > >
> > > I guess I thought if you choose Hypervisor QoS, you do so on Compute
> > > Offerings (root disks) and/or Disk Offerings (data disks).
> > >
> > > From my thinking, you're root disk could be under Hypervisor QoS, but
> > your
> > > data disk could be under Storage QoS.
> > >
> > > Is that incorrect?
> > >
> > > Thanks
> > >
> > >
> > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
> > wrote:
> > >
> > >> Mike,
> > >>
> > >> As I understand these two patches, the throttled I?O settings are
> > applied
> > >> from the hypervisor side, and possibly defined in a compute offering
> > where
> > >> provisioned IOPS are defined on the storage side through disk
> > offerings. I
> > >> don't see how the management server could enforce this mutual
> exclusion
> > >> between provisioned IOPS and throttled I/O until a compute and disk
> > >> offering were selected. These selections come together when we
> deploy a
> > >> VM. Based on these assumptions, I would expect to see enforcement of
> > this
> > >> rule in UI at the time of VM creation/definition and on the server
> > side, as
> > >> part of the VM creation. It feels like any attempt to enforce this
> rule
> > >> when defining offering would be premature, and unnecessarily limit
> > >> flexibility. Are my assumptions and understanding correct?
> > >>
> > >> Thanks,
> > >> -John
> > >>
> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > >> wrote:
> > >>
> > >>> Hi John,
> > >>>
> > >>> So, maybe I'm wrong about this, but what I was thinking is that we'd
> > >> build
> > >>> two radio buttons into the Add Disk Offering dialog (we can ignore
> > >> Compute
> > >>> Offerings for 4.2 since my feature doesn't yet support them).
> > >>>
> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2)
> Storage
> > >> QoS
> > >>> for the purpose of this e-mail.
> > >>>
> > >>> By default, neither radio button is selected and no QoS fields are
> > >> visible.
> > >>>
> > >>> If the user selects the Storage QoS radio button, then the Custom
> IOPS
> > >>> checkbox and the Min and Max text fields are made visible.
> > >>>
> > >>> If the user changes his mind and selects the Hypervisor QoS radio
> > button,
> > >>> then the Custom IOPS checkbox and the Min and Max text fields
> disappear
> > >> and
> > >>> are replaced by the two Hypervisor QoS text fields.
> > >>>
> > >>> This way, the user can choose neither QoS option or one of them or
> the
> > >>> other, but never both.
> > >>>
> > >>> On the API side, I was thinking of having logic in place when a
> request
> > >>> comes in to create a Disk Offering to confirm these fields are the
> way
> > we
> > >>> want them.
> > >>>
> > >>> Once we know the Disk Offering is in order, a user can create a data
> > disk
> > >>> from it. Since we checked the validity of the Disk Offering when it
> was
> > >>> created, the VM should never be asked to use Hypervisor QoS when
> > Storage
> > >>> QoS in being utilized.
> > >>>
> > >>> Does that make sense or did I miss something?
> > >>>
> > >>> Thanks
> > >>>
> > >>>
> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> > >> wrote:
> > >>>
> > >>>> Mike,
> > >>>>
> > >>>> Looking through the code, I am trying to understand how
> > >>>> CreateDiskOfferingCmd would have the context to identify the
> conflict.
> > >>>> Naively, it seems to me that this rule would need to be enforced
> when
> > a
> > >>>> virtual machine is being deployed. Looking through the code, it
> seems
> > >> like
> > >>>> we should add a private validateStorageQoS method to
> > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
> > >>>> ResourceAllocationException when the QoS definitions are
> inconsistent.
> > >> We
> > >>>> would then add calls to it from each of the VM creation methods in
> the
> > >>>> service. Do this type of approach sound reasonable?
> > >>>>
> > >>>> Thanks,
> > >>>> -John
> > >>>>
> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Hi John,
> > >>>>>
> > >>>>> So, here's what I was planning to do. Of course feel free to
> correct
> > me
> > >>>> on
> > >>>>> this approach.
> > >>>>>
> > >>>>> I think it's OK if Wei merges his code into master and then I can
> > draw
> > >>>> from
> > >>>>> the main repo and merge master into mine locally.
> > >>>>>
> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code
> > to
> > >>>> make
> > >>>>> it user friendly (toggle between these features on the Add Disk
> > >> Offering
> > >>>>> window).
> > >>>>>
> > >>>>> 2) I plan to write validation logic for the create-disk-offering
> API
> > >>>>> command which throws an exception if the rules are not followed
> (this
> > >>>>> should never be triggered from the GUI since the GUI will have
> > controls
> > >>>> in
> > >>>>> place to toggle between the one feature and the other).
> > >>>>>
> > >>>>> I'm not sure about documentation. I haven't had much experience
> with
> > it
> > >>>> on
> > >>>>> CloudStack projects yet.
> > >>>>>
> > >>>>> Thanks!
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
> > >>>> wrote:
> > >>>>>
> > >>>>>> Mike,
> > >>>>>>
> > >>>>>> Yes, these server-side rails need to be defined and implemented
> > before
> > >>>>>> either patch can be merged. From my perspective, I would like to
> > see
> > >>>> the
> > >>>>>> rule implemented in the hypervisor as part of the validation of
> the
> > >>>> virtual
> > >>>>>> machine definition. We also need to make sure that this mutual
> > >>>> exclusion
> > >>>>>> is documented. Do we usually include this type of documentation
> > with
> > >>>>>> patches of this nature?
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> -John
> > >>>>>>
> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
> > >>>> mike.tutkowski@solidfire.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Currently they are not yet implemented.
> > >>>>>>>
> > >>>>>>> We have to make sure they are implemented in the GUI from a
> > usability
> > >>>>>>> standpoint, but the API must check for consistency and throw an
> > >>>> exception
> > >>>>>>> if necessary.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <
> jburwell@basho.com
> > >
> > >>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Mike,
> > >>>>>>>>
> > >>>>>>>> Are the checks only implemented in the UI?
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> -John
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> > >>>>>>>> <mi...@solidfire.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi John,
> > >>>>>>>>>
> > >>>>>>>>> Wei and I have discussed making the two features mutually
> > >> exclusive.
> > >>>> We
> > >>>>>>>>> agree with you that only one should be active at a time. We
> plan
> > to
> > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn
> > his
> > >>>>>>>> feature
> > >>>>>>>>> on and mine off and vice versa.
> > >>>>>>>>>
> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I
> > update
> > >>>> and
> > >>>>>>>>> merge that I will be the person resolving merge conflicts in
> the
> > >>>>>>>> JavaScript
> > >>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed
> > to
> > >>>>>>>> putting
> > >>>>>>>>> that work on someone else.
> > >>>>>>>>>
> > >>>>>>>>> Let me know what you think.
> > >>>>>>>>>
> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
> > >>>>>>>>>
> > >>>>>>>>> Thanks!
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
> > jburwell@basho.com
> > >>>
> > >>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Mike,
> > >>>>>>>>>>
> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
> > >>>> mechanisms
> > >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected
> > FSs
> > >>>> been
> > >>>>>>>>>> updated with that decision?
> > >>>>>>>>>>
> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies
> > >> between
> > >>>>>> the
> > >>>>>>>>>> patches?
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> -John
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> > >>>>>>>>>> <mi...@solidfire.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> No problem, John.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I still want to re-review it by myself before coming up with
> a
> > >> new
> > >>>>>>>> patch
> > >>>>>>>>>>> file.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
> checked
> > >> in
> > >>>>>> and
> > >>>>>>>>>>> merge those into mine before generating a new patch file?
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> > >> jburwell@basho.com
> > >>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> > >> offline
> > >>>>>> ATM,
> > >>>>>>>>>>>> but I will publish it in the next couple of hours.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Sorry for the oversight,
> > >>>>>>>>>>>> -John
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> > >>>>>>>>>>>> <mi...@solidfire.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this
> :)
> > >> ),
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
> > we
> > >>>>>> wanted
> > >>>>>>>>>>>>> addressed.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> > afternoon,
> > >>>> then
> > >>>>>>>>>> send
> > >>>>>>>>>>>> in
> > >>>>>>>>>>>>> another patch.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for all the work on this everyone!
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Sure, that sounds good.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> > >>>> ustcweizhou@gmail.com
> > >>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi Mike,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
> Java
> > >>>> code,
> > >>>>>>>>>>>> except
> > >>>>>>>>>>>>>>> the cloudstack UI.
> > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
> branch
> > >> into
> > >>>>>>>>>> master
> > >>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>> week, so that you can develop based on it.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hey John,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> > >>>> branch,
> > >>>>>>>> but
> > >>>>>>>>>> -
> > >>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> > >>>> SolidFire
> > >>>>>>>>>> branch
> > >>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
> > >> master.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
> > merge
> > >>>>>>>>>> strategy.
> > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> > >>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> > >>>>>>>> disk_io_throttling,
> > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> > >>>>>>>> overlapping
> > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> > >> patches
> > >>>> to
> > >>>>>>>> be
> > >>>>>>>>>> as
> > >>>>>>>>>>>>>>>>> follows:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> > >> SolidFire
> > >>>>>> are
> > >>>>>>>>>>>>>>> additive
> > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> > dependency
> > >>>>>>>> between
> > >>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
> > can
> > >>>>>>>>>> determine
> > >>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> best approach to land the changes in master.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
> > his
> > >>>>>> branch
> > >>>>>>>>>>>>>>> before
> > >>>>>>>>>>>>>>>>>> going into master, I am good with that.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after
> his
> > >>>> merge
> > >>>>>>>> and
> > >>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code
> in
> > >> the
> > >>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> > plug-in,
> > >>>>>> which
> > >>>>>>>>>>>>>>>>> currently
> > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> > (although
> > >>>> it
> > >>>>>>>> does
> > >>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>> know
> > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
> actually
> > in
> > >>>>>> use))
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
> > url
> > >>>>>> field
> > >>>>>>>>>> (if
> > >>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
> > the
> > >>>>>>>>>>>>>>>>>>> storage_pool_details table.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
> hypervisor
> > in
> > >>>>>>>>>>>>>>> question, we
> > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
> > of
> > >>>> the
> > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> > checks
> > >>>> for
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> > hypervisor
> > >>>>>> data
> > >>>>>>>>>>>>>>>>> structure is
> > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes
> to
> > >>>> attach
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>>> volume.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> > >> detach a
> > >>>>>>>>>> volume,
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> > >>>> structure
> > >>>>>>>> is
> > >>>>>>>>>>>>>>>>> removed.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
> > 4.2
> > >>>>>>>> unless
> > >>>>>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> > >> have
> > >>>>>> some
> > >>>>>>>>>>>>>>> idea,
> > >>>>>>>>>>>>>>>>>>> that'd be cool.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> > >>>>>>>>>> implementation
> > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> > field.
> > >> I
> > >>>>>>>> think
> > >>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
> regarding
> > >> the
> > >>>>>>>>>>>>>>> expected
> > >>>>>>>>>>>>>>>>> format
> > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
> are
> > >> not
> > >>>>>>>>>>>>>>> returned on
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> > listStoragePool
> > >>>>>>>>>> response.
> > >>>>>>>>>>>>>>>> Why
> > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
> useful
> > >> for
> > >>>>>>>>>> clients
> > >>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
> here.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> > >> Offering-by-Disk
> > >>>>>>>>>> Offering
> > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to
> be
> > >> able
> > >>>>>> to
> > >>>>>>>>>>>>>>>> associate
> > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> > >> table.
> > >>>>>>>> Maybe
> > >>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>> could
> > >>>>>>>>>>>>>>>>>>>> go there?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
> IOPS
> > in
> > >>>> the
> > >>>>>>>> GUI
> > >>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
> DB.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>> cloud<
> > http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>> *â„¢*
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>> *â„¢*
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>> o: 303.746.7302
> > >>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>> *â„¢*
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> *Mike Tutkowski*
> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>> o: 303.746.7302
> > >>>>>>> Advancing the way the world uses the
> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>> *â„¢*
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> *Mike Tutkowski*
> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>> e: mike.tutkowski@solidfire.com
> > >>>>> o: 303.746.7302
> > >>>>> Advancing the way the world uses the
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>> *â„¢*
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> *Mike Tutkowski*
> > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>> e: mike.tutkowski@solidfire.com
> > >>> o: 303.746.7302
> > >>> Advancing the way the world uses the
> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>> *â„¢*
> > >>
> > >>
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *â„¢*
> >
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
Mike, John
The Disk I/O Throttling works like:
(1) For User VM,
(1.1) root disks: service offering -> default value in global configuration
(1.2) data disks: disk offering -> default value in global configuration
(2) For System VM
root disks: service offering
-Wei
2013/6/12 John Burwell <jb...@basho.com>
> Mike,
>
> That is one possibility. The other possibility is that hypervisor is
> going to throttle I/O on all disks attached. Therefore, we need answers to
> the following questions:
>
> 1. Is I/O throttling applied to the root disk or all disks
> attached to the VM?
> 2. If I/O throttling is applied to all disks, how is the
> throttling distributed amongst the disks if only one read/write value is
> defined?
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hey John,
> >
> > Perhaps I don't fully understand how Wei's feature works.
> >
> > I guess I thought if you choose Hypervisor QoS, you do so on Compute
> > Offerings (root disks) and/or Disk Offerings (data disks).
> >
> > From my thinking, you're root disk could be under Hypervisor QoS, but
> your
> > data disk could be under Storage QoS.
> >
> > Is that incorrect?
> >
> > Thanks
> >
> >
> > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> As I understand these two patches, the throttled I?O settings are
> applied
> >> from the hypervisor side, and possibly defined in a compute offering
> where
> >> provisioned IOPS are defined on the storage side through disk
> offerings. I
> >> don't see how the management server could enforce this mutual exclusion
> >> between provisioned IOPS and throttled I/O until a compute and disk
> >> offering were selected. These selections come together when we deploy a
> >> VM. Based on these assumptions, I would expect to see enforcement of
> this
> >> rule in UI at the time of VM creation/definition and on the server
> side, as
> >> part of the VM creation. It feels like any attempt to enforce this rule
> >> when defining offering would be premature, and unnecessarily limit
> >> flexibility. Are my assumptions and understanding correct?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Hi John,
> >>>
> >>> So, maybe I'm wrong about this, but what I was thinking is that we'd
> >> build
> >>> two radio buttons into the Add Disk Offering dialog (we can ignore
> >> Compute
> >>> Offerings for 4.2 since my feature doesn't yet support them).
> >>>
> >>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
> >> QoS
> >>> for the purpose of this e-mail.
> >>>
> >>> By default, neither radio button is selected and no QoS fields are
> >> visible.
> >>>
> >>> If the user selects the Storage QoS radio button, then the Custom IOPS
> >>> checkbox and the Min and Max text fields are made visible.
> >>>
> >>> If the user changes his mind and selects the Hypervisor QoS radio
> button,
> >>> then the Custom IOPS checkbox and the Min and Max text fields disappear
> >> and
> >>> are replaced by the two Hypervisor QoS text fields.
> >>>
> >>> This way, the user can choose neither QoS option or one of them or the
> >>> other, but never both.
> >>>
> >>> On the API side, I was thinking of having logic in place when a request
> >>> comes in to create a Disk Offering to confirm these fields are the way
> we
> >>> want them.
> >>>
> >>> Once we know the Disk Offering is in order, a user can create a data
> disk
> >>> from it. Since we checked the validity of the Disk Offering when it was
> >>> created, the VM should never be asked to use Hypervisor QoS when
> Storage
> >>> QoS in being utilized.
> >>>
> >>> Does that make sense or did I miss something?
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Looking through the code, I am trying to understand how
> >>>> CreateDiskOfferingCmd would have the context to identify the conflict.
> >>>> Naively, it seems to me that this rule would need to be enforced when
> a
> >>>> virtual machine is being deployed. Looking through the code, it seems
> >> like
> >>>> we should add a private validateStorageQoS method to
> >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
> >>>> ResourceAllocationException when the QoS definitions are inconsistent.
> >> We
> >>>> would then add calls to it from each of the VM creation methods in the
> >>>> service. Do this type of approach sound reasonable?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> Hi John,
> >>>>>
> >>>>> So, here's what I was planning to do. Of course feel free to correct
> me
> >>>> on
> >>>>> this approach.
> >>>>>
> >>>>> I think it's OK if Wei merges his code into master and then I can
> draw
> >>>> from
> >>>>> the main repo and merge master into mine locally.
> >>>>>
> >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code
> to
> >>>> make
> >>>>> it user friendly (toggle between these features on the Add Disk
> >> Offering
> >>>>> window).
> >>>>>
> >>>>> 2) I plan to write validation logic for the create-disk-offering API
> >>>>> command which throws an exception if the rules are not followed (this
> >>>>> should never be triggered from the GUI since the GUI will have
> controls
> >>>> in
> >>>>> place to toggle between the one feature and the other).
> >>>>>
> >>>>> I'm not sure about documentation. I haven't had much experience with
> it
> >>>> on
> >>>>> CloudStack projects yet.
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Yes, these server-side rails need to be defined and implemented
> before
> >>>>>> either patch can be merged. From my perspective, I would like to
> see
> >>>> the
> >>>>>> rule implemented in the hypervisor as part of the validation of the
> >>>> virtual
> >>>>>> machine definition. We also need to make sure that this mutual
> >>>> exclusion
> >>>>>> is documented. Do we usually include this type of documentation
> with
> >>>>>> patches of this nature?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Currently they are not yet implemented.
> >>>>>>>
> >>>>>>> We have to make sure they are implemented in the GUI from a
> usability
> >>>>>>> standpoint, but the API must check for consistency and throw an
> >>>> exception
> >>>>>>> if necessary.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jburwell@basho.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> Are the checks only implemented in the UI?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> >>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi John,
> >>>>>>>>>
> >>>>>>>>> Wei and I have discussed making the two features mutually
> >> exclusive.
> >>>> We
> >>>>>>>>> agree with you that only one should be active at a time. We plan
> to
> >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn
> his
> >>>>>>>> feature
> >>>>>>>>> on and mine off and vice versa.
> >>>>>>>>>
> >>>>>>>>> I was thinking if I wait until he checks in his code, then I
> update
> >>>> and
> >>>>>>>>> merge that I will be the person resolving merge conflicts in the
> >>>>>>>> JavaScript
> >>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed
> to
> >>>>>>>> putting
> >>>>>>>>> that work on someone else.
> >>>>>>>>>
> >>>>>>>>> Let me know what you think.
> >>>>>>>>>
> >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
> jburwell@basho.com
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
> >>>> mechanisms
> >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected
> FSs
> >>>> been
> >>>>>>>>>> updated with that decision?
> >>>>>>>>>>
> >>>>>>>>>> In terms of merge timing, can you describe the dependencies
> >> between
> >>>>>> the
> >>>>>>>>>> patches?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> >>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> No problem, John.
> >>>>>>>>>>>
> >>>>>>>>>>> I still want to re-review it by myself before coming up with a
> >> new
> >>>>>>>> patch
> >>>>>>>>>>> file.
> >>>>>>>>>>>
> >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
> >> in
> >>>>>> and
> >>>>>>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >> jburwell@basho.com
> >>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> >> offline
> >>>>>> ATM,
> >>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
> >> ),
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
> we
> >>>>>> wanted
> >>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> afternoon,
> >>>> then
> >>>>>>>>>> send
> >>>>>>>>>>>> in
> >>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>> ustcweizhou@gmail.com
> >>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
> >>>> code,
> >>>>>>>>>>>> except
> >>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
> >> into
> >>>>>>>>>> master
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> >>>> branch,
> >>>>>>>> but
> >>>>>>>>>> -
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> >>>> SolidFire
> >>>>>>>>>> branch
> >>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
> >> master.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
> merge
> >>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> >>>>>>>> overlapping
> >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> >> patches
> >>>> to
> >>>>>>>> be
> >>>>>>>>>> as
> >>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> >> SolidFire
> >>>>>> are
> >>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> dependency
> >>>>>>>> between
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
> can
> >>>>>>>>>> determine
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
> his
> >>>>>> branch
> >>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
> >>>> merge
> >>>>>>>> and
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
> >> the
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> plug-in,
> >>>>>> which
> >>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> (although
> >>>> it
> >>>>>>>> does
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
> in
> >>>>>> use))
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
> url
> >>>>>> field
> >>>>>>>>>> (if
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
> the
> >>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
> in
> >>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
> of
> >>>> the
> >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> checks
> >>>> for
> >>>>>>>> the
> >>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> hypervisor
> >>>>>> data
> >>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
> >>>> attach
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> >> detach a
> >>>>>>>>>> volume,
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> >>>> structure
> >>>>>>>> is
> >>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
> 4.2
> >>>>>>>> unless
> >>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> >> have
> >>>>>> some
> >>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> field.
> >> I
> >>>>>>>> think
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
> >> the
> >>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
> >> not
> >>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> listStoragePool
> >>>>>>>>>> response.
> >>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
> >> for
> >>>>>>>>>> clients
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >> Offering-by-Disk
> >>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
> >> able
> >>>>>> to
> >>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> >> table.
> >>>>>>>> Maybe
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
> in
> >>>> the
> >>>>>>>> GUI
> >>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
OK, we will have to wait for Wei on that one.
My guess (and that's all it is is a guess) is since Wei's four new fields
have been added to both the Add Compute Offering and the Add Disk Offering
dialogs is that Hypervisor QoS is on a disk-by-disk basis. Otherwise you
could have conflicting information from disks that were created from
different Disk Offerings.
On Wed, Jun 12, 2013 at 3:22 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> That is one possibility. The other possibility is that hypervisor is
> going to throttle I/O on all disks attached. Therefore, we need answers to
> the following questions:
>
> 1. Is I/O throttling applied to the root disk or all disks
> attached to the VM?
> 2. If I/O throttling is applied to all disks, how is the
> throttling distributed amongst the disks if only one read/write value is
> defined?
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hey John,
> >
> > Perhaps I don't fully understand how Wei's feature works.
> >
> > I guess I thought if you choose Hypervisor QoS, you do so on Compute
> > Offerings (root disks) and/or Disk Offerings (data disks).
> >
> > From my thinking, you're root disk could be under Hypervisor QoS, but
> your
> > data disk could be under Storage QoS.
> >
> > Is that incorrect?
> >
> > Thanks
> >
> >
> > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> As I understand these two patches, the throttled I?O settings are
> applied
> >> from the hypervisor side, and possibly defined in a compute offering
> where
> >> provisioned IOPS are defined on the storage side through disk
> offerings. I
> >> don't see how the management server could enforce this mutual exclusion
> >> between provisioned IOPS and throttled I/O until a compute and disk
> >> offering were selected. These selections come together when we deploy a
> >> VM. Based on these assumptions, I would expect to see enforcement of
> this
> >> rule in UI at the time of VM creation/definition and on the server
> side, as
> >> part of the VM creation. It feels like any attempt to enforce this rule
> >> when defining offering would be premature, and unnecessarily limit
> >> flexibility. Are my assumptions and understanding correct?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Hi John,
> >>>
> >>> So, maybe I'm wrong about this, but what I was thinking is that we'd
> >> build
> >>> two radio buttons into the Add Disk Offering dialog (we can ignore
> >> Compute
> >>> Offerings for 4.2 since my feature doesn't yet support them).
> >>>
> >>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
> >> QoS
> >>> for the purpose of this e-mail.
> >>>
> >>> By default, neither radio button is selected and no QoS fields are
> >> visible.
> >>>
> >>> If the user selects the Storage QoS radio button, then the Custom IOPS
> >>> checkbox and the Min and Max text fields are made visible.
> >>>
> >>> If the user changes his mind and selects the Hypervisor QoS radio
> button,
> >>> then the Custom IOPS checkbox and the Min and Max text fields disappear
> >> and
> >>> are replaced by the two Hypervisor QoS text fields.
> >>>
> >>> This way, the user can choose neither QoS option or one of them or the
> >>> other, but never both.
> >>>
> >>> On the API side, I was thinking of having logic in place when a request
> >>> comes in to create a Disk Offering to confirm these fields are the way
> we
> >>> want them.
> >>>
> >>> Once we know the Disk Offering is in order, a user can create a data
> disk
> >>> from it. Since we checked the validity of the Disk Offering when it was
> >>> created, the VM should never be asked to use Hypervisor QoS when
> Storage
> >>> QoS in being utilized.
> >>>
> >>> Does that make sense or did I miss something?
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Looking through the code, I am trying to understand how
> >>>> CreateDiskOfferingCmd would have the context to identify the conflict.
> >>>> Naively, it seems to me that this rule would need to be enforced when
> a
> >>>> virtual machine is being deployed. Looking through the code, it seems
> >> like
> >>>> we should add a private validateStorageQoS method to
> >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
> >>>> ResourceAllocationException when the QoS definitions are inconsistent.
> >> We
> >>>> would then add calls to it from each of the VM creation methods in the
> >>>> service. Do this type of approach sound reasonable?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> Hi John,
> >>>>>
> >>>>> So, here's what I was planning to do. Of course feel free to correct
> me
> >>>> on
> >>>>> this approach.
> >>>>>
> >>>>> I think it's OK if Wei merges his code into master and then I can
> draw
> >>>> from
> >>>>> the main repo and merge master into mine locally.
> >>>>>
> >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code
> to
> >>>> make
> >>>>> it user friendly (toggle between these features on the Add Disk
> >> Offering
> >>>>> window).
> >>>>>
> >>>>> 2) I plan to write validation logic for the create-disk-offering API
> >>>>> command which throws an exception if the rules are not followed (this
> >>>>> should never be triggered from the GUI since the GUI will have
> controls
> >>>> in
> >>>>> place to toggle between the one feature and the other).
> >>>>>
> >>>>> I'm not sure about documentation. I haven't had much experience with
> it
> >>>> on
> >>>>> CloudStack projects yet.
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Yes, these server-side rails need to be defined and implemented
> before
> >>>>>> either patch can be merged. From my perspective, I would like to
> see
> >>>> the
> >>>>>> rule implemented in the hypervisor as part of the validation of the
> >>>> virtual
> >>>>>> machine definition. We also need to make sure that this mutual
> >>>> exclusion
> >>>>>> is documented. Do we usually include this type of documentation
> with
> >>>>>> patches of this nature?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Currently they are not yet implemented.
> >>>>>>>
> >>>>>>> We have to make sure they are implemented in the GUI from a
> usability
> >>>>>>> standpoint, but the API must check for consistency and throw an
> >>>> exception
> >>>>>>> if necessary.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jburwell@basho.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> Are the checks only implemented in the UI?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> >>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi John,
> >>>>>>>>>
> >>>>>>>>> Wei and I have discussed making the two features mutually
> >> exclusive.
> >>>> We
> >>>>>>>>> agree with you that only one should be active at a time. We plan
> to
> >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn
> his
> >>>>>>>> feature
> >>>>>>>>> on and mine off and vice versa.
> >>>>>>>>>
> >>>>>>>>> I was thinking if I wait until he checks in his code, then I
> update
> >>>> and
> >>>>>>>>> merge that I will be the person resolving merge conflicts in the
> >>>>>>>> JavaScript
> >>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed
> to
> >>>>>>>> putting
> >>>>>>>>> that work on someone else.
> >>>>>>>>>
> >>>>>>>>> Let me know what you think.
> >>>>>>>>>
> >>>>>>>>> Oh, was going to ask you what "FS" stands for here.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <
> jburwell@basho.com
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
> >>>> mechanisms
> >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected
> FSs
> >>>> been
> >>>>>>>>>> updated with that decision?
> >>>>>>>>>>
> >>>>>>>>>> In terms of merge timing, can you describe the dependencies
> >> between
> >>>>>> the
> >>>>>>>>>> patches?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> >>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> No problem, John.
> >>>>>>>>>>>
> >>>>>>>>>>> I still want to re-review it by myself before coming up with a
> >> new
> >>>>>>>> patch
> >>>>>>>>>>> file.
> >>>>>>>>>>>
> >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
> >> in
> >>>>>> and
> >>>>>>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >> jburwell@basho.com
> >>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> >> offline
> >>>>>> ATM,
> >>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
> >> ),
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
> we
> >>>>>> wanted
> >>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> afternoon,
> >>>> then
> >>>>>>>>>> send
> >>>>>>>>>>>> in
> >>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>> ustcweizhou@gmail.com
> >>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
> >>>> code,
> >>>>>>>>>>>> except
> >>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
> >> into
> >>>>>>>>>> master
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> >>>> branch,
> >>>>>>>> but
> >>>>>>>>>> -
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> >>>> SolidFire
> >>>>>>>>>> branch
> >>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
> >> master.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
> merge
> >>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> >>>>>>>> overlapping
> >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> >> patches
> >>>> to
> >>>>>>>> be
> >>>>>>>>>> as
> >>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> >> SolidFire
> >>>>>> are
> >>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> dependency
> >>>>>>>> between
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
> can
> >>>>>>>>>> determine
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
> his
> >>>>>> branch
> >>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
> >>>> merge
> >>>>>>>> and
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
> >> the
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> plug-in,
> >>>>>> which
> >>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> (although
> >>>> it
> >>>>>>>> does
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
> in
> >>>>>> use))
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
> url
> >>>>>> field
> >>>>>>>>>> (if
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
> the
> >>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
> in
> >>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
> of
> >>>> the
> >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> checks
> >>>> for
> >>>>>>>> the
> >>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> hypervisor
> >>>>>> data
> >>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
> >>>> attach
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> >> detach a
> >>>>>>>>>> volume,
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> >>>> structure
> >>>>>>>> is
> >>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
> 4.2
> >>>>>>>> unless
> >>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> >> have
> >>>>>> some
> >>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> field.
> >> I
> >>>>>>>> think
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
> >> the
> >>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
> >> not
> >>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> listStoragePool
> >>>>>>>>>> response.
> >>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
> >> for
> >>>>>>>>>> clients
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >> Offering-by-Disk
> >>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
> >> able
> >>>>>> to
> >>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> >> table.
> >>>>>>>> Maybe
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
> in
> >>>> the
> >>>>>>>> GUI
> >>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
That is one possibility. The other possibility is that hypervisor is going to throttle I/O on all disks attached. Therefore, we need answers to the following questions:
1. Is I/O throttling applied to the root disk or all disks attached to the VM?
2. If I/O throttling is applied to all disks, how is the throttling distributed amongst the disks if only one read/write value is defined?
Thanks,
-John
On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hey John,
>
> Perhaps I don't fully understand how Wei's feature works.
>
> I guess I thought if you choose Hypervisor QoS, you do so on Compute
> Offerings (root disks) and/or Disk Offerings (data disks).
>
> From my thinking, you're root disk could be under Hypervisor QoS, but your
> data disk could be under Storage QoS.
>
> Is that incorrect?
>
> Thanks
>
>
> On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> As I understand these two patches, the throttled I?O settings are applied
>> from the hypervisor side, and possibly defined in a compute offering where
>> provisioned IOPS are defined on the storage side through disk offerings. I
>> don't see how the management server could enforce this mutual exclusion
>> between provisioned IOPS and throttled I/O until a compute and disk
>> offering were selected. These selections come together when we deploy a
>> VM. Based on these assumptions, I would expect to see enforcement of this
>> rule in UI at the time of VM creation/definition and on the server side, as
>> part of the VM creation. It feels like any attempt to enforce this rule
>> when defining offering would be premature, and unnecessarily limit
>> flexibility. Are my assumptions and understanding correct?
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Hi John,
>>>
>>> So, maybe I'm wrong about this, but what I was thinking is that we'd
>> build
>>> two radio buttons into the Add Disk Offering dialog (we can ignore
>> Compute
>>> Offerings for 4.2 since my feature doesn't yet support them).
>>>
>>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
>> QoS
>>> for the purpose of this e-mail.
>>>
>>> By default, neither radio button is selected and no QoS fields are
>> visible.
>>>
>>> If the user selects the Storage QoS radio button, then the Custom IOPS
>>> checkbox and the Min and Max text fields are made visible.
>>>
>>> If the user changes his mind and selects the Hypervisor QoS radio button,
>>> then the Custom IOPS checkbox and the Min and Max text fields disappear
>> and
>>> are replaced by the two Hypervisor QoS text fields.
>>>
>>> This way, the user can choose neither QoS option or one of them or the
>>> other, but never both.
>>>
>>> On the API side, I was thinking of having logic in place when a request
>>> comes in to create a Disk Offering to confirm these fields are the way we
>>> want them.
>>>
>>> Once we know the Disk Offering is in order, a user can create a data disk
>>> from it. Since we checked the validity of the Disk Offering when it was
>>> created, the VM should never be asked to use Hypervisor QoS when Storage
>>> QoS in being utilized.
>>>
>>> Does that make sense or did I miss something?
>>>
>>> Thanks
>>>
>>>
>>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> Looking through the code, I am trying to understand how
>>>> CreateDiskOfferingCmd would have the context to identify the conflict.
>>>> Naively, it seems to me that this rule would need to be enforced when a
>>>> virtual machine is being deployed. Looking through the code, it seems
>> like
>>>> we should add a private validateStorageQoS method to
>>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
>>>> ResourceAllocationException when the QoS definitions are inconsistent.
>> We
>>>> would then add calls to it from each of the VM creation methods in the
>>>> service. Do this type of approach sound reasonable?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> So, here's what I was planning to do. Of course feel free to correct me
>>>> on
>>>>> this approach.
>>>>>
>>>>> I think it's OK if Wei merges his code into master and then I can draw
>>>> from
>>>>> the main repo and merge master into mine locally.
>>>>>
>>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code to
>>>> make
>>>>> it user friendly (toggle between these features on the Add Disk
>> Offering
>>>>> window).
>>>>>
>>>>> 2) I plan to write validation logic for the create-disk-offering API
>>>>> command which throws an exception if the rules are not followed (this
>>>>> should never be triggered from the GUI since the GUI will have controls
>>>> in
>>>>> place to toggle between the one feature and the other).
>>>>>
>>>>> I'm not sure about documentation. I haven't had much experience with it
>>>> on
>>>>> CloudStack projects yet.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Yes, these server-side rails need to be defined and implemented before
>>>>>> either patch can be merged. From my perspective, I would like to see
>>>> the
>>>>>> rule implemented in the hypervisor as part of the validation of the
>>>> virtual
>>>>>> machine definition. We also need to make sure that this mutual
>>>> exclusion
>>>>>> is documented. Do we usually include this type of documentation with
>>>>>> patches of this nature?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Currently they are not yet implemented.
>>>>>>>
>>>>>>> We have to make sure they are implemented in the GUI from a usability
>>>>>>> standpoint, but the API must check for consistency and throw an
>>>> exception
>>>>>>> if necessary.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> Are the checks only implemented in the UI?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Hi John,
>>>>>>>>>
>>>>>>>>> Wei and I have discussed making the two features mutually
>> exclusive.
>>>> We
>>>>>>>>> agree with you that only one should be active at a time. We plan to
>>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn his
>>>>>>>> feature
>>>>>>>>> on and mine off and vice versa.
>>>>>>>>>
>>>>>>>>> I was thinking if I wait until he checks in his code, then I update
>>>> and
>>>>>>>>> merge that I will be the person resolving merge conflicts in the
>>>>>>>> JavaScript
>>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed to
>>>>>>>> putting
>>>>>>>>> that work on someone else.
>>>>>>>>>
>>>>>>>>> Let me know what you think.
>>>>>>>>>
>>>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jburwell@basho.com
>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>>>> mechanisms
>>>>>>>>>> between the Hypervisor and Storage layers? Have the affected FSs
>>>> been
>>>>>>>>>> updated with that decision?
>>>>>>>>>>
>>>>>>>>>> In terms of merge timing, can you describe the dependencies
>> between
>>>>>> the
>>>>>>>>>> patches?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> No problem, John.
>>>>>>>>>>>
>>>>>>>>>>> I still want to re-review it by myself before coming up with a
>> new
>>>>>>>> patch
>>>>>>>>>>> file.
>>>>>>>>>>>
>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
>> in
>>>>>> and
>>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> jburwell@basho.com
>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>> offline
>>>>>> ATM,
>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>
>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>
>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
>> ),
>>>>>>>>>>>>>
>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
>>>>>> wanted
>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
>>>> then
>>>>>>>>>> send
>>>>>>>>>>>> in
>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>> ustcweizhou@gmail.com
>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>>>> code,
>>>>>>>>>>>> except
>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
>> into
>>>>>>>>>> master
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>>>> branch,
>>>>>>>> but
>>>>>>>>>> -
>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>> SolidFire
>>>>>>>>>> branch
>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>> master.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>>>>>>> overlapping
>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>> patches
>>>> to
>>>>>>>> be
>>>>>>>>>> as
>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>> SolidFire
>>>>>> are
>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>> to the object_store branch or there are circular dependency
>>>>>>>> between
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>>>>>>>> determine
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>>>>>> branch
>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>>> merge
>>>>>>>> and
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
>> the
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>>>>>> which
>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although
>>>> it
>>>>>>>> does
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>>>>>> use))
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>>>>>> field
>>>>>>>>>> (if
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of
>>>> the
>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
>>>> for
>>>>>>>> the
>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>>>>>> data
>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>>>> attach
>>>>>>>> the
>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>> detach a
>>>>>>>>>> volume,
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>>> structure
>>>>>>>> is
>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>>>>>>>> unless
>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
>> have
>>>>>> some
>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details field.
>> I
>>>>>>>> think
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
>> the
>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
>> not
>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>>>>>>>> response.
>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
>> for
>>>>>>>>>> clients
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> Offering-by-Disk
>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>> able
>>>>>> to
>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>> table.
>>>>>>>> Maybe
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
>>>> the
>>>>>>>> GUI
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hey John,
Perhaps I don't fully understand how Wei's feature works.
I guess I thought if you choose Hypervisor QoS, you do so on Compute
Offerings (root disks) and/or Disk Offerings (data disks).
>From my thinking, you're root disk could be under Hypervisor QoS, but your
data disk could be under Storage QoS.
Is that incorrect?
Thanks
On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> As I understand these two patches, the throttled I?O settings are applied
> from the hypervisor side, and possibly defined in a compute offering where
> provisioned IOPS are defined on the storage side through disk offerings. I
> don't see how the management server could enforce this mutual exclusion
> between provisioned IOPS and throttled I/O until a compute and disk
> offering were selected. These selections come together when we deploy a
> VM. Based on these assumptions, I would expect to see enforcement of this
> rule in UI at the time of VM creation/definition and on the server side, as
> part of the VM creation. It feels like any attempt to enforce this rule
> when defining offering would be premature, and unnecessarily limit
> flexibility. Are my assumptions and understanding correct?
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hi John,
> >
> > So, maybe I'm wrong about this, but what I was thinking is that we'd
> build
> > two radio buttons into the Add Disk Offering dialog (we can ignore
> Compute
> > Offerings for 4.2 since my feature doesn't yet support them).
> >
> > Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage
> QoS
> > for the purpose of this e-mail.
> >
> > By default, neither radio button is selected and no QoS fields are
> visible.
> >
> > If the user selects the Storage QoS radio button, then the Custom IOPS
> > checkbox and the Min and Max text fields are made visible.
> >
> > If the user changes his mind and selects the Hypervisor QoS radio button,
> > then the Custom IOPS checkbox and the Min and Max text fields disappear
> and
> > are replaced by the two Hypervisor QoS text fields.
> >
> > This way, the user can choose neither QoS option or one of them or the
> > other, but never both.
> >
> > On the API side, I was thinking of having logic in place when a request
> > comes in to create a Disk Offering to confirm these fields are the way we
> > want them.
> >
> > Once we know the Disk Offering is in order, a user can create a data disk
> > from it. Since we checked the validity of the Disk Offering when it was
> > created, the VM should never be asked to use Hypervisor QoS when Storage
> > QoS in being utilized.
> >
> > Does that make sense or did I miss something?
> >
> > Thanks
> >
> >
> > On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Looking through the code, I am trying to understand how
> >> CreateDiskOfferingCmd would have the context to identify the conflict.
> >> Naively, it seems to me that this rule would need to be enforced when a
> >> virtual machine is being deployed. Looking through the code, it seems
> like
> >> we should add a private validateStorageQoS method to
> >> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
> >> ResourceAllocationException when the QoS definitions are inconsistent.
> We
> >> would then add calls to it from each of the VM creation methods in the
> >> service. Do this type of approach sound reasonable?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Hi John,
> >>>
> >>> So, here's what I was planning to do. Of course feel free to correct me
> >> on
> >>> this approach.
> >>>
> >>> I think it's OK if Wei merges his code into master and then I can draw
> >> from
> >>> the main repo and merge master into mine locally.
> >>>
> >>> 1) Once I get Wei's code and merge, I plan to add a little GUI code to
> >> make
> >>> it user friendly (toggle between these features on the Add Disk
> Offering
> >>> window).
> >>>
> >>> 2) I plan to write validation logic for the create-disk-offering API
> >>> command which throws an exception if the rules are not followed (this
> >>> should never be triggered from the GUI since the GUI will have controls
> >> in
> >>> place to toggle between the one feature and the other).
> >>>
> >>> I'm not sure about documentation. I haven't had much experience with it
> >> on
> >>> CloudStack projects yet.
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Yes, these server-side rails need to be defined and implemented before
> >>>> either patch can be merged. From my perspective, I would like to see
> >> the
> >>>> rule implemented in the hypervisor as part of the validation of the
> >> virtual
> >>>> machine definition. We also need to make sure that this mutual
> >> exclusion
> >>>> is documented. Do we usually include this type of documentation with
> >>>> patches of this nature?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> Currently they are not yet implemented.
> >>>>>
> >>>>> We have to make sure they are implemented in the GUI from a usability
> >>>>> standpoint, but the API must check for consistency and throw an
> >> exception
> >>>>> if necessary.
> >>>>>
> >>>>>
> >>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Are the checks only implemented in the UI?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> >>>>>> <mi...@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Hi John,
> >>>>>>>
> >>>>>>> Wei and I have discussed making the two features mutually
> exclusive.
> >> We
> >>>>>>> agree with you that only one should be active at a time. We plan to
> >>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn his
> >>>>>> feature
> >>>>>>> on and mine off and vice versa.
> >>>>>>>
> >>>>>>> I was thinking if I wait until he checks in his code, then I update
> >> and
> >>>>>>> merge that I will be the person resolving merge conflicts in the
> >>>>>> JavaScript
> >>>>>>> code (there shouldn't be a problem in the Java code) as opposed to
> >>>>>> putting
> >>>>>>> that work on someone else.
> >>>>>>>
> >>>>>>> Let me know what you think.
> >>>>>>>
> >>>>>>> Oh, was going to ask you what "FS" stands for here.
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jburwell@basho.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> How have Wei and you resolved the issue of conflicting QoS
> >> mechanisms
> >>>>>>>> between the Hypervisor and Storage layers? Have the affected FSs
> >> been
> >>>>>>>> updated with that decision?
> >>>>>>>>
> >>>>>>>> In terms of merge timing, can you describe the dependencies
> between
> >>>> the
> >>>>>>>> patches?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> >>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> No problem, John.
> >>>>>>>>>
> >>>>>>>>> I still want to re-review it by myself before coming up with a
> new
> >>>>>> patch
> >>>>>>>>> file.
> >>>>>>>>>
> >>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
> in
> >>>> and
> >>>>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> jburwell@basho.com
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> I just realized that I forgot to publish my review. I am
> offline
> >>>> ATM,
> >>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>
> >>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>
> >>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
> ),
> >>>>>>>>>>>
> >>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
> >>>> wanted
> >>>>>>>>>>> addressed.
> >>>>>>>>>>>
> >>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
> >> then
> >>>>>>>> send
> >>>>>>>>>> in
> >>>>>>>>>>> another patch.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >> ustcweizhou@gmail.com
> >>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
> >> code,
> >>>>>>>>>> except
> >>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
> into
> >>>>>>>> master
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> >> branch,
> >>>>>> but
> >>>>>>>> -
> >>>>>>>>>> as
> >>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> >> SolidFire
> >>>>>>>> branch
> >>>>>>>>>>>>> into
> >>>>>>>>>>>>>> the object_store branch before object_store goes into
> master.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
> >>>>>>>> strategy.
> >>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> >>>>>> overlapping
> >>>>>>>>>>>>>>> elements. I understand the dependencies between the
> patches
> >> to
> >>>>>> be
> >>>>>>>> as
> >>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Am I correct that the device management aspects of
> SolidFire
> >>>> are
> >>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>> to the object_store branch or there are circular dependency
> >>>>>> between
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
> >>>>>>>> determine
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
> >>>> branch
> >>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
> >> merge
> >>>>>> and
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
> the
> >>>>>>>> storage
> >>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
> >>>> which
> >>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although
> >> it
> >>>>>> does
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
> >>>> use))
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
> >>>> field
> >>>>>>>> (if
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of
> >> the
> >>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
> >> for
> >>>>>> the
> >>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
> >>>> data
> >>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
> >> attach
> >>>>>> the
> >>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> detach a
> >>>>>>>> volume,
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> >> structure
> >>>>>> is
> >>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
> >>>>>> unless
> >>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> have
> >>>> some
> >>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>>>>>> implementation
> >>>>>>>>>>>>>>>>>> attributes. I always wondered about the details field.
> I
> >>>>>> think
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
> the
> >>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
> not
> >>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> >>>>>>>> response.
> >>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
> for
> >>>>>>>> clients
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> Offering-by-Disk
> >>>>>>>> Offering
> >>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
> able
> >>>> to
> >>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> table.
> >>>>>> Maybe
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
> >> the
> >>>>>> GUI
> >>>>>>>>>>>>> if
> >>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
As I understand these two patches, the throttled I?O settings are applied from the hypervisor side, and possibly defined in a compute offering where provisioned IOPS are defined on the storage side through disk offerings. I don't see how the management server could enforce this mutual exclusion between provisioned IOPS and throttled I/O until a compute and disk offering were selected. These selections come together when we deploy a VM. Based on these assumptions, I would expect to see enforcement of this rule in UI at the time of VM creation/definition and on the server side, as part of the VM creation. It feels like any attempt to enforce this rule when defining offering would be premature, and unnecessarily limit flexibility. Are my assumptions and understanding correct?
Thanks,
-John
On Jun 12, 2013, at 5:04 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi John,
>
> So, maybe I'm wrong about this, but what I was thinking is that we'd build
> two radio buttons into the Add Disk Offering dialog (we can ignore Compute
> Offerings for 4.2 since my feature doesn't yet support them).
>
> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage QoS
> for the purpose of this e-mail.
>
> By default, neither radio button is selected and no QoS fields are visible.
>
> If the user selects the Storage QoS radio button, then the Custom IOPS
> checkbox and the Min and Max text fields are made visible.
>
> If the user changes his mind and selects the Hypervisor QoS radio button,
> then the Custom IOPS checkbox and the Min and Max text fields disappear and
> are replaced by the two Hypervisor QoS text fields.
>
> This way, the user can choose neither QoS option or one of them or the
> other, but never both.
>
> On the API side, I was thinking of having logic in place when a request
> comes in to create a Disk Offering to confirm these fields are the way we
> want them.
>
> Once we know the Disk Offering is in order, a user can create a data disk
> from it. Since we checked the validity of the Disk Offering when it was
> created, the VM should never be asked to use Hypervisor QoS when Storage
> QoS in being utilized.
>
> Does that make sense or did I miss something?
>
> Thanks
>
>
> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Looking through the code, I am trying to understand how
>> CreateDiskOfferingCmd would have the context to identify the conflict.
>> Naively, it seems to me that this rule would need to be enforced when a
>> virtual machine is being deployed. Looking through the code, it seems like
>> we should add a private validateStorageQoS method to
>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
>> ResourceAllocationException when the QoS definitions are inconsistent. We
>> would then add calls to it from each of the VM creation methods in the
>> service. Do this type of approach sound reasonable?
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Hi John,
>>>
>>> So, here's what I was planning to do. Of course feel free to correct me
>> on
>>> this approach.
>>>
>>> I think it's OK if Wei merges his code into master and then I can draw
>> from
>>> the main repo and merge master into mine locally.
>>>
>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code to
>> make
>>> it user friendly (toggle between these features on the Add Disk Offering
>>> window).
>>>
>>> 2) I plan to write validation logic for the create-disk-offering API
>>> command which throws an exception if the rules are not followed (this
>>> should never be triggered from the GUI since the GUI will have controls
>> in
>>> place to toggle between the one feature and the other).
>>>
>>> I'm not sure about documentation. I haven't had much experience with it
>> on
>>> CloudStack projects yet.
>>>
>>> Thanks!
>>>
>>>
>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> Yes, these server-side rails need to be defined and implemented before
>>>> either patch can be merged. From my perspective, I would like to see
>> the
>>>> rule implemented in the hypervisor as part of the validation of the
>> virtual
>>>> machine definition. We also need to make sure that this mutual
>> exclusion
>>>> is documented. Do we usually include this type of documentation with
>>>> patches of this nature?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> Currently they are not yet implemented.
>>>>>
>>>>> We have to make sure they are implemented in the GUI from a usability
>>>>> standpoint, but the API must check for consistency and throw an
>> exception
>>>>> if necessary.
>>>>>
>>>>>
>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Are the checks only implemented in the UI?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>>>>> <mi...@solidfire.com> wrote:
>>>>>>
>>>>>>> Hi John,
>>>>>>>
>>>>>>> Wei and I have discussed making the two features mutually exclusive.
>> We
>>>>>>> agree with you that only one should be active at a time. We plan to
>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn his
>>>>>> feature
>>>>>>> on and mine off and vice versa.
>>>>>>>
>>>>>>> I was thinking if I wait until he checks in his code, then I update
>> and
>>>>>>> merge that I will be the person resolving merge conflicts in the
>>>>>> JavaScript
>>>>>>> code (there shouldn't be a problem in the Java code) as opposed to
>>>>>> putting
>>>>>>> that work on someone else.
>>>>>>>
>>>>>>> Let me know what you think.
>>>>>>>
>>>>>>> Oh, was going to ask you what "FS" stands for here.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> How have Wei and you resolved the issue of conflicting QoS
>> mechanisms
>>>>>>>> between the Hypervisor and Storage layers? Have the affected FSs
>> been
>>>>>>>> updated with that decision?
>>>>>>>>
>>>>>>>> In terms of merge timing, can you describe the dependencies between
>>>> the
>>>>>>>> patches?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> No problem, John.
>>>>>>>>>
>>>>>>>>> I still want to re-review it by myself before coming up with a new
>>>>>> patch
>>>>>>>>> file.
>>>>>>>>>
>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked in
>>>> and
>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jburwell@basho.com
>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> I just realized that I forgot to publish my review. I am offline
>>>> ATM,
>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>
>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>
>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>>>>>>>
>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
>>>> wanted
>>>>>>>>>>> addressed.
>>>>>>>>>>>
>>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
>> then
>>>>>>>> send
>>>>>>>>>> in
>>>>>>>>>>> another patch.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> ustcweizhou@gmail.com
>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>> code,
>>>>>>>>>> except
>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>>>>>>>> master
>>>>>>>>>>>>> this
>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>> branch,
>>>>>> but
>>>>>>>> -
>>>>>>>>>> as
>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>> SolidFire
>>>>>>>> branch
>>>>>>>>>>>>> into
>>>>>>>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>>>>>> strategy.
>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>>>>> overlapping
>>>>>>>>>>>>>>> elements. I understand the dependencies between the patches
>> to
>>>>>> be
>>>>>>>> as
>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Am I correct that the device management aspects of SolidFire
>>>> are
>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>> to the object_store branch or there are circular dependency
>>>>>> between
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>>>>>> determine
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>>>> branch
>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>> merge
>>>>>> and
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>>>>>>>> storage
>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>>>> which
>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although
>> it
>>>>>> does
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>>>> use))
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>>>> field
>>>>>>>> (if
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of
>> the
>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
>> for
>>>>>> the
>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>>>> data
>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>> attach
>>>>>> the
>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>>>>>>>> volume,
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>> structure
>>>>>> is
>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>>>>>> unless
>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
>>>> some
>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>>>>>> implementation
>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>>>>>> think
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>>>>>> response.
>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>>>>>>>> clients
>>>>>>>>>>>>> to
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>>>>>>>> Offering
>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able
>>>> to
>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>>>>>> Maybe
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
>> the
>>>>>> GUI
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
So, maybe I'm wrong about this, but what I was thinking is that we'd build
two radio buttons into the Add Disk Offering dialog (we can ignore Compute
Offerings for 4.2 since my feature doesn't yet support them).
Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage QoS
for the purpose of this e-mail.
By default, neither radio button is selected and no QoS fields are visible.
If the user selects the Storage QoS radio button, then the Custom IOPS
checkbox and the Min and Max text fields are made visible.
If the user changes his mind and selects the Hypervisor QoS radio button,
then the Custom IOPS checkbox and the Min and Max text fields disappear and
are replaced by the two Hypervisor QoS text fields.
This way, the user can choose neither QoS option or one of them or the
other, but never both.
On the API side, I was thinking of having logic in place when a request
comes in to create a Disk Offering to confirm these fields are the way we
want them.
Once we know the Disk Offering is in order, a user can create a data disk
from it. Since we checked the validity of the Disk Offering when it was
created, the VM should never be asked to use Hypervisor QoS when Storage
QoS in being utilized.
Does that make sense or did I miss something?
Thanks
On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Looking through the code, I am trying to understand how
> CreateDiskOfferingCmd would have the context to identify the conflict.
> Naively, it seems to me that this rule would need to be enforced when a
> virtual machine is being deployed. Looking through the code, it seems like
> we should add a private validateStorageQoS method to
> com.cloud.vm.UserVmManagerImpl to check this condition and throws a
> ResourceAllocationException when the QoS definitions are inconsistent. We
> would then add calls to it from each of the VM creation methods in the
> service. Do this type of approach sound reasonable?
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hi John,
> >
> > So, here's what I was planning to do. Of course feel free to correct me
> on
> > this approach.
> >
> > I think it's OK if Wei merges his code into master and then I can draw
> from
> > the main repo and merge master into mine locally.
> >
> > 1) Once I get Wei's code and merge, I plan to add a little GUI code to
> make
> > it user friendly (toggle between these features on the Add Disk Offering
> > window).
> >
> > 2) I plan to write validation logic for the create-disk-offering API
> > command which throws an exception if the rules are not followed (this
> > should never be triggered from the GUI since the GUI will have controls
> in
> > place to toggle between the one feature and the other).
> >
> > I'm not sure about documentation. I haven't had much experience with it
> on
> > CloudStack projects yet.
> >
> > Thanks!
> >
> >
> > On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Yes, these server-side rails need to be defined and implemented before
> >> either patch can be merged. From my perspective, I would like to see
> the
> >> rule implemented in the hypervisor as part of the validation of the
> virtual
> >> machine definition. We also need to make sure that this mutual
> exclusion
> >> is documented. Do we usually include this type of documentation with
> >> patches of this nature?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Currently they are not yet implemented.
> >>>
> >>> We have to make sure they are implemented in the GUI from a usability
> >>> standpoint, but the API must check for consistency and throw an
> exception
> >>> if necessary.
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Are the checks only implemented in the UI?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> >>>> <mi...@solidfire.com> wrote:
> >>>>
> >>>>> Hi John,
> >>>>>
> >>>>> Wei and I have discussed making the two features mutually exclusive.
> We
> >>>>> agree with you that only one should be active at a time. We plan to
> >>>>> implement in the GUI a mechanism (maybe radio buttons) to turn his
> >>>> feature
> >>>>> on and mine off and vice versa.
> >>>>>
> >>>>> I was thinking if I wait until he checks in his code, then I update
> and
> >>>>> merge that I will be the person resolving merge conflicts in the
> >>>> JavaScript
> >>>>> code (there shouldn't be a problem in the Java code) as opposed to
> >>>> putting
> >>>>> that work on someone else.
> >>>>>
> >>>>> Let me know what you think.
> >>>>>
> >>>>> Oh, was going to ask you what "FS" stands for here.
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> How have Wei and you resolved the issue of conflicting QoS
> mechanisms
> >>>>>> between the Hypervisor and Storage layers? Have the affected FSs
> been
> >>>>>> updated with that decision?
> >>>>>>
> >>>>>> In terms of merge timing, can you describe the dependencies between
> >> the
> >>>>>> patches?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> >>>>>> <mi...@solidfire.com> wrote:
> >>>>>>
> >>>>>>> No problem, John.
> >>>>>>>
> >>>>>>> I still want to re-review it by myself before coming up with a new
> >>>> patch
> >>>>>>> file.
> >>>>>>>
> >>>>>>> Also, maybe I should first wait for Wei's changes to be checked in
> >> and
> >>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jburwell@basho.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> I just realized that I forgot to publish my review. I am offline
> >> ATM,
> >>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>
> >>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>
> >>>>>>>> Sorry for the oversight,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>>>>>>>
> >>>>>>>>> Just an FYI that I believe I have implemented all the areas we
> >> wanted
> >>>>>>>>> addressed.
> >>>>>>>>>
> >>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
> then
> >>>>>> send
> >>>>>>>> in
> >>>>>>>>> another patch.
> >>>>>>>>>
> >>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> ustcweizhou@gmail.com
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> It looks the two feature do not have many conflicts in Java
> code,
> >>>>>>>> except
> >>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
> >>>>>> master
> >>>>>>>>>>> this
> >>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>
> >>>>>>>>>>> -Wei
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>
> >>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>
> >>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> branch,
> >>>> but
> >>>>>> -
> >>>>>>>> as
> >>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> SolidFire
> >>>>>> branch
> >>>>>>>>>>> into
> >>>>>>>>>>>> the object_store branch before object_store goes into master.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
> >>>>>> strategy.
> >>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>> jburwell@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>> disk_io_throttling,
> >>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> >>>> overlapping
> >>>>>>>>>>>>> elements. I understand the dependencies between the patches
> to
> >>>> be
> >>>>>> as
> >>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Am I correct that the device management aspects of SolidFire
> >> are
> >>>>>>>>>>> additive
> >>>>>>>>>>>>> to the object_store branch or there are circular dependency
> >>>> between
> >>>>>>>>>>> the
> >>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
> >>>>>> determine
> >>>>>>>>>>> the
> >>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
> >> branch
> >>>>>>>>>>> before
> >>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
> merge
> >>>> and
> >>>>>>>>>>> we
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
> >>>>>> storage
> >>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
> >> which
> >>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although
> it
> >>>> does
> >>>>>>>>>>> not
> >>>>>>>>>>>>> know
> >>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
> >> use))
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
> >> field
> >>>>>> (if
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>>>>>>>> question, we
> >>>>>>>>>>>>>>> pass the managed property along (this takes the place of
> the
> >>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
> for
> >>>> the
> >>>>>>>>>>>> managed
> >>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
> >> data
> >>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>> created and the rest of the attach command executes to
> attach
> >>>> the
> >>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> >>>>>> volume,
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> structure
> >>>> is
> >>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
> >>>> unless
> >>>>>>>>>>> it is
> >>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
> >> some
> >>>>>>>>>>> idea,
> >>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>>>> implementation
> >>>>>>>>>>>>>>>> attributes. I always wondered about the details field. I
> >>>> think
> >>>>>>>>>>> we
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> beef up the description in the documentation regarding the
> >>>>>>>>>>> expected
> >>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>>>>>>>> returned on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> >>>>>> response.
> >>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
> >>>>>> clients
> >>>>>>>>>>> to
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> >>>>>> Offering
> >>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able
> >> to
> >>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
> >>>> Maybe
> >>>>>>>>>>> it
> >>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
> the
> >>>> GUI
> >>>>>>>>>>> if
> >>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Looking through the code, I am trying to understand how CreateDiskOfferingCmd would have the context to identify the conflict. Naively, it seems to me that this rule would need to be enforced when a virtual machine is being deployed. Looking through the code, it seems like we should add a private validateStorageQoS method to com.cloud.vm.UserVmManagerImpl to check this condition and throws a ResourceAllocationException when the QoS definitions are inconsistent. We would then add calls to it from each of the VM creation methods in the service. Do this type of approach sound reasonable?
Thanks,
-John
On Jun 12, 2013, at 4:30 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi John,
>
> So, here's what I was planning to do. Of course feel free to correct me on
> this approach.
>
> I think it's OK if Wei merges his code into master and then I can draw from
> the main repo and merge master into mine locally.
>
> 1) Once I get Wei's code and merge, I plan to add a little GUI code to make
> it user friendly (toggle between these features on the Add Disk Offering
> window).
>
> 2) I plan to write validation logic for the create-disk-offering API
> command which throws an exception if the rules are not followed (this
> should never be triggered from the GUI since the GUI will have controls in
> place to toggle between the one feature and the other).
>
> I'm not sure about documentation. I haven't had much experience with it on
> CloudStack projects yet.
>
> Thanks!
>
>
> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Yes, these server-side rails need to be defined and implemented before
>> either patch can be merged. From my perspective, I would like to see the
>> rule implemented in the hypervisor as part of the validation of the virtual
>> machine definition. We also need to make sure that this mutual exclusion
>> is documented. Do we usually include this type of documentation with
>> patches of this nature?
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Currently they are not yet implemented.
>>>
>>> We have to make sure they are implemented in the GUI from a usability
>>> standpoint, but the API must check for consistency and throw an exception
>>> if necessary.
>>>
>>>
>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> Are the checks only implemented in the UI?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>>
>>>>
>>>>
>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>>>> <mi...@solidfire.com> wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> Wei and I have discussed making the two features mutually exclusive. We
>>>>> agree with you that only one should be active at a time. We plan to
>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn his
>>>> feature
>>>>> on and mine off and vice versa.
>>>>>
>>>>> I was thinking if I wait until he checks in his code, then I update and
>>>>> merge that I will be the person resolving merge conflicts in the
>>>> JavaScript
>>>>> code (there shouldn't be a problem in the Java code) as opposed to
>>>> putting
>>>>> that work on someone else.
>>>>>
>>>>> Let me know what you think.
>>>>>
>>>>> Oh, was going to ask you what "FS" stands for here.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> How have Wei and you resolved the issue of conflicting QoS mechanisms
>>>>>> between the Hypervisor and Storage layers? Have the affected FSs been
>>>>>> updated with that decision?
>>>>>>
>>>>>> In terms of merge timing, can you describe the dependencies between
>> the
>>>>>> patches?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>>>> <mi...@solidfire.com> wrote:
>>>>>>
>>>>>>> No problem, John.
>>>>>>>
>>>>>>> I still want to re-review it by myself before coming up with a new
>>>> patch
>>>>>>> file.
>>>>>>>
>>>>>>> Also, maybe I should first wait for Wei's changes to be checked in
>> and
>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> I just realized that I forgot to publish my review. I am offline
>> ATM,
>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>
>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>
>>>>>>>> Sorry for the oversight,
>>>>>>>> -John
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>>>>>
>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
>> wanted
>>>>>>>>> addressed.
>>>>>>>>>
>>>>>>>>> I plan to review the code again tomorrow morning or afternoon, then
>>>>>> send
>>>>>>>> in
>>>>>>>>> another patch.
>>>>>>>>>
>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <ustcweizhou@gmail.com
>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>
>>>>>>>>>>> It looks the two feature do not have many conflicts in Java code,
>>>>>>>> except
>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>>>>>> master
>>>>>>>>>>> this
>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>
>>>>>>>>>>> -Wei
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>
>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>
>>>>>>>>>>>> The SolidFire patch does not depend on the object_store branch,
>>>> but
>>>>>> -
>>>>>>>> as
>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
>>>>>> branch
>>>>>>>>>>> into
>>>>>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>>>> strategy.
>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>> jburwell@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>> disk_io_throttling,
>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>>> overlapping
>>>>>>>>>>>>> elements. I understand the dependencies between the patches to
>>>> be
>>>>>> as
>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>
>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>
>>>>>>>>>>>>> Am I correct that the device management aspects of SolidFire
>> are
>>>>>>>>>>> additive
>>>>>>>>>>>>> to the object_store branch or there are circular dependency
>>>> between
>>>>>>>>>>> the
>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>>>> determine
>>>>>>>>>>> the
>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>> branch
>>>>>>>>>>> before
>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge
>>>> and
>>>>>>>>>>> we
>>>>>>>>>>>> can
>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>>>>>> storage
>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>> which
>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although it
>>>> does
>>>>>>>>>>> not
>>>>>>>>>>>>> know
>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>> use))
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>> field
>>>>>> (if
>>>>>>>>>>>> not
>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>>>> question, we
>>>>>>>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
>>>> the
>>>>>>>>>>>> managed
>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>> data
>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>> created and the rest of the attach command executes to attach
>>>> the
>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>>>>>> volume,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data structure
>>>> is
>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>>>> unless
>>>>>>>>>>> it is
>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
>> some
>>>>>>>>>>> idea,
>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>>>> implementation
>>>>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>>>> think
>>>>>>>>>>> we
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>>>>>> expected
>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>>>>>> returned on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>>>> response.
>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>>>>>> clients
>>>>>>>>>>> to
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>>>>>> Offering
>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able
>> to
>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>>>> Maybe
>>>>>>>>>>> it
>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the
>>>> GUI
>>>>>>>>>>> if
>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
So, here's what I was planning to do. Of course feel free to correct me on
this approach.
I think it's OK if Wei merges his code into master and then I can draw from
the main repo and merge master into mine locally.
1) Once I get Wei's code and merge, I plan to add a little GUI code to make
it user friendly (toggle between these features on the Add Disk Offering
window).
2) I plan to write validation logic for the create-disk-offering API
command which throws an exception if the rules are not followed (this
should never be triggered from the GUI since the GUI will have controls in
place to toggle between the one feature and the other).
I'm not sure about documentation. I haven't had much experience with it on
CloudStack projects yet.
Thanks!
On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Yes, these server-side rails need to be defined and implemented before
> either patch can be merged. From my perspective, I would like to see the
> rule implemented in the hypervisor as part of the validation of the virtual
> machine definition. We also need to make sure that this mutual exclusion
> is documented. Do we usually include this type of documentation with
> patches of this nature?
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Currently they are not yet implemented.
> >
> > We have to make sure they are implemented in the GUI from a usability
> > standpoint, but the API must check for consistency and throw an exception
> > if necessary.
> >
> >
> > On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Are the checks only implemented in the UI?
> >>
> >> Thanks,
> >> -John
> >>
> >>
> >>
> >>
> >> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> >> <mi...@solidfire.com> wrote:
> >>
> >>> Hi John,
> >>>
> >>> Wei and I have discussed making the two features mutually exclusive. We
> >>> agree with you that only one should be active at a time. We plan to
> >>> implement in the GUI a mechanism (maybe radio buttons) to turn his
> >> feature
> >>> on and mine off and vice versa.
> >>>
> >>> I was thinking if I wait until he checks in his code, then I update and
> >>> merge that I will be the person resolving merge conflicts in the
> >> JavaScript
> >>> code (there shouldn't be a problem in the Java code) as opposed to
> >> putting
> >>> that work on someone else.
> >>>
> >>> Let me know what you think.
> >>>
> >>> Oh, was going to ask you what "FS" stands for here.
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> How have Wei and you resolved the issue of conflicting QoS mechanisms
> >>>> between the Hypervisor and Storage layers? Have the affected FSs been
> >>>> updated with that decision?
> >>>>
> >>>> In terms of merge timing, can you describe the dependencies between
> the
> >>>> patches?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> >>>> <mi...@solidfire.com> wrote:
> >>>>
> >>>>> No problem, John.
> >>>>>
> >>>>> I still want to re-review it by myself before coming up with a new
> >> patch
> >>>>> file.
> >>>>>
> >>>>> Also, maybe I should first wait for Wei's changes to be checked in
> and
> >>>>> merge those into mine before generating a new patch file?
> >>>>>
> >>>>>
> >>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> I just realized that I forgot to publish my review. I am offline
> ATM,
> >>>>>> but I will publish it in the next couple of hours.
> >>>>>>
> >>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>
> >>>>>> Sorry for the oversight,
> >>>>>> -John
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>> <mi...@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>>>>>
> >>>>>>> Just an FYI that I believe I have implemented all the areas we
> wanted
> >>>>>>> addressed.
> >>>>>>>
> >>>>>>> I plan to review the code again tomorrow morning or afternoon, then
> >>>> send
> >>>>>> in
> >>>>>>> another patch.
> >>>>>>>
> >>>>>>> Thanks for all the work on this everyone!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> Sure, that sounds good.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <ustcweizhou@gmail.com
> >
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Mike,
> >>>>>>>>>
> >>>>>>>>> It looks the two feature do not have many conflicts in Java code,
> >>>>>> except
> >>>>>>>>> the cloudstack UI.
> >>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
> >>>> master
> >>>>>>>>> this
> >>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>
> >>>>>>>>> -Wei
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>
> >>>>>>>>>> Hey John,
> >>>>>>>>>>
> >>>>>>>>>> The SolidFire patch does not depend on the object_store branch,
> >> but
> >>>> -
> >>>>>> as
> >>>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
> >>>> branch
> >>>>>>>>> into
> >>>>>>>>>> the object_store branch before object_store goes into master.
> >>>>>>>>>>
> >>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
> >>>> strategy.
> >>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >> jburwell@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> We have a delicate merge dance to perform. The
> >> disk_io_throttling,
> >>>>>>>>>>> solidfire, and object_store appear to have a number of
> >> overlapping
> >>>>>>>>>>> elements. I understand the dependencies between the patches to
> >> be
> >>>> as
> >>>>>>>>>>> follows:
> >>>>>>>>>>>
> >>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>
> >>>>>>>>>>> Am I correct that the device management aspects of SolidFire
> are
> >>>>>>>>> additive
> >>>>>>>>>>> to the object_store branch or there are circular dependency
> >> between
> >>>>>>>>> the
> >>>>>>>>>>> branches? Once we understand the dependency graph, we can
> >>>> determine
> >>>>>>>>> the
> >>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Also, if we are good with Edison merging my code into his
> branch
> >>>>>>>>> before
> >>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>
> >>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge
> >> and
> >>>>>>>>> we
> >>>>>>>>>> can
> >>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
> >>>> storage
> >>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
> which
> >>>>>>>>>>> currently
> >>>>>>>>>>>>> sends several commands to the hypervisor in use (although it
> >> does
> >>>>>>>>> not
> >>>>>>>>>>> know
> >>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
> use))
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
> field
> >>>> (if
> >>>>>>>>>> not
> >>>>>>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>>>>>> question, we
> >>>>>>>>>>>>> pass the managed property along (this takes the place of the
> >>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
> >> the
> >>>>>>>>>> managed
> >>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
> data
> >>>>>>>>>>> structure is
> >>>>>>>>>>>>> created and the rest of the attach command executes to attach
> >> the
> >>>>>>>>>>> volume.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> >>>> volume,
> >>>>>>>>>> the
> >>>>>>>>>>>>> same check is made. If managed, the hypervisor data structure
> >> is
> >>>>>>>>>>> removed.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
> >> unless
> >>>>>>>>> it is
> >>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
> some
> >>>>>>>>> idea,
> >>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>> implementation
> >>>>>>>>>>>>>> attributes. I always wondered about the details field. I
> >> think
> >>>>>>>>> we
> >>>>>>>>>>> should
> >>>>>>>>>>>>>> beef up the description in the documentation regarding the
> >>>>>>>>> expected
> >>>>>>>>>>> format
> >>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>>>>>> returned on
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> >>>> response.
> >>>>>>>>>> Why
> >>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
> >>>> clients
> >>>>>>>>> to
> >>>>>>>>>> be
> >>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> >>>> Offering
> >>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able
> to
> >>>>>>>>>> associate
> >>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
> >> Maybe
> >>>>>>>>> it
> >>>>>>>>>>> could
> >>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the
> >> GUI
> >>>>>>>>> if
> >>>>>>>>>>> it's
> >>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Yes, these server-side rails need to be defined and implemented before either patch can be merged. From my perspective, I would like to see the rule implemented in the hypervisor as part of the validation of the virtual machine definition. We also need to make sure that this mutual exclusion is documented. Do we usually include this type of documentation with patches of this nature?
Thanks,
-John
On Jun 12, 2013, at 2:18 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Currently they are not yet implemented.
>
> We have to make sure they are implemented in the GUI from a usability
> standpoint, but the API must check for consistency and throw an exception
> if necessary.
>
>
> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Are the checks only implemented in the UI?
>>
>> Thanks,
>> -John
>>
>>
>>
>>
>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
>> <mi...@solidfire.com> wrote:
>>
>>> Hi John,
>>>
>>> Wei and I have discussed making the two features mutually exclusive. We
>>> agree with you that only one should be active at a time. We plan to
>>> implement in the GUI a mechanism (maybe radio buttons) to turn his
>> feature
>>> on and mine off and vice versa.
>>>
>>> I was thinking if I wait until he checks in his code, then I update and
>>> merge that I will be the person resolving merge conflicts in the
>> JavaScript
>>> code (there shouldn't be a problem in the Java code) as opposed to
>> putting
>>> that work on someone else.
>>>
>>> Let me know what you think.
>>>
>>> Oh, was going to ask you what "FS" stands for here.
>>>
>>> Thanks!
>>>
>>>
>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> How have Wei and you resolved the issue of conflicting QoS mechanisms
>>>> between the Hypervisor and Storage layers? Have the affected FSs been
>>>> updated with that decision?
>>>>
>>>> In terms of merge timing, can you describe the dependencies between the
>>>> patches?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>>
>>>>
>>>>
>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>>>> <mi...@solidfire.com> wrote:
>>>>
>>>>> No problem, John.
>>>>>
>>>>> I still want to re-review it by myself before coming up with a new
>> patch
>>>>> file.
>>>>>
>>>>> Also, maybe I should first wait for Wei's changes to be checked in and
>>>>> merge those into mine before generating a new patch file?
>>>>>
>>>>>
>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> I just realized that I forgot to publish my review. I am offline ATM,
>>>>>> but I will publish it in the next couple of hours.
>>>>>>
>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>
>>>>>> Sorry for the oversight,
>>>>>> -John
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>> <mi...@solidfire.com> wrote:
>>>>>>
>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>>>
>>>>>>> Just an FYI that I believe I have implemented all the areas we wanted
>>>>>>> addressed.
>>>>>>>
>>>>>>> I plan to review the code again tomorrow morning or afternoon, then
>>>> send
>>>>>> in
>>>>>>> another patch.
>>>>>>>
>>>>>>> Thanks for all the work on this everyone!
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> Sure, that sounds good.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Mike,
>>>>>>>>>
>>>>>>>>> It looks the two feature do not have many conflicts in Java code,
>>>>>> except
>>>>>>>>> the cloudstack UI.
>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>>>> master
>>>>>>>>> this
>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>
>>>>>>>>> -Wei
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>
>>>>>>>>>> Hey John,
>>>>>>>>>>
>>>>>>>>>> The SolidFire patch does not depend on the object_store branch,
>> but
>>>> -
>>>>>> as
>>>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
>>>> branch
>>>>>>>>> into
>>>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>>>
>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>> strategy.
>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> jburwell@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> We have a delicate merge dance to perform. The
>> disk_io_throttling,
>>>>>>>>>>> solidfire, and object_store appear to have a number of
>> overlapping
>>>>>>>>>>> elements. I understand the dependencies between the patches to
>> be
>>>> as
>>>>>>>>>>> follows:
>>>>>>>>>>>
>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>
>>>>>>>>>>> Am I correct that the device management aspects of SolidFire are
>>>>>>>>> additive
>>>>>>>>>>> to the object_store branch or there are circular dependency
>> between
>>>>>>>>> the
>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>> determine
>>>>>>>>> the
>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Also, if we are good with Edison merging my code into his branch
>>>>>>>>> before
>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>
>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge
>> and
>>>>>>>>> we
>>>>>>>>>> can
>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>>>> storage
>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
>>>>>>>>>>> currently
>>>>>>>>>>>>> sends several commands to the hypervisor in use (although it
>> does
>>>>>>>>> not
>>>>>>>>>>> know
>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url field
>>>> (if
>>>>>>>>>> not
>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>> question, we
>>>>>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>
>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
>> the
>>>>>>>>>> managed
>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor data
>>>>>>>>>>> structure is
>>>>>>>>>>>>> created and the rest of the attach command executes to attach
>> the
>>>>>>>>>>> volume.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>>>> volume,
>>>>>>>>>> the
>>>>>>>>>>>>> same check is made. If managed, the hypervisor data structure
>> is
>>>>>>>>>>> removed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>> unless
>>>>>>>>> it is
>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
>>>>>>>>> idea,
>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>> implementation
>>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>> think
>>>>>>>>> we
>>>>>>>>>>> should
>>>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>>>> expected
>>>>>>>>>>> format
>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>>>> returned on
>>>>>>>>>>> the
>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>> response.
>>>>>>>>>> Why
>>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>>>> clients
>>>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>>>> Offering
>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
>>>>>>>>>> associate
>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>> Maybe
>>>>>>>>> it
>>>>>>>>>>> could
>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the
>> GUI
>>>>>>>>> if
>>>>>>>>>>> it's
>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Currently they are not yet implemented.
We have to make sure they are implemented in the GUI from a usability
standpoint, but the API must check for consistency and throw an exception
if necessary.
On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Are the checks only implemented in the UI?
>
> Thanks,
> -John
>
>
>
>
> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
> <mi...@solidfire.com> wrote:
>
> > Hi John,
> >
> > Wei and I have discussed making the two features mutually exclusive. We
> > agree with you that only one should be active at a time. We plan to
> > implement in the GUI a mechanism (maybe radio buttons) to turn his
> feature
> > on and mine off and vice versa.
> >
> > I was thinking if I wait until he checks in his code, then I update and
> > merge that I will be the person resolving merge conflicts in the
> JavaScript
> > code (there shouldn't be a problem in the Java code) as opposed to
> putting
> > that work on someone else.
> >
> > Let me know what you think.
> >
> > Oh, was going to ask you what "FS" stands for here.
> >
> > Thanks!
> >
> >
> > On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> How have Wei and you resolved the issue of conflicting QoS mechanisms
> >> between the Hypervisor and Storage layers? Have the affected FSs been
> >> updated with that decision?
> >>
> >> In terms of merge timing, can you describe the dependencies between the
> >> patches?
> >>
> >> Thanks,
> >> -John
> >>
> >>
> >>
> >>
> >> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> >> <mi...@solidfire.com> wrote:
> >>
> >>> No problem, John.
> >>>
> >>> I still want to re-review it by myself before coming up with a new
> patch
> >>> file.
> >>>
> >>> Also, maybe I should first wait for Wei's changes to be checked in and
> >>> merge those into mine before generating a new patch file?
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> I just realized that I forgot to publish my review. I am offline ATM,
> >>>> but I will publish it in the next couple of hours.
> >>>>
> >>>> Do you plan to update your the patch in Review Board?
> >>>>
> >>>> Sorry for the oversight,
> >>>> -John
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>> <mi...@solidfire.com> wrote:
> >>>>
> >>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>>>
> >>>>> Just an FYI that I believe I have implemented all the areas we wanted
> >>>>> addressed.
> >>>>>
> >>>>> I plan to review the code again tomorrow morning or afternoon, then
> >> send
> >>>> in
> >>>>> another patch.
> >>>>>
> >>>>> Thanks for all the work on this everyone!
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>>> Sure, that sounds good.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Hi Mike,
> >>>>>>>
> >>>>>>> It looks the two feature do not have many conflicts in Java code,
> >>>> except
> >>>>>>> the cloudstack UI.
> >>>>>>> If you do not mind, I will merge disk_io_throttling branch into
> >> master
> >>>>>>> this
> >>>>>>> week, so that you can develop based on it.
> >>>>>>>
> >>>>>>> -Wei
> >>>>>>>
> >>>>>>>
> >>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>
> >>>>>>>> Hey John,
> >>>>>>>>
> >>>>>>>> The SolidFire patch does not depend on the object_store branch,
> but
> >> -
> >>>> as
> >>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
> >> branch
> >>>>>>> into
> >>>>>>>> the object_store branch before object_store goes into master.
> >>>>>>>>
> >>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
> >> strategy.
> >>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> jburwell@basho.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> We have a delicate merge dance to perform. The
> disk_io_throttling,
> >>>>>>>>> solidfire, and object_store appear to have a number of
> overlapping
> >>>>>>>>> elements. I understand the dependencies between the patches to
> be
> >> as
> >>>>>>>>> follows:
> >>>>>>>>>
> >>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>
> >>>>>>>>> Am I correct that the device management aspects of SolidFire are
> >>>>>>> additive
> >>>>>>>>> to the object_store branch or there are circular dependency
> between
> >>>>>>> the
> >>>>>>>>> branches? Once we understand the dependency graph, we can
> >> determine
> >>>>>>> the
> >>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Also, if we are good with Edison merging my code into his branch
> >>>>>>> before
> >>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>
> >>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge
> and
> >>>>>>> we
> >>>>>>>> can
> >>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>
> >>>>>>>>>>> 1) There should be NO references to hypervisor code in the
> >> storage
> >>>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
> >>>>>>>>> currently
> >>>>>>>>>>> sends several commands to the hypervisor in use (although it
> does
> >>>>>>> not
> >>>>>>>>> know
> >>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>>>>>>>>>>
> >>>>>>>>>>> 2) managed=true or managed=false can be placed in the url field
> >> (if
> >>>>>>>> not
> >>>>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>
> >>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>>>> question, we
> >>>>>>>>>>> pass the managed property along (this takes the place of the
> >>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>
> >>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
> the
> >>>>>>>> managed
> >>>>>>>>>>> property. If true for an attach, the necessary hypervisor data
> >>>>>>>>> structure is
> >>>>>>>>>>> created and the rest of the attach command executes to attach
> the
> >>>>>>>>> volume.
> >>>>>>>>>>>
> >>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> >> volume,
> >>>>>>>> the
> >>>>>>>>>>> same check is made. If managed, the hypervisor data structure
> is
> >>>>>>>>> removed.
> >>>>>>>>>>>
> >>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
> unless
> >>>>>>> it is
> >>>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
> >>>>>>> idea,
> >>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >> implementation
> >>>>>>>>>>>> attributes. I always wondered about the details field. I
> think
> >>>>>>> we
> >>>>>>>>> should
> >>>>>>>>>>>> beef up the description in the documentation regarding the
> >>>>>>> expected
> >>>>>>>>> format
> >>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>>>> returned on
> >>>>>>>>> the
> >>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> >> response.
> >>>>>>>> Why
> >>>>>>>>>>>> don't we return it? It seems like it would be useful for
> >> clients
> >>>>>>> to
> >>>>>>>> be
> >>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>
> >>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> >> Offering
> >>>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
> >>>>>>>> associate
> >>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
> Maybe
> >>>>>>> it
> >>>>>>>>> could
> >>>>>>>>>>>> go there?
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the
> GUI
> >>>>>>> if
> >>>>>>>>> it's
> >>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Are the checks only implemented in the UI?
Thanks,
-John
On Jun 12, 2013, at 1:02 PM, Mike Tutkowski
<mi...@solidfire.com> wrote:
> Hi John,
>
> Wei and I have discussed making the two features mutually exclusive. We
> agree with you that only one should be active at a time. We plan to
> implement in the GUI a mechanism (maybe radio buttons) to turn his feature
> on and mine off and vice versa.
>
> I was thinking if I wait until he checks in his code, then I update and
> merge that I will be the person resolving merge conflicts in the JavaScript
> code (there shouldn't be a problem in the Java code) as opposed to putting
> that work on someone else.
>
> Let me know what you think.
>
> Oh, was going to ask you what "FS" stands for here.
>
> Thanks!
>
>
> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> How have Wei and you resolved the issue of conflicting QoS mechanisms
>> between the Hypervisor and Storage layers? Have the affected FSs been
>> updated with that decision?
>>
>> In terms of merge timing, can you describe the dependencies between the
>> patches?
>>
>> Thanks,
>> -John
>>
>>
>>
>>
>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
>> <mi...@solidfire.com> wrote:
>>
>>> No problem, John.
>>>
>>> I still want to re-review it by myself before coming up with a new patch
>>> file.
>>>
>>> Also, maybe I should first wait for Wei's changes to be checked in and
>>> merge those into mine before generating a new patch file?
>>>
>>>
>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> I just realized that I forgot to publish my review. I am offline ATM,
>>>> but I will publish it in the next couple of hours.
>>>>
>>>> Do you plan to update your the patch in Review Board?
>>>>
>>>> Sorry for the oversight,
>>>> -John
>>>>
>>>>
>>>>
>>>>
>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>> <mi...@solidfire.com> wrote:
>>>>
>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>
>>>>> Just an FYI that I believe I have implemented all the areas we wanted
>>>>> addressed.
>>>>>
>>>>> I plan to review the code again tomorrow morning or afternoon, then
>> send
>>>> in
>>>>> another patch.
>>>>>
>>>>> Thanks for all the work on this everyone!
>>>>>
>>>>>
>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> Sure, that sounds good.
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>>> Hi Mike,
>>>>>>>
>>>>>>> It looks the two feature do not have many conflicts in Java code,
>>>> except
>>>>>>> the cloudstack UI.
>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>> master
>>>>>>> this
>>>>>>> week, so that you can develop based on it.
>>>>>>>
>>>>>>> -Wei
>>>>>>>
>>>>>>>
>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>
>>>>>>>> Hey John,
>>>>>>>>
>>>>>>>> The SolidFire patch does not depend on the object_store branch, but
>> -
>>>> as
>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
>> branch
>>>>>>> into
>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>
>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>> strategy.
>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
>>>>>>>>> solidfire, and object_store appear to have a number of overlapping
>>>>>>>>> elements. I understand the dependencies between the patches to be
>> as
>>>>>>>>> follows:
>>>>>>>>>
>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>
>>>>>>>>> Am I correct that the device management aspects of SolidFire are
>>>>>>> additive
>>>>>>>>> to the object_store branch or there are circular dependency between
>>>>>>> the
>>>>>>>>> branches? Once we understand the dependency graph, we can
>> determine
>>>>>>> the
>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Also, if we are good with Edison merging my code into his branch
>>>>>>> before
>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>
>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
>>>>>>> we
>>>>>>>> can
>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>
>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>> storage
>>>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
>>>>>>>>> currently
>>>>>>>>>>> sends several commands to the hypervisor in use (although it does
>>>>>>> not
>>>>>>>>> know
>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>>>>>>>>
>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url field
>> (if
>>>>>>>> not
>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>
>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>> question, we
>>>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>
>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
>>>>>>>> managed
>>>>>>>>>>> property. If true for an attach, the necessary hypervisor data
>>>>>>>>> structure is
>>>>>>>>>>> created and the rest of the attach command executes to attach the
>>>>>>>>> volume.
>>>>>>>>>>>
>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>> volume,
>>>>>>>> the
>>>>>>>>>>> same check is made. If managed, the hypervisor data structure is
>>>>>>>>> removed.
>>>>>>>>>>>
>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
>>>>>>> it is
>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
>>>>>>> idea,
>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>> implementation
>>>>>>>>>>>> attributes. I always wondered about the details field. I think
>>>>>>> we
>>>>>>>>> should
>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>> expected
>>>>>>>>> format
>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>> returned on
>>>>>>>>> the
>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>> response.
>>>>>>>> Why
>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>> clients
>>>>>>> to
>>>>>>>> be
>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>
>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>
>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>> Offering
>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
>>>>>>>> associate
>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
>>>>>>> it
>>>>>>>>> could
>>>>>>>>>>>> go there?
>>>>>>>>>>>>
>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
>>>>>>> if
>>>>>>>>> it's
>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
Wei and I have discussed making the two features mutually exclusive. We
agree with you that only one should be active at a time. We plan to
implement in the GUI a mechanism (maybe radio buttons) to turn his feature
on and mine off and vice versa.
I was thinking if I wait until he checks in his code, then I update and
merge that I will be the person resolving merge conflicts in the JavaScript
code (there shouldn't be a problem in the Java code) as opposed to putting
that work on someone else.
Let me know what you think.
Oh, was going to ask you what "FS" stands for here.
Thanks!
On Wed, Jun 12, 2013 at 10:56 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> How have Wei and you resolved the issue of conflicting QoS mechanisms
> between the Hypervisor and Storage layers? Have the affected FSs been
> updated with that decision?
>
> In terms of merge timing, can you describe the dependencies between the
> patches?
>
> Thanks,
> -John
>
>
>
>
> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
> <mi...@solidfire.com> wrote:
>
> > No problem, John.
> >
> > I still want to re-review it by myself before coming up with a new patch
> > file.
> >
> > Also, maybe I should first wait for Wei's changes to be checked in and
> > merge those into mine before generating a new patch file?
> >
> >
> > On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> I just realized that I forgot to publish my review. I am offline ATM,
> >> but I will publish it in the next couple of hours.
> >>
> >> Do you plan to update your the patch in Review Board?
> >>
> >> Sorry for the oversight,
> >> -John
> >>
> >>
> >>
> >>
> >> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >> <mi...@solidfire.com> wrote:
> >>
> >>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>
> >>> Just an FYI that I believe I have implemented all the areas we wanted
> >>> addressed.
> >>>
> >>> I plan to review the code again tomorrow morning or afternoon, then
> send
> >> in
> >>> another patch.
> >>>
> >>> Thanks for all the work on this everyone!
> >>>
> >>>
> >>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> Sure, that sounds good.
> >>>>
> >>>>
> >>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
> >> wrote:
> >>>>
> >>>>> Hi Mike,
> >>>>>
> >>>>> It looks the two feature do not have many conflicts in Java code,
> >> except
> >>>>> the cloudstack UI.
> >>>>> If you do not mind, I will merge disk_io_throttling branch into
> master
> >>>>> this
> >>>>> week, so that you can develop based on it.
> >>>>>
> >>>>> -Wei
> >>>>>
> >>>>>
> >>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>
> >>>>>> Hey John,
> >>>>>>
> >>>>>> The SolidFire patch does not depend on the object_store branch, but
> -
> >> as
> >>>>>> Edison mentioned - it might be easier if we merge the SolidFire
> branch
> >>>>> into
> >>>>>> the object_store branch before object_store goes into master.
> >>>>>>
> >>>>>> I'm not sure how the disk_io_throttling fits into this merge
> strategy.
> >>>>>> Perhaps Wei can chime in on that.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
> >>>>>>> solidfire, and object_store appear to have a number of overlapping
> >>>>>>> elements. I understand the dependencies between the patches to be
> as
> >>>>>>> follows:
> >>>>>>>
> >>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>
> >>>>>>> Am I correct that the device management aspects of SolidFire are
> >>>>> additive
> >>>>>>> to the object_store branch or there are circular dependency between
> >>>>> the
> >>>>>>> branches? Once we understand the dependency graph, we can
> determine
> >>>>> the
> >>>>>>> best approach to land the changes in master.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>>
> >>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Also, if we are good with Edison merging my code into his branch
> >>>>> before
> >>>>>>>> going into master, I am good with that.
> >>>>>>>>
> >>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
> >>>>> we
> >>>>>> can
> >>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>
> >>>>>>>>> 1) There should be NO references to hypervisor code in the
> storage
> >>>>>>>>> plug-ins code (this includes the default storage plug-in, which
> >>>>>>> currently
> >>>>>>>>> sends several commands to the hypervisor in use (although it does
> >>>>> not
> >>>>>>> know
> >>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>>>>>>>>
> >>>>>>>>> 2) managed=true or managed=false can be placed in the url field
> (if
> >>>>>> not
> >>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>> storage_pool_details table.
> >>>>>>>>>
> >>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>> question, we
> >>>>>>>>> pass the managed property along (this takes the place of the
> >>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>
> >>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> >>>>>> managed
> >>>>>>>>> property. If true for an attach, the necessary hypervisor data
> >>>>>>> structure is
> >>>>>>>>> created and the rest of the attach command executes to attach the
> >>>>>>> volume.
> >>>>>>>>>
> >>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> volume,
> >>>>>> the
> >>>>>>>>> same check is made. If managed, the hypervisor data structure is
> >>>>>>> removed.
> >>>>>>>>>
> >>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
> >>>>> it is
> >>>>>>>>> stored in the volumes and disk_offerings table. If we have some
> >>>>> idea,
> >>>>>>>>> that'd be cool.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> implementation
> >>>>>>>>>> attributes. I always wondered about the details field. I think
> >>>>> we
> >>>>>>> should
> >>>>>>>>>> beef up the description in the documentation regarding the
> >>>>> expected
> >>>>>>> format
> >>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>> returned on
> >>>>>>> the
> >>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> response.
> >>>>>> Why
> >>>>>>>>>> don't we return it? It seems like it would be useful for
> clients
> >>>>> to
> >>>>>> be
> >>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>
> >>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>
> >>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> Offering
> >>>>>>>>>> basis. For each Disk Offering created, you have to be able to
> >>>>>> associate
> >>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
> >>>>> it
> >>>>>>> could
> >>>>>>>>>> go there?
> >>>>>>>>>>
> >>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
> >>>>> if
> >>>>>>> it's
> >>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
How have Wei and you resolved the issue of conflicting QoS mechanisms
between the Hypervisor and Storage layers? Have the affected FSs been
updated with that decision?
In terms of merge timing, can you describe the dependencies between the patches?
Thanks,
-John
On Jun 12, 2013, at 12:43 PM, Mike Tutkowski
<mi...@solidfire.com> wrote:
> No problem, John.
>
> I still want to re-review it by myself before coming up with a new patch
> file.
>
> Also, maybe I should first wait for Wei's changes to be checked in and
> merge those into mine before generating a new patch file?
>
>
> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I just realized that I forgot to publish my review. I am offline ATM,
>> but I will publish it in the next couple of hours.
>>
>> Do you plan to update your the patch in Review Board?
>>
>> Sorry for the oversight,
>> -John
>>
>>
>>
>>
>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> <mi...@solidfire.com> wrote:
>>
>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>
>>> Just an FYI that I believe I have implemented all the areas we wanted
>>> addressed.
>>>
>>> I plan to review the code again tomorrow morning or afternoon, then send
>> in
>>> another patch.
>>>
>>> Thanks for all the work on this everyone!
>>>
>>>
>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> Sure, that sounds good.
>>>>
>>>>
>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
>> wrote:
>>>>
>>>>> Hi Mike,
>>>>>
>>>>> It looks the two feature do not have many conflicts in Java code,
>> except
>>>>> the cloudstack UI.
>>>>> If you do not mind, I will merge disk_io_throttling branch into master
>>>>> this
>>>>> week, so that you can develop based on it.
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>
>>>>>> Hey John,
>>>>>>
>>>>>> The SolidFire patch does not depend on the object_store branch, but -
>> as
>>>>>> Edison mentioned - it might be easier if we merge the SolidFire branch
>>>>> into
>>>>>> the object_store branch before object_store goes into master.
>>>>>>
>>>>>> I'm not sure how the disk_io_throttling fits into this merge strategy.
>>>>>> Perhaps Wei can chime in on that.
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
>>>>>>> solidfire, and object_store appear to have a number of overlapping
>>>>>>> elements. I understand the dependencies between the patches to be as
>>>>>>> follows:
>>>>>>>
>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>
>>>>>>> Am I correct that the device management aspects of SolidFire are
>>>>> additive
>>>>>>> to the object_store branch or there are circular dependency between
>>>>> the
>>>>>>> branches? Once we understand the dependency graph, we can determine
>>>>> the
>>>>>>> best approach to land the changes in master.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>>
>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Also, if we are good with Edison merging my code into his branch
>>>>> before
>>>>>>>> going into master, I am good with that.
>>>>>>>>
>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
>>>>> we
>>>>>> can
>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>
>>>>>>>>> 1) There should be NO references to hypervisor code in the storage
>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
>>>>>>> currently
>>>>>>>>> sends several commands to the hypervisor in use (although it does
>>>>> not
>>>>>>> know
>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>>>>>>
>>>>>>>>> 2) managed=true or managed=false can be placed in the url field (if
>>>>>> not
>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>> storage_pool_details table.
>>>>>>>>>
>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>> question, we
>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>
>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
>>>>>> managed
>>>>>>>>> property. If true for an attach, the necessary hypervisor data
>>>>>>> structure is
>>>>>>>>> created and the rest of the attach command executes to attach the
>>>>>>> volume.
>>>>>>>>>
>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
>>>>>> the
>>>>>>>>> same check is made. If managed, the hypervisor data structure is
>>>>>>> removed.
>>>>>>>>>
>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
>>>>> it is
>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
>>>>> idea,
>>>>>>>>> that'd be cool.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>>>>>>>>>> attributes. I always wondered about the details field. I think
>>>>> we
>>>>>>> should
>>>>>>>>>> beef up the description in the documentation regarding the
>>>>> expected
>>>>>>> format
>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>> returned on
>>>>>>> the
>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool response.
>>>>>> Why
>>>>>>>>>> don't we return it? It seems like it would be useful for clients
>>>>> to
>>>>>> be
>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>
>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>
>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
>>>>>> associate
>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
>>>>> it
>>>>>>> could
>>>>>>>>>> go there?
>>>>>>>>>>
>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
>>>>> if
>>>>>>> it's
>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
No problem, John. Thanks for taking the time!
I've addressed most issues in the review tonight and should be able to
complete the remaining tomorrow.
Talk to you later!
On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I just published my review @ https://reviews.apache.org/r/11479/.
>
> I apologize for the delay,
> -John
>
> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > No problem, John.
> >
> > I still want to re-review it by myself before coming up with a new patch
> > file.
> >
> > Also, maybe I should first wait for Wei's changes to be checked in and
> > merge those into mine before generating a new patch file?
> >
> >
> > On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> I just realized that I forgot to publish my review. I am offline ATM,
> >> but I will publish it in the next couple of hours.
> >>
> >> Do you plan to update your the patch in Review Board?
> >>
> >> Sorry for the oversight,
> >> -John
> >>
> >>
> >>
> >>
> >> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >> <mi...@solidfire.com> wrote:
> >>
> >>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>
> >>> Just an FYI that I believe I have implemented all the areas we wanted
> >>> addressed.
> >>>
> >>> I plan to review the code again tomorrow morning or afternoon, then
> send
> >> in
> >>> another patch.
> >>>
> >>> Thanks for all the work on this everyone!
> >>>
> >>>
> >>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> Sure, that sounds good.
> >>>>
> >>>>
> >>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
> >> wrote:
> >>>>
> >>>>> Hi Mike,
> >>>>>
> >>>>> It looks the two feature do not have many conflicts in Java code,
> >> except
> >>>>> the cloudstack UI.
> >>>>> If you do not mind, I will merge disk_io_throttling branch into
> master
> >>>>> this
> >>>>> week, so that you can develop based on it.
> >>>>>
> >>>>> -Wei
> >>>>>
> >>>>>
> >>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>
> >>>>>> Hey John,
> >>>>>>
> >>>>>> The SolidFire patch does not depend on the object_store branch, but
> -
> >> as
> >>>>>> Edison mentioned - it might be easier if we merge the SolidFire
> branch
> >>>>> into
> >>>>>> the object_store branch before object_store goes into master.
> >>>>>>
> >>>>>> I'm not sure how the disk_io_throttling fits into this merge
> strategy.
> >>>>>> Perhaps Wei can chime in on that.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
> >>>>>>> solidfire, and object_store appear to have a number of overlapping
> >>>>>>> elements. I understand the dependencies between the patches to be
> as
> >>>>>>> follows:
> >>>>>>>
> >>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>
> >>>>>>> Am I correct that the device management aspects of SolidFire are
> >>>>> additive
> >>>>>>> to the object_store branch or there are circular dependency between
> >>>>> the
> >>>>>>> branches? Once we understand the dependency graph, we can
> determine
> >>>>> the
> >>>>>>> best approach to land the changes in master.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>>
> >>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Also, if we are good with Edison merging my code into his branch
> >>>>> before
> >>>>>>>> going into master, I am good with that.
> >>>>>>>>
> >>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
> >>>>> we
> >>>>>> can
> >>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>
> >>>>>>>>> 1) There should be NO references to hypervisor code in the
> storage
> >>>>>>>>> plug-ins code (this includes the default storage plug-in, which
> >>>>>>> currently
> >>>>>>>>> sends several commands to the hypervisor in use (although it does
> >>>>> not
> >>>>>>> know
> >>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>>>>>>>>
> >>>>>>>>> 2) managed=true or managed=false can be placed in the url field
> (if
> >>>>>> not
> >>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>> storage_pool_details table.
> >>>>>>>>>
> >>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>> question, we
> >>>>>>>>> pass the managed property along (this takes the place of the
> >>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>
> >>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> >>>>>> managed
> >>>>>>>>> property. If true for an attach, the necessary hypervisor data
> >>>>>>> structure is
> >>>>>>>>> created and the rest of the attach command executes to attach the
> >>>>>>> volume.
> >>>>>>>>>
> >>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> volume,
> >>>>>> the
> >>>>>>>>> same check is made. If managed, the hypervisor data structure is
> >>>>>>> removed.
> >>>>>>>>>
> >>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
> >>>>> it is
> >>>>>>>>> stored in the volumes and disk_offerings table. If we have some
> >>>>> idea,
> >>>>>>>>> that'd be cool.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> implementation
> >>>>>>>>>> attributes. I always wondered about the details field. I think
> >>>>> we
> >>>>>>> should
> >>>>>>>>>> beef up the description in the documentation regarding the
> >>>>> expected
> >>>>>>> format
> >>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>> returned on
> >>>>>>> the
> >>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> response.
> >>>>>> Why
> >>>>>>>>>> don't we return it? It seems like it would be useful for
> clients
> >>>>> to
> >>>>>> be
> >>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>
> >>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>
> >>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> Offering
> >>>>>>>>>> basis. For each Disk Offering created, you have to be able to
> >>>>>> associate
> >>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
> >>>>> it
> >>>>>>> could
> >>>>>>>>>> go there?
> >>>>>>>>>>
> >>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
> >>>>> if
> >>>>>>> it's
> >>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
See my comments in-line below.
Thanks,
-John
On Jun 13, 2013, at 5:31 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> My thinking is, for 4.2, while not ideal, we will need to put some burden
> on the admin to configure a Disk Offering in a way that makes sense. For
> example, if he wants to use storage QoS with supported Min and Max values,
> he'll have to put in a storage tag that references the SolidFire primary
> storage (plug-in). If he puts in a storage tag that doesn't, then he's not
> going to get the Min and Max feature. We could add help text to the pop-up
> dialog that's displayed when you click in the Min and Max text fields.
>
> Same idea for Wei's feature.
>
> Not idea, true...perhaps we can brainstorm on a more comprehensive approach
> for post 4.2.
I am with you until we get to the point where we can have these two approaches to I/O QoS used simultaneously. Admittedly, it is a bit ham-fisted, but I think we need to enforce the mutual exclusion we have discussed for 4.2. The problems that can result will be difficult to diagnose, and counter-intuitive to users.
>
> Maybe in the future we could have the drivers advertise their capabilities
> and if the allocator feels a request is not being satisfied (say Min was
> entered, but it not's supported by any storage plug-in) it can throw an
> exception.
That is the direction I would like to head. To me, one of the advantage of a cloud orchestration platform like CloudStack is assigning the best set of resources to meet a set of requirements. The tags model is static -- requiring too much intrinsic knowledge between resource types. I would also say that such an approach will require a good amount thought. As an a small example, we will likely need to express most requirements as a range to permit proper fitting of requirements to resources.
>
>
> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Comments below in red.
>>
>> Thanks
>>
>>
>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> Please see my comment in-line below.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>>> Hi John,
>>>>
>>>> I've put comments below in red.
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> First and foremost, we must ensure that these two features are mutually
>>>>> exclusive in 4.2. We don't want to find a configuration that contains
>>> both
>>>>> hypervisor and storage IOPS guarantees that leads to non-deterministic
>>>>> operations.
>>>>
>>>>
>>>> Agreed
>>>>
>>>>
>>>>> Restricting QoS expression to be either hypervisor or storage oriented
>>>>> solves the problem in short term. As I understand storage tags, we
>>> have no
>>>>> means of expressing this type of mutual exclusion. I wasn't
>>> necessarily
>>>>> intending that we implement this allocation model in 4.3, but instead,
>>>>> determine if this type model would be one we would want to support in
>>> the
>>>>> future. If so, I would encourage us to ensure that the data model and
>>>>> current implementation would not preclude evolution in that direction.
>>> My
>>>>> view is that this type of allocation model is what user's expect of
>>> "cloud"
>>>>> systems -- selecting the best available resource set to fulfill a set
>>> of
>>>>> system requirements.
>>>>>
>>>>
>>>> I believe we have meet your requirement here in that what we've
>>> implemented
>>>> should not make refinement difficult in the future. If we don't modify
>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
>>> changes
>>>> to enhance the current functioning of the system.
>>>>
>>>
>>> Looking through both patches, I have to say that the aggregated result
>>> seems a bit confusing. There are six new attributes for throttled I/O and
>>> two for provisioned IOPS with no obvious grouping. My concern is not
>>> technical, but rather, about maintainability. At minimum, Javadoc should
>>> be added explaining the two sets of attributes and their mutual exclusion.
>>>
>>
>> I agree: We need JavaDoc to explain them and their mutual exclusion.
>>
>>
>>>
>>> The other part that is interesting is that throttled I/O provides both an
>>> IOPS and byte measured QoS as a rate where provisioned IOPS uses a range.
>>> In order to select the best available resource to fulfill a QoS, we would
>>> need to have the QoS expression normalized in terms of units (IOPS or
>>> bytes) and their expression (rate vs. range). If we want to achieve a
>>> model like I described, I think we would need to resolve this issue in 4.2
>>> to ensure a viable migration path.
>>
>>
>> I think we're not likely to be able to normalize the input for 4.2. Plus
>> people probably want to input the data in terms they're familiar with for
>> the products in question.
>>
>> Ideally we would fix the way we do storage tagging and let the user send
>> key/value pairs to each vendor that could be selected due to a given
>> storage tag. I'm still not sure that would solve it because what happens if
>> you change the storage tag of a given Primary Storage after having created
>> a Disk Offering?
>>
>> Basically storage tagging is kind of a mess and we should re-think it.
>>
>> Also, we need to have a way for the drivers to expose their supported
>> feature sets so the allocators can make good choices.
>>
>>
>>>
>>>>
>>>>>
>>>>> As I think through the implications of these requirements and reflect
>>> on
>>>>> the reviews, I don't understand why they haven't already impacted the
>>>>> allocators and planners. As it stands, the current provisioned IOPS
>>> has no
>>>>> checks to ensure that the volumes are allocated to devices that have
>>>>> capacity to fulfill the requested QoS. Therefore, as I understand the
>>>>> current patch, we can overcommit storage resources -- potentially
>>> causing
>>>>> none of the QoS obligations from being fulfilled. It seems to me that
>>> a
>>>>> DataStore supporting provisioned IOPS should express the maximum IOPS
>>> which
>>>>> it can support and some type of overcommitment factor. This
>>> information
>>>>> should be used by the storage allocators to determine the device best
>>> able
>>>>> to support the resources needs of a volume. It seems that a similar
>>> set of
>>>>> considerations would need to be added to the Hypervisor layer which
>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>
>>>>
>>>> Yeah, for this first release, we just followed the path that was
>>> previously
>>>> established for other properties you see on dialogs in CS: Just because
>>>> they're there doesn't mean the vendor your VM is deployed to supports
>>> them.
>>>> It is then up to the admin to make sure he inputs, say, a storage tag
>>> that
>>>> confines the deployment only to storage that supports the selected
>>>> features. This is not ideal, but it's kind of the way CloudStack works
>>>> today.
>>>
>>> I understand the tag functionality, and the need for the administrator to
>>> very carefully construct offerings. My concern is that we over guarantee a
>>> resource's available IOPS. For the purposes of illustration, let's say we
>>> have a SolidFire, and the max IOPS for that device is 100000. We also
>>> know that we can safely oversubscribe by 50%. Therefore, we need to ensure
>>> that we don't allocate more than 150,000 guaranteed IOPS from that device.
>>> Intuitively, it seems like the DataStore configuration should have a max
>>> assignable IOPS and overcommitment factor. As we allocate volumes and
>>> attach VMs, we need to ensure that guarantee more IOPS exceed the
>>> configured maximum for a DataStore. Does that make sense?
>>>
>>
>> I think that's a good idea for a future enhancement. I'm not even sure I
>> can query the SAN to find out how many IOPS safely remain. I'd have to get
>> all of the min values for all of the volumes on the SAN and total them up,
>> I suppose, and subtract it from the total (user facing) supported IOPS of
>> the system.
>>
>>
>>>
>>>>
>>>>
>>>>>
>>>>> Another question occurs to me -- should we allow non-QoS resources to
>>> be
>>>>> assigned to hosts/storage devices that ensure QoS? For provisioned
>>> IOPS, I
>>>>> think a side effect of the current implementation is SolidFire volumes
>>>>> always have a QoS. However, for hypervisor throttled I/O, it seems
>>>>> entirely possible for non-QoS VMs to allocated side-by-side with QoS
>>> VMs.
>>>>> In this scenario, a greedy, unbounded VM could potentially starve out
>>> the
>>>>> other VMs on the host -- preventing the QoSes defined the collocated
>>> VMs
>>>>> from being fulfilled.
>>>>>
>>>>
>>>> You can make SolidFire volumes (inside and outside of CS) and not
>>> specify
>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the defaults
>>> we
>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will have
>>>> your Mins met.
>>>>
>>>> It sounds like you're perhaps looking for a storage tags exclusions
>>> list,
>>>> which might be nice to have at some point (i.e. don't deploy my volume
>>> to
>>>> storage with these following tags).
>>>
>>> I don't like the idea of a storage tags exclusion list as it would
>>> complicate component assembly. It would require a storage plugin to
>>> anticipate all of the possible component assemblies and determine the
>>> invalid relationships. I prefer that drivers express their capabilities
>>> which can be matched to a set of requested requirements.
>>>
>>
>> I'm not sure why a storage plug-in would care about inclusion or exclusion
>> lists. It just needs to advertise its functionality in a way the allocator
>> understands.
>>
>>
>>>
>>>>
>>>> I agree with your assessment of Hypervisor QoS. Since it only limits
>>> IOPS,
>>>> it does not solve the Noisy Neighbor problem. Only a system with
>>> guaranteed
>>>> minimum IOPS does this.
>>>
>>> As I said, for SolidFire, it sounds like this problem does not exist.
>>> However, I am concerned with the more general case as we supported more
>>> devices with provisioned IOPS.
>>>
>>
>> Post 4.2 we need to investigate a way to pass vendor-specific values to
>> drivers. Min and Max and pretty industry standard for provisioned IOPS, but
>> what if you break them out by read and write or do something else? We need
>> a more general mechanism.
>>
>>>
>>>>
>>>>
>>>>>
>>>>> In my opinion, we need to ensure that hypervisor throttled I/O and
>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>
>>>>
>>>> Agreed
>>>>
>>>>
>>>>> We also need to understand the implications of these QoS guarantees on
>>>>> operation of the system to ensure that the underlying hardware
>>> resources
>>>>> can fulfill them. Given the time frame, we will likely be forced to
>>> make
>>>>> compromises to achieve these goals, and refine the implementation in
>>> 4.3.
>>>>>
>>>>
>>>> I agree, John. I also think you've come up with some great ideas for
>>> 4.3. :)
>>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
>>> tags).
>>>>> I
>>>>>> believe John was just throwing out an idea that - in addition to
>>> storage
>>>>>> tags - we could look into these allocators (storage QoS being
>>> preferred,
>>>>>> then hypervisor QoS if storage QoS is not available, but hypervisor
>>> QoS
>>>>> is).
>>>>>>
>>>>>> I think John's concern is that you can enter in values for Wei's and
>>> my
>>>>>> feature that are not honored by other vendors (at least yet), so he
>>> was
>>>>>> hoping - in addition to storage tags - for the allocators to prefer
>>> these
>>>>>> vendors when these fields are filled in. As it stands today in
>>>>> CloudStack,
>>>>>> we already have this kind of an issue with other features (fields in
>>>>>> dialogs for features that not all vendors support). Perhaps post 4.2
>>> we
>>>>>> could look into generic name/value pairs (this is how OpenStack
>>> addresses
>>>>>> the issue).
>>>>>>
>>>>>> Honestly, I think we're too late in the game (two weeks until code
>>>>> freeze)
>>>>>> to go too deeply down that path in 4.2.
>>>>>>
>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields and
>>> my
>>>>>> fields as is, make sure only one or the other feature has data entered
>>>>> for
>>>>>> it (or neither), and call it good for 4.2.
>>>>>>
>>>>>> Then let's step back and look into a more general-purpose design that
>>> can
>>>>>> be applied throughout CloudStack where we have these situations.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> I just published my review @ https://reviews.apache.org/r/11479/.
>>>>>>>
>>>>>>> I apologize for the delay,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> No problem, John.
>>>>>>>>
>>>>>>>> I still want to re-review it by myself before coming up with a new
>>>>> patch
>>>>>>>> file.
>>>>>>>>
>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked in
>>> and
>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> I just realized that I forgot to publish my review. I am offline
>>> ATM,
>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>
>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>
>>>>>>>>> Sorry for the oversight,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>>>>>>
>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
>>> wanted
>>>>>>>>>> addressed.
>>>>>>>>>>
>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
>>> then
>>>>>>> send
>>>>>>>>> in
>>>>>>>>>> another patch.
>>>>>>>>>>
>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> ustcweizhou@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>>> code,
>>>>>>>>> except
>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>>>>>>> master
>>>>>>>>>>>> this
>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>
>>>>>>>>>>>> -Wei
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store branch,
>>>>> but
>>>>>>> -
>>>>>>>>> as
>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
>>>>>>> branch
>>>>>>>>>>>> into
>>>>>>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>>>>> strategy.
>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>> jburwell@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>> disk_io_throttling,
>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>>>> overlapping
>>>>>>>>>>>>>> elements. I understand the dependencies between the patches
>>> to
>>>>> be
>>>>>>> as
>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Am I correct that the device management aspects of SolidFire
>>> are
>>>>>>>>>>>> additive
>>>>>>>>>>>>>> to the object_store branch or there are circular dependency
>>>>> between
>>>>>>>>>>>> the
>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>>>>> determine
>>>>>>>>>>>> the
>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>>> branch
>>>>>>>>>>>> before
>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>> merge
>>>>> and
>>>>>>>>>>>> we
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>>>>>>> storage
>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>>> which
>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although it
>>>>> does
>>>>>>>>>>>> not
>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>>> use))
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>>> field
>>>>>>> (if
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
>>>>> the
>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>>> data
>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>>> attach
>>>>> the
>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>>>>>>> volume,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> structure
>>>>> is
>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>>>>> unless
>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
>>> some
>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>>>>> implementation
>>>>>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>>>>> think
>>>>>>>>>>>> we
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>>>>>>> expected
>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>>>>>>> returned on
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>>>>> response.
>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>>>>>>> clients
>>>>>>>>>>>> to
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>>>>>>> Offering
>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able
>>> to
>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>>>>> Maybe
>>>>>>>>>>>> it
>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
>>> the
>>>>> GUI
>>>>>>>>>>>> if
>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Alternatively, we could always add a new table, storage_features. It could
contain three columns: Our typical ID column, a foreign key that maps back
to the storage_pool table, and a column for the feature's name (this string
would map to an enumeration in the codebase).
Ex.
storage_pool
ID Features
1 8
storage_pool_features
ID Feature Feature_Name
1 8 Min
2 8 Max
The storage pool with ID of 1 supports Min and Max.
If your DiskProfile (passed to the allocator) has a Min and Max greater
than 0, then the storage pool we choose has to support Min and Max.
We could always worry about exceeding a certain number of IOPS in the
storage pool and skipping that storage pool at some future release.
On Thu, Jun 13, 2013 at 6:45 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Overall, I agree with the steps to below for 4.2. However, we may want to
>> throw an exception if we can not fulfill a requested QoS. If the user is
>> expecting that the hypervisor will provide a particular QoS, and that is
>> not possible, it seems like we should inform them rather than silently
>> ignoring their request.
>>
>
> Sure, that sounds reasonable.
>
> We'd have to come up with some way for the allocators to know about the
> requested storage QoS and then query the candidate drivers.
>
> Any thoughts on how we might do that?
>
> ********************
>
> It looks like we might be able to modify the
> ZoneWideStoragePoolAllocator's filter method to check if the storage in
> question supports, say, Min and Max IOPS values (if either value is greater
> than 0, then we consider it set).
>
> We'd have to decide how we want to store information about the features a
> storage pool supports. Would it be in the storage_pool_details table
> perhaps?
>
>
> On Thu, Jun 13, 2013 at 4:09 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Comments below in red.
>>
>> Thanks
>>
>>
>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> Overall, I agree with the steps to below for 4.2. However, we may want
>>> to throw an exception if we can not fulfill a requested QoS. If the user
>>> is expecting that the hypervisor will provide a particular QoS, and that is
>>> not possible, it seems like we should inform them rather than silently
>>> ignoring their request.
>>>
>>
>> Sure, that sounds reasonable.
>>
>> We'd have to come up with some way for the allocators to know about the
>> requested storage QoS and then query the candidate drivers.
>>
>> Any thoughts on how we might do that?
>>
>>
>>>
>>> To collect my thoughts from previous parts of the thread, I am
>>> uncomfortable with the idea that the management server can overcommit a
>>> resource. You had mentioned querying the device for available IOPS. While
>>> that would be nice, it seems like we could fall back to a max IOPS and
>>> overcommit factor manually calculated and entered by the
>>> administrator/operator. I think such threshold and allocation rails should
>>> be added for both provisioned IOPS and throttled I/O -- it is a basic
>>> feature of any cloud orchestration platform.
>>>
>>
>> Are you thinking this ability would make it into 4.2? Just curious what
>> release we're talking about here. For the SolidFire SAN, you might have,
>> say, four separate storage nodes to start (200,000 IOPS) and then later add
>> a new node (now you're at 250,000 IOPS). CS would have to have a way to
>> know that the number of supported IOPS has increased.
>>
>>
>>>
>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>> implementation specific manner. I think we need to arrive at a general
>>> model that can be exploited by the allocators and planners. We should
>>> restrict implementation specific key-value pairs (call them details,
>>> extended data, whatever) to information that is unique to the driver and
>>> would provide no useful information to the management server's
>>> orchestration functions. A resource QoS does not seem to fit those
>>> criteria.
>>>
>>
>> I wonder if this would be a good discussion topic for Sunday's CS Collab
>> Conf hack day that Joe just sent out an e-mail about?
>>
>>
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> > So, here's my suggestion for 4.2:
>>> >
>>> > Accept the values as they are currently required (four new fields for
>>> Wei's
>>> > feature or two new fields for mine).
>>> >
>>> > The Add Disk Offering dialog needs three new radio buttons:
>>> >
>>> > 1) No QoS
>>> >
>>> > 2) Hypervisor QoS
>>> >
>>> > 3) Storage Qos
>>> >
>>> > The admin needs to specify storage tags that only map to storage that
>>> > supports Storage QoS.
>>> >
>>> > The admin needs to be aware for Hypervisor QoS that unless all
>>> hypervisors
>>> > in use support the new fields, they may not be enforced.
>>> >
>>> > Post 4.3:
>>> >
>>> > Come up with a way to more generally enter these parameters (probably
>>> just
>>> > key/value pairs sent to the drivers).
>>> >
>>> > Have the drivers expose their feature set so the allocators can
>>> consider
>>> > them more fully and throw an exception if there is not a sufficient
>>> match.
>>> >
>>> >
>>> > On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>> > mike.tutkowski@solidfire.com> wrote:
>>> >
>>> >> My thinking is, for 4.2, while not ideal, we will need to put some
>>> burden
>>> >> on the admin to configure a Disk Offering in a way that makes sense.
>>> For
>>> >> example, if he wants to use storage QoS with supported Min and Max
>>> values,
>>> >> he'll have to put in a storage tag that references the SolidFire
>>> primary
>>> >> storage (plug-in). If he puts in a storage tag that doesn't, then
>>> he's not
>>> >> going to get the Min and Max feature. We could add help text to the
>>> pop-up
>>> >> dialog that's displayed when you click in the Min and Max text fields.
>>> >>
>>> >> Same idea for Wei's feature.
>>> >>
>>> >> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>> >> approach for post 4.2.
>>> >>
>>> >> Maybe in the future we could have the drivers advertise their
>>> capabilities
>>> >> and if the allocator feels a request is not being satisfied (say Min
>>> was
>>> >> entered, but it not's supported by any storage plug-in) it can throw
>>> an
>>> >> exception.
>>> >>
>>> >>
>>> >> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>> >> mike.tutkowski@solidfire.com> wrote:
>>> >>
>>> >>> Comments below in red.
>>> >>>
>>> >>> Thanks
>>> >>>
>>> >>>
>>> >>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>> >>>
>>> >>>> Mike,
>>> >>>>
>>> >>>> Please see my comment in-line below.
>>> >>>>
>>> >>>> Thanks,
>>> >>>> -John
>>> >>>>
>>> >>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>> >>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>
>>> >>>>> Hi John,
>>> >>>>>
>>> >>>>> I've put comments below in red.
>>> >>>>>
>>> >>>>> Thanks!
>>> >>>>>
>>> >>>>>
>>> >>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jburwell@basho.com
>>> >
>>> >>>> wrote:
>>> >>>>>
>>> >>>>>> Mike,
>>> >>>>>>
>>> >>>>>> First and foremost, we must ensure that these two features are
>>> >>>> mutually
>>> >>>>>> exclusive in 4.2. We don't want to find a configuration that
>>> >>>> contains both
>>> >>>>>> hypervisor and storage IOPS guarantees that leads to
>>> non-deterministic
>>> >>>>>> operations.
>>> >>>>>
>>> >>>>>
>>> >>>>> Agreed
>>> >>>>>
>>> >>>>>
>>> >>>>>> Restricting QoS expression to be either hypervisor or storage
>>> oriented
>>> >>>>>> solves the problem in short term. As I understand storage tags,
>>> we
>>> >>>> have no
>>> >>>>>> means of expressing this type of mutual exclusion. I wasn't
>>> >>>> necessarily
>>> >>>>>> intending that we implement this allocation model in 4.3, but
>>> instead,
>>> >>>>>> determine if this type model would be one we would want to
>>> support in
>>> >>>> the
>>> >>>>>> future. If so, I would encourage us to ensure that the data
>>> model and
>>> >>>>>> current implementation would not preclude evolution in that
>>> >>>> direction. My
>>> >>>>>> view is that this type of allocation model is what user's expect
>>> of
>>> >>>> "cloud"
>>> >>>>>> systems -- selecting the best available resource set to fulfill a
>>> >>>> set of
>>> >>>>>> system requirements.
>>> >>>>>>
>>> >>>>>
>>> >>>>> I believe we have meet your requirement here in that what we've
>>> >>>> implemented
>>> >>>>> should not make refinement difficult in the future. If we don't
>>> modify
>>> >>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
>>> >>>> changes
>>> >>>>> to enhance the current functioning of the system.
>>> >>>>>
>>> >>>>
>>> >>>> Looking through both patches, I have to say that the aggregated
>>> result
>>> >>>> seems a bit confusing. There are six new attributes for throttled
>>> I/O and
>>> >>>> two for provisioned IOPS with no obvious grouping. My concern is
>>> not
>>> >>>> technical, but rather, about maintainability. At minimum, Javadoc
>>> should
>>> >>>> be added explaining the two sets of attributes and their mutual
>>> exclusion.
>>> >>>>
>>> >>>
>>> >>> I agree: We need JavaDoc to explain them and their mutual exclusion.
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> The other part that is interesting is that throttled I/O provides
>>> both
>>> >>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses
>>> a
>>> >>>> range. In order to select the best available resource to fulfill a
>>> QoS, we
>>> >>>> would need to have the QoS expression normalized in terms of units
>>> (IOPS or
>>> >>>> bytes) and their expression (rate vs. range). If we want to
>>> achieve a
>>> >>>> model like I described, I think we would need to resolve this issue
>>> in 4.2
>>> >>>> to ensure a viable migration path.
>>> >>>
>>> >>>
>>> >>> I think we're not likely to be able to normalize the input for 4.2.
>>> Plus
>>> >>> people probably want to input the data in terms they're familiar
>>> with for
>>> >>> the products in question.
>>> >>>
>>> >>> Ideally we would fix the way we do storage tagging and let the user
>>> send
>>> >>> key/value pairs to each vendor that could be selected due to a given
>>> >>> storage tag. I'm still not sure that would solve it because what
>>> happens if
>>> >>> you change the storage tag of a given Primary Storage after having
>>> created
>>> >>> a Disk Offering?
>>> >>>
>>> >>> Basically storage tagging is kind of a mess and we should re-think
>>> it.
>>> >>>
>>> >>> Also, we need to have a way for the drivers to expose their supported
>>> >>> feature sets so the allocators can make good choices.
>>> >>>
>>> >>>
>>> >>>>
>>> >>>>>
>>> >>>>>>
>>> >>>>>> As I think through the implications of these requirements and
>>> reflect
>>> >>>> on
>>> >>>>>> the reviews, I don't understand why they haven't already impacted
>>> the
>>> >>>>>> allocators and planners. As it stands, the current provisioned
>>> IOPS
>>> >>>> has no
>>> >>>>>> checks to ensure that the volumes are allocated to devices that
>>> have
>>> >>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>> understand the
>>> >>>>>> current patch, we can overcommit storage resources -- potentially
>>> >>>> causing
>>> >>>>>> none of the QoS obligations from being fulfilled. It seems to me
>>> >>>> that a
>>> >>>>>> DataStore supporting provisioned IOPS should express the maximum
>>> IOPS
>>> >>>> which
>>> >>>>>> it can support and some type of overcommitment factor. This
>>> >>>> information
>>> >>>>>> should be used by the storage allocators to determine the device
>>> best
>>> >>>> able
>>> >>>>>> to support the resources needs of a volume. It seems that a
>>> similar
>>> >>>> set of
>>> >>>>>> considerations would need to be added to the Hypervisor layer
>>> which
>>> >>>>>> allocating a VM to a host to prevent oversubscription.
>>> >>>>>>
>>> >>>>>
>>> >>>>> Yeah, for this first release, we just followed the path that was
>>> >>>> previously
>>> >>>>> established for other properties you see on dialogs in CS: Just
>>> because
>>> >>>>> they're there doesn't mean the vendor your VM is deployed to
>>> supports
>>> >>>> them.
>>> >>>>> It is then up to the admin to make sure he inputs, say, a storage
>>> tag
>>> >>>> that
>>> >>>>> confines the deployment only to storage that supports the selected
>>> >>>>> features. This is not ideal, but it's kind of the way CloudStack
>>> works
>>> >>>>> today.
>>> >>>>
>>> >>>> I understand the tag functionality, and the need for the
>>> administrator
>>> >>>> to very carefully construct offerings. My concern is that we over
>>> >>>> guarantee a resource's available IOPS. For the purposes of
>>> illustration,
>>> >>>> let's say we have a SolidFire, and the max IOPS for that device is
>>> 100000.
>>> >>>> We also know that we can safely oversubscribe by 50%. Therefore,
>>> we
>>> >>>> need to ensure that we don't allocate more than 150,000 guaranteed
>>> IOPS
>>> >>>> from that device. Intuitively, it seems like the DataStore
>>> configuration
>>> >>>> should have a max assignable IOPS and overcommitment factor. As we
>>> >>>> allocate volumes and attach VMs, we need to ensure that guarantee
>>> more IOPS
>>> >>>> exceed the configured maximum for a DataStore. Does that make
>>> sense?
>>> >>>>
>>> >>>
>>> >>> I think that's a good idea for a future enhancement. I'm not even
>>> sure I
>>> >>> can query the SAN to find out how many IOPS safely remain. I'd have
>>> to get
>>> >>> all of the min values for all of the volumes on the SAN and total
>>> them up,
>>> >>> I suppose, and subtract it from the total (user facing) supported
>>> IOPS of
>>> >>> the system.
>>> >>>
>>> >>>
>>> >>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>>
>>> >>>>>> Another question occurs to me -- should we allow non-QoS
>>> resources to
>>> >>>> be
>>> >>>>>> assigned to hosts/storage devices that ensure QoS? For
>>> provisioned
>>> >>>> IOPS, I
>>> >>>>>> think a side effect of the current implementation is SolidFire
>>> volumes
>>> >>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>>> seems
>>> >>>>>> entirely possible for non-QoS VMs to allocated side-by-side with
>>> QoS
>>> >>>> VMs.
>>> >>>>>> In this scenario, a greedy, unbounded VM could potentially starve
>>> out
>>> >>>> the
>>> >>>>>> other VMs on the host -- preventing the QoSes defined the
>>> collocated
>>> >>>> VMs
>>> >>>>>> from being fulfilled.
>>> >>>>>>
>>> >>>>>
>>> >>>>> You can make SolidFire volumes (inside and outside of CS) and not
>>> >>>> specify
>>> >>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>> defaults
>>> >>>> we
>>> >>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will
>>> >>>> have
>>> >>>>> your Mins met.
>>> >>>>>
>>> >>>>> It sounds like you're perhaps looking for a storage tags exclusions
>>> >>>> list,
>>> >>>>> which might be nice to have at some point (i.e. don't deploy my
>>> volume
>>> >>>> to
>>> >>>>> storage with these following tags).
>>> >>>>
>>> >>>> I don't like the idea of a storage tags exclusion list as it would
>>> >>>> complicate component assembly. It would require a storage plugin to
>>> >>>> anticipate all of the possible component assemblies and determine
>>> the
>>> >>>> invalid relationships. I prefer that drivers express their
>>> capabilities
>>> >>>> which can be matched to a set of requested requirements.
>>> >>>>
>>> >>>
>>> >>> I'm not sure why a storage plug-in would care about inclusion or
>>> >>> exclusion lists. It just needs to advertise its functionality in a
>>> way the
>>> >>> allocator understands.
>>> >>>
>>> >>>
>>> >>>>
>>> >>>>>
>>> >>>>> I agree with your assessment of Hypervisor QoS. Since it only
>>> limits
>>> >>>> IOPS,
>>> >>>>> it does not solve the Noisy Neighbor problem. Only a system with
>>> >>>> guaranteed
>>> >>>>> minimum IOPS does this.
>>> >>>>
>>> >>>> As I said, for SolidFire, it sounds like this problem does not
>>> exist.
>>> >>>> However, I am concerned with the more general case as we supported
>>> more
>>> >>>> devices with provisioned IOPS.
>>> >>>>
>>> >>>
>>> >>> Post 4.2 we need to investigate a way to pass vendor-specific values
>>> to
>>> >>> drivers. Min and Max and pretty industry standard for provisioned
>>> IOPS, but
>>> >>> what if you break them out by read and write or do something else?
>>> We need
>>> >>> a more general mechanism.
>>> >>>
>>> >>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>>
>>> >>>>>> In my opinion, we need to ensure that hypervisor throttled I/O
>>> and
>>> >>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>> >>>>>
>>> >>>>>
>>> >>>>> Agreed
>>> >>>>>
>>> >>>>>
>>> >>>>>> We also need to understand the implications of these QoS
>>> guarantees on
>>> >>>>>> operation of the system to ensure that the underlying hardware
>>> >>>> resources
>>> >>>>>> can fulfill them. Given the time frame, we will likely be forced
>>> to
>>> >>>> make
>>> >>>>>> compromises to achieve these goals, and refine the implementation
>>> in
>>> >>>> 4.3.
>>> >>>>>>
>>> >>>>>
>>> >>>>> I agree, John. I also think you've come up with some great ideas
>>> for
>>> >>>> 4.3. :)
>>> >>>>>
>>> >>>>>
>>> >>>>>>
>>> >>>>>> Thanks,
>>> >>>>>> -John
>>> >>>>>>
>>> >>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>> >>>> mike.tutkowski@solidfire.com>
>>> >>>>>> wrote:
>>> >>>>>>
>>> >>>>>>> Hi,
>>> >>>>>>>
>>> >>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
>>> >>>> tags).
>>> >>>>>> I
>>> >>>>>>> believe John was just throwing out an idea that - in addition to
>>> >>>> storage
>>> >>>>>>> tags - we could look into these allocators (storage QoS being
>>> >>>> preferred,
>>> >>>>>>> then hypervisor QoS if storage QoS is not available, but
>>> hypervisor
>>> >>>> QoS
>>> >>>>>> is).
>>> >>>>>>>
>>> >>>>>>> I think John's concern is that you can enter in values for Wei's
>>> and
>>> >>>> my
>>> >>>>>>> feature that are not honored by other vendors (at least yet), so
>>> he
>>> >>>> was
>>> >>>>>>> hoping - in addition to storage tags - for the allocators to
>>> prefer
>>> >>>> these
>>> >>>>>>> vendors when these fields are filled in. As it stands today in
>>> >>>>>> CloudStack,
>>> >>>>>>> we already have this kind of an issue with other features
>>> (fields in
>>> >>>>>>> dialogs for features that not all vendors support). Perhaps post
>>> 4.2
>>> >>>> we
>>> >>>>>>> could look into generic name/value pairs (this is how OpenStack
>>> >>>> addresses
>>> >>>>>>> the issue).
>>> >>>>>>>
>>> >>>>>>> Honestly, I think we're too late in the game (two weeks until
>>> code
>>> >>>>>> freeze)
>>> >>>>>>> to go too deeply down that path in 4.2.
>>> >>>>>>>
>>> >>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields
>>> >>>> and my
>>> >>>>>>> fields as is, make sure only one or the other feature has data
>>> >>>> entered
>>> >>>>>> for
>>> >>>>>>> it (or neither), and call it good for 4.2.
>>> >>>>>>>
>>> >>>>>>> Then let's step back and look into a more general-purpose design
>>> >>>> that can
>>> >>>>>>> be applied throughout CloudStack where we have these situations.
>>> >>>>>>>
>>> >>>>>>> What do you think?
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>> jburwell@basho.com>
>>> >>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>>> Mike,
>>> >>>>>>>>
>>> >>>>>>>> I just published my review @
>>> https://reviews.apache.org/r/11479/.
>>> >>>>>>>>
>>> >>>>>>>> I apologize for the delay,
>>> >>>>>>>> -John
>>> >>>>>>>>
>>> >>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>> >>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> No problem, John.
>>> >>>>>>>>>
>>> >>>>>>>>> I still want to re-review it by myself before coming up with a
>>> new
>>> >>>>>> patch
>>> >>>>>>>>> file.
>>> >>>>>>>>>
>>> >>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>> checked in
>>> >>>> and
>>> >>>>>>>>> merge those into mine before generating a new patch file?
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>> jburwell@basho.com
>>> >>>>>
>>> >>>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>> Mike,
>>> >>>>>>>>>>
>>> >>>>>>>>>> I just realized that I forgot to publish my review. I am
>>> offline
>>> >>>> ATM,
>>> >>>>>>>>>> but I will publish it in the next couple of hours.
>>> >>>>>>>>>>
>>> >>>>>>>>>> Do you plan to update your the patch in Review Board?
>>> >>>>>>>>>>
>>> >>>>>>>>>> Sorry for the oversight,
>>> >>>>>>>>>> -John
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>> >>>>>>>>>> <mi...@solidfire.com> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this
>>> :) ),
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
>>> we
>>> >>>> wanted
>>> >>>>>>>>>>> addressed.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I plan to review the code again tomorrow morning or
>>> afternoon,
>>> >>>> then
>>> >>>>>>>> send
>>> >>>>>>>>>> in
>>> >>>>>>>>>>> another patch.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks for all the work on this everyone!
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Sure, that sounds good.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> >>>> ustcweizhou@gmail.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> Hi Mike,
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>>> >>>> code,
>>> >>>>>>>>>> except
>>> >>>>>>>>>>>>> the cloudstack UI.
>>> >>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
>>> into
>>> >>>>>>>> master
>>> >>>>>>>>>>>>> this
>>> >>>>>>>>>>>>> week, so that you can develop based on it.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Hey John,
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>>> >>>> branch,
>>> >>>>>> but
>>> >>>>>>>> -
>>> >>>>>>>>>> as
>>> >>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>> >>>> SolidFire
>>> >>>>>>>> branch
>>> >>>>>>>>>>>>> into
>>> >>>>>>>>>>>>>> the object_store branch before object_store goes into
>>> master.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>> merge
>>> >>>>>>>> strategy.
>>> >>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>> >>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>> >>>>>> disk_io_throttling,
>>> >>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>> >>>>>> overlapping
>>> >>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>> patches
>>> >>>> to
>>> >>>>>> be
>>> >>>>>>>> as
>>> >>>>>>>>>>>>>>> follows:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>> SolidFire
>>> >>>> are
>>> >>>>>>>>>>>>> additive
>>> >>>>>>>>>>>>>>> to the object_store branch or there are circular
>>> dependency
>>> >>>>>> between
>>> >>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
>>> can
>>> >>>>>>>> determine
>>> >>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>> best approach to land the changes in master.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
>>> his
>>> >>>> branch
>>> >>>>>>>>>>>>> before
>>> >>>>>>>>>>>>>>>> going into master, I am good with that.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>> >>>> merge
>>> >>>>>> and
>>> >>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
>>> the
>>> >>>>>>>> storage
>>> >>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>> plug-in,
>>> >>>> which
>>> >>>>>>>>>>>>>>> currently
>>> >>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>> (although
>>> >>>> it
>>> >>>>>> does
>>> >>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>> know
>>> >>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
>>> in
>>> >>>> use))
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
>>> url
>>> >>>> field
>>> >>>>>>>> (if
>>> >>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
>>> the
>>> >>>>>>>>>>>>>>>>> storage_pool_details table.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
>>> in
>>> >>>>>>>>>>>>> question, we
>>> >>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
>>> of
>>> >>>> the
>>> >>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>> checks
>>> >>>> for
>>> >>>>>> the
>>> >>>>>>>>>>>>>> managed
>>> >>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>> hypervisor
>>> >>>> data
>>> >>>>>>>>>>>>>>> structure is
>>> >>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>>> >>>> attach
>>> >>>>>> the
>>> >>>>>>>>>>>>>>> volume.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>> detach a
>>> >>>>>>>> volume,
>>> >>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> >>>> structure
>>> >>>>>> is
>>> >>>>>>>>>>>>>>> removed.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
>>> 4.2
>>> >>>>>> unless
>>> >>>>>>>>>>>>> it is
>>> >>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
>>> have
>>> >>>> some
>>> >>>>>>>>>>>>> idea,
>>> >>>>>>>>>>>>>>>>> that'd be cool.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>> >>>>>>>> implementation
>>> >>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>> field. I
>>> >>>>>> think
>>> >>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>> regarding the
>>> >>>>>>>>>>>>> expected
>>> >>>>>>>>>>>>>>> format
>>> >>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
>>> not
>>> >>>>>>>>>>>>> returned on
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>> listStoragePool
>>> >>>>>>>> response.
>>> >>>>>>>>>>>>>> Why
>>> >>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
>>> for
>>> >>>>>>>> clients
>>> >>>>>>>>>>>>> to
>>> >>>>>>>>>>>>>> be
>>> >>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>> Offering-by-Disk
>>> >>>>>>>> Offering
>>> >>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>>> >>>> able to
>>> >>>>>>>>>>>>>> associate
>>> >>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>>> table.
>>> >>>>>> Maybe
>>> >>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>> could
>>> >>>>>>>>>>>>>>>>>> go there?
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
>>> in
>>> >>>> the
>>> >>>>>> GUI
>>> >>>>>>>>>>>>> if
>>> >>>>>>>>>>>>>>> it's
>>> >>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> --
>>> >>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> --
>>> >>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>> o: 303.746.7302
>>> >>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>> *â„¢*
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> *Mike Tutkowski*
>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>> o: 303.746.7302
>>> >>>>>>> Advancing the way the world uses the
>>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>> *â„¢*
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> *Mike Tutkowski*
>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>> e: mike.tutkowski@solidfire.com
>>> >>>>> o: 303.746.7302
>>> >>>>> Advancing the way the world uses the
>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>> *â„¢*
>>> >>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Mike Tutkowski*
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>> e: mike.tutkowski@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>> *â„¢*
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> *Mike Tutkowski*
>>> >> *Senior CloudStack Developer, SolidFire Inc.*
>>> >> e: mike.tutkowski@solidfire.com
>>> >> o: 303.746.7302
>>> >> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >> *â„¢*
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
>
> Overall, I agree with the steps to below for 4.2. However, we may want to
> throw an exception if we can not fulfill a requested QoS. If the user is
> expecting that the hypervisor will provide a particular QoS, and that is
> not possible, it seems like we should inform them rather than silently
> ignoring their request.
>
Sure, that sounds reasonable.
We'd have to come up with some way for the allocators to know about the
requested storage QoS and then query the candidate drivers.
Any thoughts on how we might do that?
********************
It looks like we might be able to modify the ZoneWideStoragePoolAllocator's
filter method to check if the storage in question supports, say, Min and
Max IOPS values (if either value is greater than 0, then we consider it
set).
We'd have to decide how we want to store information about the features a
storage pool supports. Would it be in the storage_pool_details table
perhaps?
On Thu, Jun 13, 2013 at 4:09 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Comments below in red.
>
> Thanks
>
>
> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Overall, I agree with the steps to below for 4.2. However, we may want
>> to throw an exception if we can not fulfill a requested QoS. If the user
>> is expecting that the hypervisor will provide a particular QoS, and that is
>> not possible, it seems like we should inform them rather than silently
>> ignoring their request.
>>
>
> Sure, that sounds reasonable.
>
> We'd have to come up with some way for the allocators to know about the
> requested storage QoS and then query the candidate drivers.
>
> Any thoughts on how we might do that?
>
>
>>
>> To collect my thoughts from previous parts of the thread, I am
>> uncomfortable with the idea that the management server can overcommit a
>> resource. You had mentioned querying the device for available IOPS. While
>> that would be nice, it seems like we could fall back to a max IOPS and
>> overcommit factor manually calculated and entered by the
>> administrator/operator. I think such threshold and allocation rails should
>> be added for both provisioned IOPS and throttled I/O -- it is a basic
>> feature of any cloud orchestration platform.
>>
>
> Are you thinking this ability would make it into 4.2? Just curious what
> release we're talking about here. For the SolidFire SAN, you might have,
> say, four separate storage nodes to start (200,000 IOPS) and then later add
> a new node (now you're at 250,000 IOPS). CS would have to have a way to
> know that the number of supported IOPS has increased.
>
>
>>
>> For 4.3, I don't like the idea that a QoS would be expressed in a
>> implementation specific manner. I think we need to arrive at a general
>> model that can be exploited by the allocators and planners. We should
>> restrict implementation specific key-value pairs (call them details,
>> extended data, whatever) to information that is unique to the driver and
>> would provide no useful information to the management server's
>> orchestration functions. A resource QoS does not seem to fit those
>> criteria.
>>
>
> I wonder if this would be a good discussion topic for Sunday's CS Collab
> Conf hack day that Joe just sent out an e-mail about?
>
>
>>
>> Thanks,
>> -John
>>
>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > So, here's my suggestion for 4.2:
>> >
>> > Accept the values as they are currently required (four new fields for
>> Wei's
>> > feature or two new fields for mine).
>> >
>> > The Add Disk Offering dialog needs three new radio buttons:
>> >
>> > 1) No QoS
>> >
>> > 2) Hypervisor QoS
>> >
>> > 3) Storage Qos
>> >
>> > The admin needs to specify storage tags that only map to storage that
>> > supports Storage QoS.
>> >
>> > The admin needs to be aware for Hypervisor QoS that unless all
>> hypervisors
>> > in use support the new fields, they may not be enforced.
>> >
>> > Post 4.3:
>> >
>> > Come up with a way to more generally enter these parameters (probably
>> just
>> > key/value pairs sent to the drivers).
>> >
>> > Have the drivers expose their feature set so the allocators can consider
>> > them more fully and throw an exception if there is not a sufficient
>> match.
>> >
>> >
>> > On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com> wrote:
>> >
>> >> My thinking is, for 4.2, while not ideal, we will need to put some
>> burden
>> >> on the admin to configure a Disk Offering in a way that makes sense.
>> For
>> >> example, if he wants to use storage QoS with supported Min and Max
>> values,
>> >> he'll have to put in a storage tag that references the SolidFire
>> primary
>> >> storage (plug-in). If he puts in a storage tag that doesn't, then he's
>> not
>> >> going to get the Min and Max feature. We could add help text to the
>> pop-up
>> >> dialog that's displayed when you click in the Min and Max text fields.
>> >>
>> >> Same idea for Wei's feature.
>> >>
>> >> Not idea, true...perhaps we can brainstorm on a more comprehensive
>> >> approach for post 4.2.
>> >>
>> >> Maybe in the future we could have the drivers advertise their
>> capabilities
>> >> and if the allocator feels a request is not being satisfied (say Min
>> was
>> >> entered, but it not's supported by any storage plug-in) it can throw an
>> >> exception.
>> >>
>> >>
>> >> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com> wrote:
>> >>
>> >>> Comments below in red.
>> >>>
>> >>> Thanks
>> >>>
>> >>>
>> >>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> Please see my comment in-line below.
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>
>> >>>>> Hi John,
>> >>>>>
>> >>>>> I've put comments below in red.
>> >>>>>
>> >>>>> Thanks!
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> First and foremost, we must ensure that these two features are
>> >>>> mutually
>> >>>>>> exclusive in 4.2. We don't want to find a configuration that
>> >>>> contains both
>> >>>>>> hypervisor and storage IOPS guarantees that leads to
>> non-deterministic
>> >>>>>> operations.
>> >>>>>
>> >>>>>
>> >>>>> Agreed
>> >>>>>
>> >>>>>
>> >>>>>> Restricting QoS expression to be either hypervisor or storage
>> oriented
>> >>>>>> solves the problem in short term. As I understand storage tags, we
>> >>>> have no
>> >>>>>> means of expressing this type of mutual exclusion. I wasn't
>> >>>> necessarily
>> >>>>>> intending that we implement this allocation model in 4.3, but
>> instead,
>> >>>>>> determine if this type model would be one we would want to support
>> in
>> >>>> the
>> >>>>>> future. If so, I would encourage us to ensure that the data model
>> and
>> >>>>>> current implementation would not preclude evolution in that
>> >>>> direction. My
>> >>>>>> view is that this type of allocation model is what user's expect of
>> >>>> "cloud"
>> >>>>>> systems -- selecting the best available resource set to fulfill a
>> >>>> set of
>> >>>>>> system requirements.
>> >>>>>>
>> >>>>>
>> >>>>> I believe we have meet your requirement here in that what we've
>> >>>> implemented
>> >>>>> should not make refinement difficult in the future. If we don't
>> modify
>> >>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
>> >>>> changes
>> >>>>> to enhance the current functioning of the system.
>> >>>>>
>> >>>>
>> >>>> Looking through both patches, I have to say that the aggregated
>> result
>> >>>> seems a bit confusing. There are six new attributes for throttled
>> I/O and
>> >>>> two for provisioned IOPS with no obvious grouping. My concern is not
>> >>>> technical, but rather, about maintainability. At minimum, Javadoc
>> should
>> >>>> be added explaining the two sets of attributes and their mutual
>> exclusion.
>> >>>>
>> >>>
>> >>> I agree: We need JavaDoc to explain them and their mutual exclusion.
>> >>>
>> >>>
>> >>>>
>> >>>> The other part that is interesting is that throttled I/O provides
>> both
>> >>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses a
>> >>>> range. In order to select the best available resource to fulfill a
>> QoS, we
>> >>>> would need to have the QoS expression normalized in terms of units
>> (IOPS or
>> >>>> bytes) and their expression (rate vs. range). If we want to achieve
>> a
>> >>>> model like I described, I think we would need to resolve this issue
>> in 4.2
>> >>>> to ensure a viable migration path.
>> >>>
>> >>>
>> >>> I think we're not likely to be able to normalize the input for 4.2.
>> Plus
>> >>> people probably want to input the data in terms they're familiar with
>> for
>> >>> the products in question.
>> >>>
>> >>> Ideally we would fix the way we do storage tagging and let the user
>> send
>> >>> key/value pairs to each vendor that could be selected due to a given
>> >>> storage tag. I'm still not sure that would solve it because what
>> happens if
>> >>> you change the storage tag of a given Primary Storage after having
>> created
>> >>> a Disk Offering?
>> >>>
>> >>> Basically storage tagging is kind of a mess and we should re-think it.
>> >>>
>> >>> Also, we need to have a way for the drivers to expose their supported
>> >>> feature sets so the allocators can make good choices.
>> >>>
>> >>>
>> >>>>
>> >>>>>
>> >>>>>>
>> >>>>>> As I think through the implications of these requirements and
>> reflect
>> >>>> on
>> >>>>>> the reviews, I don't understand why they haven't already impacted
>> the
>> >>>>>> allocators and planners. As it stands, the current provisioned
>> IOPS
>> >>>> has no
>> >>>>>> checks to ensure that the volumes are allocated to devices that
>> have
>> >>>>>> capacity to fulfill the requested QoS. Therefore, as I understand
>> the
>> >>>>>> current patch, we can overcommit storage resources -- potentially
>> >>>> causing
>> >>>>>> none of the QoS obligations from being fulfilled. It seems to me
>> >>>> that a
>> >>>>>> DataStore supporting provisioned IOPS should express the maximum
>> IOPS
>> >>>> which
>> >>>>>> it can support and some type of overcommitment factor. This
>> >>>> information
>> >>>>>> should be used by the storage allocators to determine the device
>> best
>> >>>> able
>> >>>>>> to support the resources needs of a volume. It seems that a
>> similar
>> >>>> set of
>> >>>>>> considerations would need to be added to the Hypervisor layer which
>> >>>>>> allocating a VM to a host to prevent oversubscription.
>> >>>>>>
>> >>>>>
>> >>>>> Yeah, for this first release, we just followed the path that was
>> >>>> previously
>> >>>>> established for other properties you see on dialogs in CS: Just
>> because
>> >>>>> they're there doesn't mean the vendor your VM is deployed to
>> supports
>> >>>> them.
>> >>>>> It is then up to the admin to make sure he inputs, say, a storage
>> tag
>> >>>> that
>> >>>>> confines the deployment only to storage that supports the selected
>> >>>>> features. This is not ideal, but it's kind of the way CloudStack
>> works
>> >>>>> today.
>> >>>>
>> >>>> I understand the tag functionality, and the need for the
>> administrator
>> >>>> to very carefully construct offerings. My concern is that we over
>> >>>> guarantee a resource's available IOPS. For the purposes of
>> illustration,
>> >>>> let's say we have a SolidFire, and the max IOPS for that device is
>> 100000.
>> >>>> We also know that we can safely oversubscribe by 50%. Therefore,
>> we
>> >>>> need to ensure that we don't allocate more than 150,000 guaranteed
>> IOPS
>> >>>> from that device. Intuitively, it seems like the DataStore
>> configuration
>> >>>> should have a max assignable IOPS and overcommitment factor. As we
>> >>>> allocate volumes and attach VMs, we need to ensure that guarantee
>> more IOPS
>> >>>> exceed the configured maximum for a DataStore. Does that make sense?
>> >>>>
>> >>>
>> >>> I think that's a good idea for a future enhancement. I'm not even
>> sure I
>> >>> can query the SAN to find out how many IOPS safely remain. I'd have
>> to get
>> >>> all of the min values for all of the volumes on the SAN and total
>> them up,
>> >>> I suppose, and subtract it from the total (user facing) supported
>> IOPS of
>> >>> the system.
>> >>>
>> >>>
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>>>
>> >>>>>> Another question occurs to me -- should we allow non-QoS resources
>> to
>> >>>> be
>> >>>>>> assigned to hosts/storage devices that ensure QoS? For provisioned
>> >>>> IOPS, I
>> >>>>>> think a side effect of the current implementation is SolidFire
>> volumes
>> >>>>>> always have a QoS. However, for hypervisor throttled I/O, it seems
>> >>>>>> entirely possible for non-QoS VMs to allocated side-by-side with
>> QoS
>> >>>> VMs.
>> >>>>>> In this scenario, a greedy, unbounded VM could potentially starve
>> out
>> >>>> the
>> >>>>>> other VMs on the host -- preventing the QoSes defined the
>> collocated
>> >>>> VMs
>> >>>>>> from being fulfilled.
>> >>>>>>
>> >>>>>
>> >>>>> You can make SolidFire volumes (inside and outside of CS) and not
>> >>>> specify
>> >>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>> defaults
>> >>>> we
>> >>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will
>> >>>> have
>> >>>>> your Mins met.
>> >>>>>
>> >>>>> It sounds like you're perhaps looking for a storage tags exclusions
>> >>>> list,
>> >>>>> which might be nice to have at some point (i.e. don't deploy my
>> volume
>> >>>> to
>> >>>>> storage with these following tags).
>> >>>>
>> >>>> I don't like the idea of a storage tags exclusion list as it would
>> >>>> complicate component assembly. It would require a storage plugin to
>> >>>> anticipate all of the possible component assemblies and determine the
>> >>>> invalid relationships. I prefer that drivers express their
>> capabilities
>> >>>> which can be matched to a set of requested requirements.
>> >>>>
>> >>>
>> >>> I'm not sure why a storage plug-in would care about inclusion or
>> >>> exclusion lists. It just needs to advertise its functionality in a
>> way the
>> >>> allocator understands.
>> >>>
>> >>>
>> >>>>
>> >>>>>
>> >>>>> I agree with your assessment of Hypervisor QoS. Since it only limits
>> >>>> IOPS,
>> >>>>> it does not solve the Noisy Neighbor problem. Only a system with
>> >>>> guaranteed
>> >>>>> minimum IOPS does this.
>> >>>>
>> >>>> As I said, for SolidFire, it sounds like this problem does not exist.
>> >>>> However, I am concerned with the more general case as we supported
>> more
>> >>>> devices with provisioned IOPS.
>> >>>>
>> >>>
>> >>> Post 4.2 we need to investigate a way to pass vendor-specific values
>> to
>> >>> drivers. Min and Max and pretty industry standard for provisioned
>> IOPS, but
>> >>> what if you break them out by read and write or do something else? We
>> need
>> >>> a more general mechanism.
>> >>>
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>>>
>> >>>>>> In my opinion, we need to ensure that hypervisor throttled I/O and
>> >>>>>> storage provisioned IOPS are mutually exclusive per volume.
>> >>>>>
>> >>>>>
>> >>>>> Agreed
>> >>>>>
>> >>>>>
>> >>>>>> We also need to understand the implications of these QoS
>> guarantees on
>> >>>>>> operation of the system to ensure that the underlying hardware
>> >>>> resources
>> >>>>>> can fulfill them. Given the time frame, we will likely be forced
>> to
>> >>>> make
>> >>>>>> compromises to achieve these goals, and refine the implementation
>> in
>> >>>> 4.3.
>> >>>>>>
>> >>>>>
>> >>>>> I agree, John. I also think you've come up with some great ideas for
>> >>>> 4.3. :)
>> >>>>>
>> >>>>>
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> Hi,
>> >>>>>>>
>> >>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
>> >>>> tags).
>> >>>>>> I
>> >>>>>>> believe John was just throwing out an idea that - in addition to
>> >>>> storage
>> >>>>>>> tags - we could look into these allocators (storage QoS being
>> >>>> preferred,
>> >>>>>>> then hypervisor QoS if storage QoS is not available, but
>> hypervisor
>> >>>> QoS
>> >>>>>> is).
>> >>>>>>>
>> >>>>>>> I think John's concern is that you can enter in values for Wei's
>> and
>> >>>> my
>> >>>>>>> feature that are not honored by other vendors (at least yet), so
>> he
>> >>>> was
>> >>>>>>> hoping - in addition to storage tags - for the allocators to
>> prefer
>> >>>> these
>> >>>>>>> vendors when these fields are filled in. As it stands today in
>> >>>>>> CloudStack,
>> >>>>>>> we already have this kind of an issue with other features (fields
>> in
>> >>>>>>> dialogs for features that not all vendors support). Perhaps post
>> 4.2
>> >>>> we
>> >>>>>>> could look into generic name/value pairs (this is how OpenStack
>> >>>> addresses
>> >>>>>>> the issue).
>> >>>>>>>
>> >>>>>>> Honestly, I think we're too late in the game (two weeks until code
>> >>>>>> freeze)
>> >>>>>>> to go too deeply down that path in 4.2.
>> >>>>>>>
>> >>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields
>> >>>> and my
>> >>>>>>> fields as is, make sure only one or the other feature has data
>> >>>> entered
>> >>>>>> for
>> >>>>>>> it (or neither), and call it good for 4.2.
>> >>>>>>>
>> >>>>>>> Then let's step back and look into a more general-purpose design
>> >>>> that can
>> >>>>>>> be applied throughout CloudStack where we have these situations.
>> >>>>>>>
>> >>>>>>> What do you think?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jburwell@basho.com
>> >
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> I just published my review @ https://reviews.apache.org/r/11479/
>> .
>> >>>>>>>>
>> >>>>>>>> I apologize for the delay,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> No problem, John.
>> >>>>>>>>>
>> >>>>>>>>> I still want to re-review it by myself before coming up with a
>> new
>> >>>>>> patch
>> >>>>>>>>> file.
>> >>>>>>>>>
>> >>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
>> in
>> >>>> and
>> >>>>>>>>> merge those into mine before generating a new patch file?
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> jburwell@basho.com
>> >>>>>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Mike,
>> >>>>>>>>>>
>> >>>>>>>>>> I just realized that I forgot to publish my review. I am
>> offline
>> >>>> ATM,
>> >>>>>>>>>> but I will publish it in the next couple of hours.
>> >>>>>>>>>>
>> >>>>>>>>>> Do you plan to update your the patch in Review Board?
>> >>>>>>>>>>
>> >>>>>>>>>> Sorry for the oversight,
>> >>>>>>>>>> -John
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> >>>>>>>>>> <mi...@solidfire.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
>> ),
>> >>>>>>>>>>>
>> >>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
>> >>>> wanted
>> >>>>>>>>>>> addressed.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
>> >>>> then
>> >>>>>>>> send
>> >>>>>>>>>> in
>> >>>>>>>>>>> another patch.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks for all the work on this everyone!
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Sure, that sounds good.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> >>>> ustcweizhou@gmail.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Hi Mike,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>> >>>> code,
>> >>>>>>>>>> except
>> >>>>>>>>>>>>> the cloudstack UI.
>> >>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
>> into
>> >>>>>>>> master
>> >>>>>>>>>>>>> this
>> >>>>>>>>>>>>> week, so that you can develop based on it.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hey John,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>> >>>> branch,
>> >>>>>> but
>> >>>>>>>> -
>> >>>>>>>>>> as
>> >>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>> >>>> SolidFire
>> >>>>>>>> branch
>> >>>>>>>>>>>>> into
>> >>>>>>>>>>>>>> the object_store branch before object_store goes into
>> master.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>> merge
>> >>>>>>>> strategy.
>> >>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> >>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>> >>>>>> disk_io_throttling,
>> >>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>> >>>>>> overlapping
>> >>>>>>>>>>>>>>> elements. I understand the dependencies between the
>> patches
>> >>>> to
>> >>>>>> be
>> >>>>>>>> as
>> >>>>>>>>>>>>>>> follows:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Am I correct that the device management aspects of
>> SolidFire
>> >>>> are
>> >>>>>>>>>>>>> additive
>> >>>>>>>>>>>>>>> to the object_store branch or there are circular
>> dependency
>> >>>>>> between
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>> >>>>>>>> determine
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> best approach to land the changes in master.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>> >>>> branch
>> >>>>>>>>>>>>> before
>> >>>>>>>>>>>>>>>> going into master, I am good with that.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>> >>>> merge
>> >>>>>> and
>> >>>>>>>>>>>>> we
>> >>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
>> the
>> >>>>>>>> storage
>> >>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>> plug-in,
>> >>>> which
>> >>>>>>>>>>>>>>> currently
>> >>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>> (although
>> >>>> it
>> >>>>>> does
>> >>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>> know
>> >>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
>> in
>> >>>> use))
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
>> url
>> >>>> field
>> >>>>>>>> (if
>> >>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
>> the
>> >>>>>>>>>>>>>>>>> storage_pool_details table.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
>> in
>> >>>>>>>>>>>>> question, we
>> >>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of
>> >>>> the
>> >>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
>> >>>> for
>> >>>>>> the
>> >>>>>>>>>>>>>> managed
>> >>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>> hypervisor
>> >>>> data
>> >>>>>>>>>>>>>>> structure is
>> >>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>> >>>> attach
>> >>>>>> the
>> >>>>>>>>>>>>>>> volume.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>> detach a
>> >>>>>>>> volume,
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>> >>>> structure
>> >>>>>> is
>> >>>>>>>>>>>>>>> removed.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
>> 4.2
>> >>>>>> unless
>> >>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
>> have
>> >>>> some
>> >>>>>>>>>>>>> idea,
>> >>>>>>>>>>>>>>>>> that'd be cool.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>> >>>>>>>> implementation
>> >>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>> field. I
>> >>>>>> think
>> >>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
>> the
>> >>>>>>>>>>>>> expected
>> >>>>>>>>>>>>>>> format
>> >>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
>> not
>> >>>>>>>>>>>>> returned on
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>> >>>>>>>> response.
>> >>>>>>>>>>>>>> Why
>> >>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
>> for
>> >>>>>>>> clients
>> >>>>>>>>>>>>> to
>> >>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> Offering-by-Disk
>> >>>>>>>> Offering
>> >>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>> >>>> able to
>> >>>>>>>>>>>>>> associate
>> >>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>> table.
>> >>>>>> Maybe
>> >>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>> could
>> >>>>>>>>>>>>>>>>>> go there?
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
>> in
>> >>>> the
>> >>>>>> GUI
>> >>>>>>>>>>>>> if
>> >>>>>>>>>>>>>>> it's
>> >>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>> >
>> >>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >> *â„¢*
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
That is the route I was trying to go with this, Simon.
I agree with what you're saying.
We cannot assume the SAN is dedicated to CS. That is why - if we have to do
this for 4.2 - I need to have a dedicated timer thread (or something) that
updates the total and consumed IOPS. If we just use the volumes created
from CS, it will often be wrong (if the SAN is used for other purposes).
I really don't think this is a 4.2 feature. It seems like feature
creep...and really late in the game.
On Fri, Jun 14, 2013 at 12:20 PM, Simon Weller <sw...@ena.com> wrote:
> I'd like to comment on this briefly.
>
>
>
> I think an assumption is being made that the SAN is being dedicated to a
> CS instance.
>
> My person opinion that this whole IOPS calculation is getting rather
> complicated, and could probably be much simpler than this. Over
> subscription is a fact of life on virtually all storage, and is really no
> different in concept than multiple virt instances on a single piece of
> hardware. All decent SANs offer many management options for the storage
> engineers to keep track of IOPS utilization, and plan for spindle
> augmentation as required.
> Is it really the job of CS to become yet another management layer on top
> of this?
>
> ----- Original Message -----
>
> From: "Mike Tutkowski" <mi...@solidfire.com>
> To: dev@cloudstack.apache.org
> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <ustcweizhou@gmail.com
> >
> Sent: Friday, June 14, 2013 1:00:26 PM
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> 1) We want number of IOPS currently supported by the SAN.
>
> 2) We want the number of IOPS that are committed (sum of min IOPS for each
> volume).
>
> We could do the following to keep track of IOPS:
>
> The plug-in could have a timer thread that goes off every, say, 1 minute.
>
> It could query the SAN for the number of nodes that make up the SAN and
> multiple this by 50,000. This is essentially the number of supported IOPS
> of the SAN.
>
> The next API call could be to get all of the volumes on the SAN. Iterate
> through them all and add up their min IOPS values. This is the number of
> IOPS the SAN is committed to.
>
> These two numbers can then be updated in the storage_pool table (a column
> for each value).
>
> The allocators can get these values as needed (and they would be as
> accurate as the last time the thread asked the SAN for this info).
>
> These two fields, the min IOPS of the volume to create, and the overcommit
> ratio of the plug-in would tell the allocator if it can select the given
> storage pool.
>
> What do you think?
>
>
> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
> > "As I mentioned previously, I am very reluctant for any feature to come
> > into master that can exhaust resources."
> >
> > Just wanted to mention that, worst case, the SAN would fail creation of
> > the volume before allowing a new volume to break the system.
> >
> >
> > On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> Hi John,
> >>
> >> Are you thinking we add a column on to the storage pool table,
> >> IOPS_Count, where we add and subtract committed IOPS?
> >>
> >> That is easy enough.
> >>
> >> How do you want to determine what the SAN is capable of supporting IOPS
> >> wise? Remember we're dealing with a dynamic SAN here...as you add
> storage
> >> nodes to the cluster, the number of IOPS increases. Do we have a thread
> we
> >> can use to query external devices like this SAN to update the supported
> >> number of IOPS?
> >>
> >> Also, how do you want to enforce the IOPS limit? Do we pass in an
> >> overcommit ration to the plug-in when it's created? We would need to
> store
> >> this in the storage_pool table, as well, I believe.
> >>
> >> We should also get Wei involved in this as his feature will need similar
> >> functionality.
> >>
> >> Also, we should do this FAST as we have only two weeks left and many of
> >> us will be out for several days for the CS Collab Conference.
> >>
> >> Thanks
> >>
> >>
> >> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
> >wrote:
> >>
> >>> Mike,
> >>>
> >>> Querying the SAN only indicates the number of IOPS currently in use.
> >>> The allocator needs to check the number of IOPS committed which is
> tracked
> >>> by CloudStack. For 4.2, we should be able to query the number of IOPS
> >>> committed to a DataStore, and determine whether or not the number
> requested
> >>> can be fulfilled by that device. It seems to be that a
> >>> DataStore#getCommittedIOPS() : Long method would be sufficient.
> >>> DataStore's that don't support provisioned IOPS would return null.
> >>>
> >>> As I mentioned previously, I am very reluctant for any feature to come
> >>> into master that can exhaust resources.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>> > Yeah, I'm not sure I could come up with anything near an accurate
> >>> > assessment of how many IOPS are currently available on the SAN (or
> >>> even a
> >>> > total number that are available for volumes). Not sure if there's yet
> >>> an
> >>> > API call for that.
> >>> >
> >>> > If I did know this number (total number of IOPS supported by the
> SAN),
> >>> we'd
> >>> > still have to keep track of the total number of volumes we've created
> >>> from
> >>> > CS on the SAN in terms of their IOPS. Also, if an admin issues an API
> >>> call
> >>> > directly to the SAN to tweak the number of IOPS on a given volume or
> >>> set of
> >>> > volumes (not supported from CS, but supported via the SolidFire API),
> >>> our
> >>> > numbers in CS would be off.
> >>> >
> >>> > I'm thinking verifying sufficient number of IOPS is a really good
> idea
> >>> for
> >>> > a future release.
> >>> >
> >>> > For 4.2 I think we should stick to having the allocator detect if
> >>> storage
> >>> > QoS is desired and if the storage pool in question supports that
> >>> feature.
> >>> >
> >>> > If you really are over provisioned on your SAN in terms of IOPS or
> >>> > capacity, the SAN can let the admin know in several different ways
> >>> (e-mail,
> >>> > SNMP, GUI).
> >>> >
> >>> >
> >>> > On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
> >>> wrote:
> >>> >
> >>> >> Mike,
> >>> >>
> >>> >> Please see my comments in-line below.
> >>> >>
> >>> >> Thanks,
> >>> >> -John
> >>> >>
> >>> >> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com>
> >>> >> wrote:
> >>> >>
> >>> >>> Comments below in red.
> >>> >>>
> >>> >>> Thanks
> >>> >>>
> >>> >>>
> >>> >>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
> >>> >> wrote:
> >>> >>>
> >>> >>>> Mike,
> >>> >>>>
> >>> >>>> Overall, I agree with the steps to below for 4.2. However, we may
> >>> want
> >>> >> to
> >>> >>>> throw an exception if we can not fulfill a requested QoS. If the
> >>> user
> >>> >> is
> >>> >>>> expecting that the hypervisor will provide a particular QoS, and
> >>> that is
> >>> >>>> not possible, it seems like we should inform them rather than
> >>> silently
> >>> >>>> ignoring their request.
> >>> >>>>
> >>> >>>
> >>> >>> Sure, that sounds reasonable.
> >>> >>>
> >>> >>> We'd have to come up with some way for the allocators to know about
> >>> the
> >>> >>> requested storage QoS and then query the candidate drivers.
> >>> >>>
> >>> >>> Any thoughts on how we might do that?
> >>> >>>
> >>> >>>
> >>> >>>>
> >>> >>>> To collect my thoughts from previous parts of the thread, I am
> >>> >>>> uncomfortable with the idea that the management server can
> >>> overcommit a
> >>> >>>> resource. You had mentioned querying the device for available
> IOPS.
> >>> >> While
> >>> >>>> that would be nice, it seems like we could fall back to a max IOPS
> >>> and
> >>> >>>> overcommit factor manually calculated and entered by the
> >>> >>>> administrator/operator. I think such threshold and allocation
> rails
> >>> >> should
> >>> >>>> be added for both provisioned IOPS and throttled I/O -- it is a
> >>> basic
> >>> >>>> feature of any cloud orchestration platform.
> >>> >>>>
> >>> >>>
> >>> >>> Are you thinking this ability would make it into 4.2? Just curious
> >>> what
> >>> >>> release we're talking about here. For the SolidFire SAN, you might
> >>> have,
> >>> >>> say, four separate storage nodes to start (200,000 IOPS) and then
> >>> later
> >>> >> add
> >>> >>> a new node (now you're at 250,000 IOPS). CS would have to have a
> way
> >>> to
> >>> >>> know that the number of supported IOPS has increased.
> >>> >>
> >>> >> Yes, I think we need some *basic*/conservative rails in 4.2. For
> >>> example,
> >>> >> we may only support expanding capacity in 4.2, and not handle any
> >>> >> re-balance scenarios -- node failure, addition, etc. Extrapolating
> >>> a
> >>> >> bit, the throttled I/O enhancement seems like it needs a similar set
> >>> of
> >>> >> rails defined per host.
> >>> >>
> >>> >>>
> >>> >>>
> >>> >>>>
> >>> >>>> For 4.3, I don't like the idea that a QoS would be expressed in a
> >>> >>>> implementation specific manner. I think we need to arrive at a
> >>> general
> >>> >>>> model that can be exploited by the allocators and planners. We
> >>> should
> >>> >>>> restrict implementation specific key-value pairs (call them
> details,
> >>> >>>> extended data, whatever) to information that is unique to the
> >>> driver and
> >>> >>>> would provide no useful information to the management server's
> >>> >>>> orchestration functions. A resource QoS does not seem to fit those
> >>> >>>> criteria.
> >>> >>>>
> >>> >>>
> >>> >>> I wonder if this would be a good discussion topic for Sunday's CS
> >>> Collab
> >>> >>> Conf hack day that Joe just sent out an e-mail about?
> >>> >>
> >>> >> Yes, it would -- I will put something in the wiki topic. It will
> >>> also be
> >>> >> part of my talk on Monday -- How to Run from Zombie which include
> >>> some of
> >>> >> my opinions on the topic.
> >>> >>
> >>> >>>
> >>> >>>
> >>> >>>>
> >>> >>>> Thanks,
> >>> >>>> -John
> >>> >>>>
> >>> >>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>> >> mike.tutkowski@solidfire.com>
> >>> >>>> wrote:
> >>> >>>>
> >>> >>>>> So, here's my suggestion for 4.2:
> >>> >>>>>
> >>> >>>>> Accept the values as they are currently required (four new fields
> >>> for
> >>> >>>> Wei's
> >>> >>>>> feature or two new fields for mine).
> >>> >>>>>
> >>> >>>>> The Add Disk Offering dialog needs three new radio buttons:
> >>> >>>>>
> >>> >>>>> 1) No QoS
> >>> >>>>>
> >>> >>>>> 2) Hypervisor QoS
> >>> >>>>>
> >>> >>>>> 3) Storage Qos
> >>> >>>>>
> >>> >>>>> The admin needs to specify storage tags that only map to storage
> >>> that
> >>> >>>>> supports Storage QoS.
> >>> >>>>>
> >>> >>>>> The admin needs to be aware for Hypervisor QoS that unless all
> >>> >>>> hypervisors
> >>> >>>>> in use support the new fields, they may not be enforced.
> >>> >>>>>
> >>> >>>>> Post 4.3:
> >>> >>>>>
> >>> >>>>> Come up with a way to more generally enter these parameters
> >>> (probably
> >>> >>>> just
> >>> >>>>> key/value pairs sent to the drivers).
> >>> >>>>>
> >>> >>>>> Have the drivers expose their feature set so the allocators can
> >>> >> consider
> >>> >>>>> them more fully and throw an exception if there is not a
> sufficient
> >>> >>>> match.
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>> >>>>>
> >>> >>>>>> My thinking is, for 4.2, while not ideal, we will need to put
> some
> >>> >>>> burden
> >>> >>>>>> on the admin to configure a Disk Offering in a way that makes
> >>> sense.
> >>> >> For
> >>> >>>>>> example, if he wants to use storage QoS with supported Min and
> Max
> >>> >>>> values,
> >>> >>>>>> he'll have to put in a storage tag that references the SolidFire
> >>> >> primary
> >>> >>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
> then
> >>> he's
> >>> >>>> not
> >>> >>>>>> going to get the Min and Max feature. We could add help text to
> >>> the
> >>> >>>> pop-up
> >>> >>>>>> dialog that's displayed when you click in the Min and Max text
> >>> fields.
> >>> >>>>>>
> >>> >>>>>> Same idea for Wei's feature.
> >>> >>>>>>
> >>> >>>>>> Not idea, true...perhaps we can brainstorm on a more
> comprehensive
> >>> >>>>>> approach for post 4.2.
> >>> >>>>>>
> >>> >>>>>> Maybe in the future we could have the drivers advertise their
> >>> >>>> capabilities
> >>> >>>>>> and if the allocator feels a request is not being satisfied (say
> >>> Min
> >>> >> was
> >>> >>>>>> entered, but it not's supported by any storage plug-in) it can
> >>> throw
> >>> >> an
> >>> >>>>>> exception.
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>> >>>>>>
> >>> >>>>>>> Comments below in red.
> >>> >>>>>>>
> >>> >>>>>>> Thanks
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>> jburwell@basho.com>
> >>> >>>> wrote:
> >>> >>>>>>>
> >>> >>>>>>>> Mike,
> >>> >>>>>>>>
> >>> >>>>>>>> Please see my comment in-line below.
> >>> >>>>>>>>
> >>> >>>>>>>> Thanks,
> >>> >>>>>>>> -John
> >>> >>>>>>>>
> >>> >>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>> >>>>>>>>
> >>> >>>>>>>>> Hi John,
> >>> >>>>>>>>>
> >>> >>>>>>>>> I've put comments below in red.
> >>> >>>>>>>>>
> >>> >>>>>>>>> Thanks!
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>> jburwell@basho.com
> >>> >>>
> >>> >>>>>>>> wrote:
> >>> >>>>>>>>>
> >>> >>>>>>>>>> Mike,
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> First and foremost, we must ensure that these two features
> are
> >>> >>>>>>>> mutually
> >>> >>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
> >>> >>>>>>>> contains both
> >>> >>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >>> >>>> non-deterministic
> >>> >>>>>>>>>> operations.
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> Agreed
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>> Restricting QoS expression to be either hypervisor or
> storage
> >>> >>>> oriented
> >>> >>>>>>>>>> solves the problem in short term. As I understand storage
> >>> tags,
> >>> >> we
> >>> >>>>>>>> have no
> >>> >>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
> >>> >>>>>>>> necessarily
> >>> >>>>>>>>>> intending that we implement this allocation model in 4.3,
> but
> >>> >>>> instead,
> >>> >>>>>>>>>> determine if this type model would be one we would want to
> >>> support
> >>> >>>> in
> >>> >>>>>>>> the
> >>> >>>>>>>>>> future. If so, I would encourage us to ensure that the data
> >>> model
> >>> >>>> and
> >>> >>>>>>>>>> current implementation would not preclude evolution in that
> >>> >>>>>>>> direction. My
> >>> >>>>>>>>>> view is that this type of allocation model is what user's
> >>> expect
> >>> >> of
> >>> >>>>>>>> "cloud"
> >>> >>>>>>>>>> systems -- selecting the best available resource set to
> >>> fulfill a
> >>> >>>>>>>> set of
> >>> >>>>>>>>>> system requirements.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> I believe we have meet your requirement here in that what
> we've
> >>> >>>>>>>> implemented
> >>> >>>>>>>>> should not make refinement difficult in the future. If we
> don't
> >>> >>>> modify
> >>> >>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
> >>> simple
> >>> >>>>>>>> changes
> >>> >>>>>>>>> to enhance the current functioning of the system.
> >>> >>>>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>> Looking through both patches, I have to say that the
> aggregated
> >>> >> result
> >>> >>>>>>>> seems a bit confusing. There are six new attributes for
> >>> throttled
> >>> >>>> I/O and
> >>> >>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
> >>> is
> >>> >> not
> >>> >>>>>>>> technical, but rather, about maintainability. At minimum,
> >>> Javadoc
> >>> >>>> should
> >>> >>>>>>>> be added explaining the two sets of attributes and their
> mutual
> >>> >>>> exclusion.
> >>> >>>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> I agree: We need JavaDoc to explain them and their mutual
> >>> exclusion.
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>> The other part that is interesting is that throttled I/O
> >>> provides
> >>> >> both
> >>> >>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
> >>> uses
> >>> >> a
> >>> >>>>>>>> range. In order to select the best available resource to
> >>> fulfill a
> >>> >>>> QoS, we
> >>> >>>>>>>> would need to have the QoS expression normalized in terms of
> >>> units
> >>> >>>> (IOPS or
> >>> >>>>>>>> bytes) and their expression (rate vs. range). If we want to
> >>> >> achieve a
> >>> >>>>>>>> model like I described, I think we would need to resolve this
> >>> issue
> >>> >>>> in 4.2
> >>> >>>>>>>> to ensure a viable migration path.
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> I think we're not likely to be able to normalize the input for
> >>> 4.2.
> >>> >>>> Plus
> >>> >>>>>>> people probably want to input the data in terms they're
> familiar
> >>> with
> >>> >>>> for
> >>> >>>>>>> the products in question.
> >>> >>>>>>>
> >>> >>>>>>> Ideally we would fix the way we do storage tagging and let the
> >>> user
> >>> >>>> send
> >>> >>>>>>> key/value pairs to each vendor that could be selected due to a
> >>> given
> >>> >>>>>>> storage tag. I'm still not sure that would solve it because
> what
> >>> >>>> happens if
> >>> >>>>>>> you change the storage tag of a given Primary Storage after
> >>> having
> >>> >>>> created
> >>> >>>>>>> a Disk Offering?
> >>> >>>>>>>
> >>> >>>>>>> Basically storage tagging is kind of a mess and we should
> >>> re-think
> >>> >> it.
> >>> >>>>>>>
> >>> >>>>>>> Also, we need to have a way for the drivers to expose their
> >>> supported
> >>> >>>>>>> feature sets so the allocators can make good choices.
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> As I think through the implications of these requirements
> and
> >>> >>>> reflect
> >>> >>>>>>>> on
> >>> >>>>>>>>>> the reviews, I don't understand why they haven't already
> >>> impacted
> >>> >>>> the
> >>> >>>>>>>>>> allocators and planners. As it stands, the current
> >>> provisioned
> >>> >> IOPS
> >>> >>>>>>>> has no
> >>> >>>>>>>>>> checks to ensure that the volumes are allocated to devices
> >>> that
> >>> >> have
> >>> >>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
> >>> understand
> >>> >>>> the
> >>> >>>>>>>>>> current patch, we can overcommit storage resources --
> >>> potentially
> >>> >>>>>>>> causing
> >>> >>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
> >>> to me
> >>> >>>>>>>> that a
> >>> >>>>>>>>>> DataStore supporting provisioned IOPS should express the
> >>> maximum
> >>> >>>> IOPS
> >>> >>>>>>>> which
> >>> >>>>>>>>>> it can support and some type of overcommitment factor. This
> >>> >>>>>>>> information
> >>> >>>>>>>>>> should be used by the storage allocators to determine the
> >>> device
> >>> >>>> best
> >>> >>>>>>>> able
> >>> >>>>>>>>>> to support the resources needs of a volume. It seems that a
> >>> >> similar
> >>> >>>>>>>> set of
> >>> >>>>>>>>>> considerations would need to be added to the Hypervisor
> layer
> >>> >> which
> >>> >>>>>>>>>> allocating a VM to a host to prevent oversubscription.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> Yeah, for this first release, we just followed the path that
> >>> was
> >>> >>>>>>>> previously
> >>> >>>>>>>>> established for other properties you see on dialogs in CS:
> Just
> >>> >>>> because
> >>> >>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
> >>> >> supports
> >>> >>>>>>>> them.
> >>> >>>>>>>>> It is then up to the admin to make sure he inputs, say, a
> >>> storage
> >>> >> tag
> >>> >>>>>>>> that
> >>> >>>>>>>>> confines the deployment only to storage that supports the
> >>> selected
> >>> >>>>>>>>> features. This is not ideal, but it's kind of the way
> >>> CloudStack
> >>> >>>> works
> >>> >>>>>>>>> today.
> >>> >>>>>>>>
> >>> >>>>>>>> I understand the tag functionality, and the need for the
> >>> >> administrator
> >>> >>>>>>>> to very carefully construct offerings. My concern is that we
> >>> over
> >>> >>>>>>>> guarantee a resource's available IOPS. For the purposes of
> >>> >>>> illustration,
> >>> >>>>>>>> let's say we have a SolidFire, and the max IOPS for that
> device
> >>> is
> >>> >>>> 100000.
> >>> >>>>>>>> We also know that we can safely oversubscribe by 50%.
> >>> Therefore,
> >>> >> we
> >>> >>>>>>>> need to ensure that we don't allocate more than 150,000
> >>> guaranteed
> >>> >>>> IOPS
> >>> >>>>>>>> from that device. Intuitively, it seems like the DataStore
> >>> >>>> configuration
> >>> >>>>>>>> should have a max assignable IOPS and overcommitment factor.
> >>> As we
> >>> >>>>>>>> allocate volumes and attach VMs, we need to ensure that
> >>> guarantee
> >>> >>>> more IOPS
> >>> >>>>>>>> exceed the configured maximum for a DataStore. Does that make
> >>> >> sense?
> >>> >>>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> I think that's a good idea for a future enhancement. I'm not
> even
> >>> >> sure
> >>> >>>> I
> >>> >>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
> >>> have
> >>> >> to
> >>> >>>> get
> >>> >>>>>>> all of the min values for all of the volumes on the SAN and
> total
> >>> >> them
> >>> >>>> up,
> >>> >>>>>>> I suppose, and subtract it from the total (user facing)
> supported
> >>> >> IOPS
> >>> >>>> of
> >>> >>>>>>> the system.
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> Another question occurs to me -- should we allow non-QoS
> >>> resources
> >>> >>>> to
> >>> >>>>>>>> be
> >>> >>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
> >>> >> provisioned
> >>> >>>>>>>> IOPS, I
> >>> >>>>>>>>>> think a side effect of the current implementation is
> SolidFire
> >>> >>>> volumes
> >>> >>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
> >>> >> seems
> >>> >>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
> >>> with
> >>> >> QoS
> >>> >>>>>>>> VMs.
> >>> >>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
> >>> starve
> >>> >>>> out
> >>> >>>>>>>> the
> >>> >>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
> >>> >> collocated
> >>> >>>>>>>> VMs
> >>> >>>>>>>>>> from being fulfilled.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
> >>> not
> >>> >>>>>>>> specify
> >>> >>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
> >>> >>>> defaults
> >>> >>>>>>>> we
> >>> >>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN,
> you
> >>> will
> >>> >>>>>>>> have
> >>> >>>>>>>>> your Mins met.
> >>> >>>>>>>>>
> >>> >>>>>>>>> It sounds like you're perhaps looking for a storage tags
> >>> exclusions
> >>> >>>>>>>> list,
> >>> >>>>>>>>> which might be nice to have at some point (i.e. don't deploy
> my
> >>> >>>> volume
> >>> >>>>>>>> to
> >>> >>>>>>>>> storage with these following tags).
> >>> >>>>>>>>
> >>> >>>>>>>> I don't like the idea of a storage tags exclusion list as it
> >>> would
> >>> >>>>>>>> complicate component assembly. It would require a storage
> >>> plugin to
> >>> >>>>>>>> anticipate all of the possible component assemblies and
> >>> determine
> >>> >> the
> >>> >>>>>>>> invalid relationships. I prefer that drivers express their
> >>> >>>> capabilities
> >>> >>>>>>>> which can be matched to a set of requested requirements.
> >>> >>>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> I'm not sure why a storage plug-in would care about inclusion
> or
> >>> >>>>>>> exclusion lists. It just needs to advertise its functionality
> in
> >>> a
> >>> >> way
> >>> >>>> the
> >>> >>>>>>> allocator understands.
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
> >>> >> limits
> >>> >>>>>>>> IOPS,
> >>> >>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
> >>> with
> >>> >>>>>>>> guaranteed
> >>> >>>>>>>>> minimum IOPS does this.
> >>> >>>>>>>>
> >>> >>>>>>>> As I said, for SolidFire, it sounds like this problem does not
> >>> >> exist.
> >>> >>>>>>>> However, I am concerned with the more general case as we
> >>> supported
> >>> >>>> more
> >>> >>>>>>>> devices with provisioned IOPS.
> >>> >>>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
> >>> values
> >>> >> to
> >>> >>>>>>> drivers. Min and Max and pretty industry standard for
> provisioned
> >>> >>>> IOPS, but
> >>> >>>>>>> what if you break them out by read and write or do something
> >>> else? We
> >>> >>>> need
> >>> >>>>>>> a more general mechanism.
> >>> >>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
> >>> I/O
> >>> >> and
> >>> >>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> Agreed
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>> We also need to understand the implications of these QoS
> >>> >> guarantees
> >>> >>>> on
> >>> >>>>>>>>>> operation of the system to ensure that the underlying
> hardware
> >>> >>>>>>>> resources
> >>> >>>>>>>>>> can fulfill them. Given the time frame, we will likely be
> >>> forced
> >>> >> to
> >>> >>>>>>>> make
> >>> >>>>>>>>>> compromises to achieve these goals, and refine the
> >>> implementation
> >>> >> in
> >>> >>>>>>>> 4.3.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> I agree, John. I also think you've come up with some great
> >>> ideas
> >>> >> for
> >>> >>>>>>>> 4.3. :)
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> Thanks,
> >>> >>>>>>>>>> -John
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>> >>>>>>>> mike.tutkowski@solidfire.com>
> >>> >>>>>>>>>> wrote:
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>> Hi,
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
> >>> storage
> >>> >>>>>>>> tags).
> >>> >>>>>>>>>> I
> >>> >>>>>>>>>>> believe John was just throwing out an idea that - in
> >>> addition to
> >>> >>>>>>>> storage
> >>> >>>>>>>>>>> tags - we could look into these allocators (storage QoS
> being
> >>> >>>>>>>> preferred,
> >>> >>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
> >>> >> hypervisor
> >>> >>>>>>>> QoS
> >>> >>>>>>>>>> is).
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> I think John's concern is that you can enter in values for
> >>> Wei's
> >>> >>>> and
> >>> >>>>>>>> my
> >>> >>>>>>>>>>> feature that are not honored by other vendors (at least
> >>> yet), so
> >>> >> he
> >>> >>>>>>>> was
> >>> >>>>>>>>>>> hoping - in addition to storage tags - for the allocators
> to
> >>> >> prefer
> >>> >>>>>>>> these
> >>> >>>>>>>>>>> vendors when these fields are filled in. As it stands today
> >>> in
> >>> >>>>>>>>>> CloudStack,
> >>> >>>>>>>>>>> we already have this kind of an issue with other features
> >>> (fields
> >>> >>>> in
> >>> >>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
> >>> post
> >>> >>>> 4.2
> >>> >>>>>>>> we
> >>> >>>>>>>>>>> could look into generic name/value pairs (this is how
> >>> OpenStack
> >>> >>>>>>>> addresses
> >>> >>>>>>>>>>> the issue).
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
> until
> >>> >> code
> >>> >>>>>>>>>> freeze)
> >>> >>>>>>>>>>> to go too deeply down that path in 4.2.
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
> >>> fields
> >>> >>>>>>>> and my
> >>> >>>>>>>>>>> fields as is, make sure only one or the other feature has
> >>> data
> >>> >>>>>>>> entered
> >>> >>>>>>>>>> for
> >>> >>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> Then let's step back and look into a more general-purpose
> >>> design
> >>> >>>>>>>> that can
> >>> >>>>>>>>>>> be applied throughout CloudStack where we have these
> >>> situations.
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> What do you think?
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>> >> jburwell@basho.com>
> >>> >>>>>>>>>> wrote:
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>> Mike,
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> I just published my review @
> >>> >> https://reviews.apache.org/r/11479/.
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> I apologize for the delay,
> >>> >>>>>>>>>>>> -John
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>> >>>>>>>>>>>> wrote:
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>> No problem, John.
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>> I still want to re-review it by myself before coming up
> >>> with a
> >>> >>>> new
> >>> >>>>>>>>>> patch
> >>> >>>>>>>>>>>>> file.
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
> >>> checked
> >>> >>>> in
> >>> >>>>>>>> and
> >>> >>>>>>>>>>>>> merge those into mine before generating a new patch file?
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>> >>>> jburwell@basho.com
> >>> >>>>>>>>>
> >>> >>>>>>>>>>>> wrote:
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>> Mike,
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> >>> >>>> offline
> >>> >>>>>>>> ATM,
> >>> >>>>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>> Sorry for the oversight,
> >>> >>>>>>>>>>>>>> -John
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>> >>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
> >>> this :)
> >>> >>>> ),
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
> >>> areas
> >>> >> we
> >>> >>>>>>>> wanted
> >>> >>>>>>>>>>>>>>> addressed.
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> >>> >> afternoon,
> >>> >>>>>>>> then
> >>> >>>>>>>>>>>> send
> >>> >>>>>>>>>>>>>> in
> >>> >>>>>>>>>>>>>>> another patch.
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>> Sure, that sounds good.
> >>> >>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>> >>>>>>>> ustcweizhou@gmail.com>
> >>> >>>>>>>>>>>>>> wrote:
> >>> >>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>> Hi Mike,
> >>> >>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
> in
> >>> Java
> >>> >>>>>>>> code,
> >>> >>>>>>>>>>>>>> except
> >>> >>>>>>>>>>>>>>>>> the cloudstack UI.
> >>> >>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
> >>> branch
> >>> >>>> into
> >>> >>>>>>>>>>>> master
> >>> >>>>>>>>>>>>>>>>> this
> >>> >>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>> >>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>> -Wei
> >>> >>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >>> >
> >>> >>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>> Hey John,
> >>> >>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>> object_store
> >>> >>>>>>>> branch,
> >>> >>>>>>>>>> but
> >>> >>>>>>>>>>>> -
> >>> >>>>>>>>>>>>>> as
> >>> >>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
> the
> >>> >>>>>>>> SolidFire
> >>> >>>>>>>>>>>> branch
> >>> >>>>>>>>>>>>>>>>> into
> >>> >>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
> into
> >>> >>>> master.
> >>> >>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
> this
> >>> >> merge
> >>> >>>>>>>>>>>> strategy.
> >>> >>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>> >>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>> >>>>>>>>>> jburwell@basho.com>
> >>> >>>>>>>>>>>>>>>>> wrote:
> >>> >>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>> Mike,
> >>> >>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>> >>>>>>>>>> disk_io_throttling,
> >>> >>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
> >>> of
> >>> >>>>>>>>>> overlapping
> >>> >>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> >>> >>>> patches
> >>> >>>>>>>> to
> >>> >>>>>>>>>> be
> >>> >>>>>>>>>>>> as
> >>> >>>>>>>>>>>>>>>>>>> follows:
> >>> >>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>> >>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> >>> >>>> SolidFire
> >>> >>>>>>>> are
> >>> >>>>>>>>>>>>>>>>> additive
> >>> >>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> >>> >> dependency
> >>> >>>>>>>>>> between
> >>> >>>>>>>>>>>>>>>>> the
> >>> >>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
> >>> we
> >>> >> can
> >>> >>>>>>>>>>>> determine
> >>> >>>>>>>>>>>>>>>>> the
> >>> >>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>> >>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>> Thanks,
> >>> >>>>>>>>>>>>>>>>>>> -John
> >>> >>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>> >>>>>>>>>>>>>>>>>>> wrote:
> >>> >>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
> >>> into
> >>> >> his
> >>> >>>>>>>> branch
> >>> >>>>>>>>>>>>>>>>> before
> >>> >>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>> >>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
> >>> after his
> >>> >>>>>>>> merge
> >>> >>>>>>>>>> and
> >>> >>>>>>>>>>>>>>>>> we
> >>> >>>>>>>>>>>>>>>>>> can
> >>> >>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>> >>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>> >>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
> >>> code in
> >>> >>>> the
> >>> >>>>>>>>>>>> storage
> >>> >>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> >>> >> plug-in,
> >>> >>>>>>>> which
> >>> >>>>>>>>>>>>>>>>>>> currently
> >>> >>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> >>> >> (although
> >>> >>>>>>>> it
> >>> >>>>>>>>>> does
> >>> >>>>>>>>>>>>>>>>> not
> >>> >>>>>>>>>>>>>>>>>>> know
> >>> >>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
> >>> actually
> >>> >> in
> >>> >>>>>>>> use))
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
> >>> the
> >>> >> url
> >>> >>>>>>>> field
> >>> >>>>>>>>>>>> (if
> >>> >>>>>>>>>>>>>>>>>> not
> >>> >>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
> stored
> >>> in
> >>> >> the
> >>> >>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
> >>> hypervisor
> >>> >> in
> >>> >>>>>>>>>>>>>>>>> question, we
> >>> >>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
> >>> place
> >>> >> of
> >>> >>>>>>>> the
> >>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> >>> >> checks
> >>> >>>>>>>> for
> >>> >>>>>>>>>> the
> >>> >>>>>>>>>>>>>>>>>> managed
> >>> >>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> >>> >> hypervisor
> >>> >>>>>>>> data
> >>> >>>>>>>>>>>>>>>>>>> structure is
> >>> >>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
> >>> executes to
> >>> >>>>>>>> attach
> >>> >>>>>>>>>> the
> >>> >>>>>>>>>>>>>>>>>>> volume.
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
> to
> >>> >>>> detach a
> >>> >>>>>>>>>>>> volume,
> >>> >>>>>>>>>>>>>>>>>> the
> >>> >>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
> data
> >>> >>>>>>>> structure
> >>> >>>>>>>>>> is
> >>> >>>>>>>>>>>>>>>>>>> removed.
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
> IOPS
> >>> in
> >>> >> 4.2
> >>> >>>>>>>>>> unless
> >>> >>>>>>>>>>>>>>>>> it is
> >>> >>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
> If
> >>> we
> >>> >>>> have
> >>> >>>>>>>> some
> >>> >>>>>>>>>>>>>>>>> idea,
> >>> >>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> Thanks!
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
> >>> avoiding
> >>> >>>>>>>>>>>> implementation
> >>> >>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> >>> >> field.
> >>> >>>> I
> >>> >>>>>>>>>> think
> >>> >>>>>>>>>>>>>>>>> we
> >>> >>>>>>>>>>>>>>>>>>> should
> >>> >>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
> >>> regarding
> >>> >>>> the
> >>> >>>>>>>>>>>>>>>>> expected
> >>> >>>>>>>>>>>>>>>>>>> format
> >>> >>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
> >>> are
> >>> >>>> not
> >>> >>>>>>>>>>>>>>>>> returned on
> >>> >>>>>>>>>>>>>>>>>>> the
> >>> >>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >>> >> listStoragePool
> >>> >>>>>>>>>>>> response.
> >>> >>>>>>>>>>>>>>>>>> Why
> >>> >>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
> >>> useful
> >>> >>>> for
> >>> >>>>>>>>>>>> clients
> >>> >>>>>>>>>>>>>>>>> to
> >>> >>>>>>>>>>>>>>>>>> be
> >>> >>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
> >>> field."
> >>> >>>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
> >>> here.
> >>> >>>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>> >>>> Offering-by-Disk
> >>> >>>>>>>>>>>> Offering
> >>> >>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
> >>> to be
> >>> >>>>>>>> able to
> >>> >>>>>>>>>>>>>>>>>> associate
> >>> >>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>> disk_offering_details
> >>> >>>> table.
> >>> >>>>>>>>>> Maybe
> >>> >>>>>>>>>>>>>>>>> it
> >>> >>>>>>>>>>>>>>>>>>> could
> >>> >>>>>>>>>>>>>>>>>>>>>> go there?
> >>> >>>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
> >>> IOPS
> >>> >> in
> >>> >>>>>>>> the
> >>> >>>>>>>>>> GUI
> >>> >>>>>>>>>>>>>>>>> if
> >>> >>>>>>>>>>>>>>>>>>> it's
> >>> >>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
> the
> >>> DB.
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>> --
> >>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >
> >>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>> >>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>>> --
> >>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>> >>>>>>>>>>>>>>>>>>>> cloud<
> >>> >> http://solidfire.com/solution/overview/?video=play>
> >>> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>> >>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>>> --
> >>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>> >>>>>>>>>>>>>>>>>> cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>> >>>>>>>>>>>>>>>>>> *â„¢*
> >>> >>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>> --
> >>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>> >>>>>>>>>>>>>>>> *â„¢*
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>>> --
> >>> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>> >>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>> >
> >>> >>>>>>>>>>>>>>> *â„¢*
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>> --
> >>> >>>>>>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>>>>>> Advancing the way the world uses the
> >>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>> >>>>>>>>>>>>> *â„¢*
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> --
> >>> >>>>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>>>> Advancing the way the world uses the
> >>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> >>>>>>>>>>> *â„¢*
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> --
> >>> >>>>>>>>> *Mike Tutkowski*
> >>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>>>> o: 303.746.7302
> >>> >>>>>>>>> Advancing the way the world uses the
> >>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> >>>>>>>>> *â„¢*
> >>> >>>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> --
> >>> >>>>>>> *Mike Tutkowski*
> >>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>>> o: 303.746.7302
> >>> >>>>>>> Advancing the way the world uses the cloud<
> >>> >>>> http://solidfire.com/solution/overview/?video=play>
> >>> >>>>>>> *â„¢*
> >>> >>>>>>>
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> --
> >>> >>>>>> *Mike Tutkowski*
> >>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>>> o: 303.746.7302
> >>> >>>>>> Advancing the way the world uses the cloud<
> >>> >>>> http://solidfire.com/solution/overview/?video=play>
> >>> >>>>>> *â„¢*
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>> *Mike Tutkowski*
> >>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>>>> e: mike.tutkowski@solidfire.com
> >>> >>>>> o: 303.746.7302
> >>> >>>>> Advancing the way the world uses the
> >>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> >>>>> *â„¢*
> >>> >>>>
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> *Mike Tutkowski*
> >>> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> >>> e: mike.tutkowski@solidfire.com
> >>> >>> o: 303.746.7302
> >>> >>> Advancing the way the world uses the
> >>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> >>> *â„¢*
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>> > --
> >>> > *Mike Tutkowski*
> >>> > *Senior CloudStack Developer, SolidFire Inc.*
> >>> > e: mike.tutkowski@solidfire.com
> >>> > o: 303.746.7302
> >>> > Advancing the way the world uses the
> >>> > cloud<http://solidfire.com/solution/overview/?video=play>
> >>> > *â„¢*
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I've started in on making Wei's feature and mine mutually exclusive.
We had talked about using radio buttons to do this in the GUI, but I don't
see radio buttons in use at present in the CS GUI.
Instead, I'm following the current paradigm in the GUI where a combo box
presents options to accomplish this. If you select the empty option, you
get neither Wei's fields nor mine. If you select 'hypervisor', you get
Wei's fields. If you select 'storage', you get mine. The combo box has the
name 'QoS Type' and has a tool tip.
This technique is working well and fits in nicely with how a similar
selection process happens on the Add Volume dialog via a combo box.
I don't have all of the code written at present, but it should be done
tomorrow (well...technically later today).
Talk to you later!
On Tue, Jun 18, 2013 at 6:05 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> OK, I've gone ahead and updated Review Board with a new diff.
>
> Please check it out and let me know what you think.
>
> Thanks!
>
>
> On Mon, Jun 17, 2013 at 2:52 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Sounds good
>>
>> I have a little bit of clean-up work to do and then I must figure out how
>> to generate a patch (since I was using merge rather than rebase during
>> development), so it will probably be tomorrow.
>>
>>
>> On Mon, Jun 17, 2013 at 2:50 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> Great news. Let me know when Review Board is updated, and I will
>>> perform a second round review.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 17, 2013, at 4:48 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> > FYI: I added the IOPS-capacity parameter and related code over the
>>> weekend.
>>> >
>>> > The final bit of work for me comes when Wei's code is merged into
>>> master
>>> > and I pull it down.
>>> >
>>> > At that point, I need to add the GUI and API logic to support mutual
>>> > exclusion of our features.
>>> >
>>> >
>>> > On Sat, Jun 15, 2013 at 11:12 AM, Chip Childers
>>> > <ch...@sungard.com>wrote:
>>> >
>>> >> On Fri, Jun 14, 2013 at 05:56:27PM -0400, John Burwell wrote:
>>> >>> Mike,
>>> >>>
>>> >>> Awesome. +1.
>>> >>>
>>> >>> Thanks for your patience with the back and forth,
>>> >>> -John
>>> >>
>>> >> +1 - this now looks like a great approach.
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
OK, I've gone ahead and updated Review Board with a new diff.
Please check it out and let me know what you think.
Thanks!
On Mon, Jun 17, 2013 at 2:52 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Sounds good
>
> I have a little bit of clean-up work to do and then I must figure out how
> to generate a patch (since I was using merge rather than rebase during
> development), so it will probably be tomorrow.
>
>
> On Mon, Jun 17, 2013 at 2:50 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Great news. Let me know when Review Board is updated, and I will perform
>> a second round review.
>>
>> Thanks,
>> -John
>>
>> On Jun 17, 2013, at 4:48 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > FYI: I added the IOPS-capacity parameter and related code over the
>> weekend.
>> >
>> > The final bit of work for me comes when Wei's code is merged into master
>> > and I pull it down.
>> >
>> > At that point, I need to add the GUI and API logic to support mutual
>> > exclusion of our features.
>> >
>> >
>> > On Sat, Jun 15, 2013 at 11:12 AM, Chip Childers
>> > <ch...@sungard.com>wrote:
>> >
>> >> On Fri, Jun 14, 2013 at 05:56:27PM -0400, John Burwell wrote:
>> >>> Mike,
>> >>>
>> >>> Awesome. +1.
>> >>>
>> >>> Thanks for your patience with the back and forth,
>> >>> -John
>> >>
>> >> +1 - this now looks like a great approach.
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Sounds good
I have a little bit of clean-up work to do and then I must figure out how
to generate a patch (since I was using merge rather than rebase during
development), so it will probably be tomorrow.
On Mon, Jun 17, 2013 at 2:50 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Great news. Let me know when Review Board is updated, and I will perform
> a second round review.
>
> Thanks,
> -John
>
> On Jun 17, 2013, at 4:48 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > FYI: I added the IOPS-capacity parameter and related code over the
> weekend.
> >
> > The final bit of work for me comes when Wei's code is merged into master
> > and I pull it down.
> >
> > At that point, I need to add the GUI and API logic to support mutual
> > exclusion of our features.
> >
> >
> > On Sat, Jun 15, 2013 at 11:12 AM, Chip Childers
> > <ch...@sungard.com>wrote:
> >
> >> On Fri, Jun 14, 2013 at 05:56:27PM -0400, John Burwell wrote:
> >>> Mike,
> >>>
> >>> Awesome. +1.
> >>>
> >>> Thanks for your patience with the back and forth,
> >>> -John
> >>
> >> +1 - this now looks like a great approach.
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Great news. Let me know when Review Board is updated, and I will perform a second round review.
Thanks,
-John
On Jun 17, 2013, at 4:48 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> FYI: I added the IOPS-capacity parameter and related code over the weekend.
>
> The final bit of work for me comes when Wei's code is merged into master
> and I pull it down.
>
> At that point, I need to add the GUI and API logic to support mutual
> exclusion of our features.
>
>
> On Sat, Jun 15, 2013 at 11:12 AM, Chip Childers
> <ch...@sungard.com>wrote:
>
>> On Fri, Jun 14, 2013 at 05:56:27PM -0400, John Burwell wrote:
>>> Mike,
>>>
>>> Awesome. +1.
>>>
>>> Thanks for your patience with the back and forth,
>>> -John
>>
>> +1 - this now looks like a great approach.
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
FYI: I added the IOPS-capacity parameter and related code over the weekend.
The final bit of work for me comes when Wei's code is merged into master
and I pull it down.
At that point, I need to add the GUI and API logic to support mutual
exclusion of our features.
On Sat, Jun 15, 2013 at 11:12 AM, Chip Childers
<ch...@sungard.com>wrote:
> On Fri, Jun 14, 2013 at 05:56:27PM -0400, John Burwell wrote:
> > Mike,
> >
> > Awesome. +1.
> >
> > Thanks for your patience with the back and forth,
> > -John
>
> +1 - this now looks like a great approach.
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Chip Childers <ch...@sungard.com>.
On Fri, Jun 14, 2013 at 05:56:27PM -0400, John Burwell wrote:
> Mike,
>
> Awesome. +1.
>
> Thanks for your patience with the back and forth,
> -John
+1 - this now looks like a great approach.
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Right...Wei can work on the relevant code to support his feature and I'll
do mine and we can coordinate on the GUI and API mutual-exclusion logic.
On Fri, Jun 14, 2013 at 4:06 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> It is important to note that we need a similar rail for throttled I/O as
> well. Hopefully, we can work that out early next week.
>
> Thanks,
> -John
>
> On Jun 14, 2013, at 6:01 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > We should be on our way to getting a really good 1.0 release out for
> this,
> > so that's cool. :)
> >
> > Thanks for the time...I should be able to find time this weekend to work
> on
> > this.
> >
> > Next week I can look into the GUI and API changes for mutual exclusion of
> > mine and Wei's features.
> >
> >
> > On Fri, Jun 14, 2013 at 3:56 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Awesome. +1.
> >>
> >> Thanks for your patience with the back and forth,
> >> -John
> >>
> >> On Jun 14, 2013, at 5:50 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Quick comments below in red, but I think we're good to go.
> >>>
> >>>
> >>> On Fri, Jun 14, 2013 at 3:42 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Please see my comments in-line below.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 14, 2013, at 5:37 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> Comments below in red (but I believe we are reaching a consensus
> here).
> >>>> :)
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Please see my comments in-line below.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I am OK with that approach, John.
> >>>>>>>
> >>>>>>> So, let me review to make sure I follow you correctly:
> >>>>>>>
> >>>>>>> We introduce two new parameters to the plug-in: Number of Total
> IOPS
> >>>> for
> >>>>>>> the SAN and an overcommit ratio. (On a side note, if we are just
> >>>>>>> multiplying the two numbers, why don't we have the user just input
> >>>> their
> >>>>>>> product?)
> >>>>>>
> >>>>>> It is a straw man suggestion to allow some fine tuning of the
> >> allocation
> >>>>>> algorithm. I defer to you to determine whether or not such a tunable
> >>>> would
> >>>>>> be valuable in this use case. My thinking was that it is tunable to
> >>>>>> support operators as they more deeply understand/observe the CS
> >> workload
> >>>>>> while allowing them to think about IOPS absolutely. For example, I
> >>>> start
> >>>>>> out at say 50% overcommit and realize that the load is actually
> >> heavier
> >>>>>> than expected, I can reduce it or vice versa.
> >>>>>>
> >>>>>
> >>>>> Using your example of support people changing the value after seeing
> >> how
> >>>>> the system is performing:
> >>>>>
> >>>>> If we had them just pass in a new Total IOPS value, that should cover
> >> it.
> >>>>
> >>>> That is fine answer to me. I expect that we will be refining this
> >>>> capability over the next few releases. Therefore, I am cool with
> >> starting
> >>>> as simple as possible as it will ease/simplify future evolution.
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> Are both of these new parameters to the create storage pool API
> >> command
> >>>>>> or
> >>>>>>> are they passed into the create storage pool API command through
> its
> >>>> url
> >>>>>>> parameter?
> >>>>>>>
> >>>>>>> If they are new parameters, we should make two new columns in the
> >>>>>>> storage_pool table.
> >>>>>>>
> >>>>>>> If they are passed in via the url parameter, they should go in the
> >>>>>>> storage_pool_details table.
> >>>>>>
> >>>>>> I think these allocation parameters implementation agnostic, and
> >> should
> >>>> be
> >>>>>> columns on the storage_pool table and first class properties on the
> >>>>>> StoragePool class.
> >>>>>>
> >>>>>
> >>>>> Per my most recent e-mail, I agree.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> For 4.2, if someone wants to change these values, they must update
> >> the
> >>>> DB
> >>>>>>> manually.
> >>>>>>
> >>>>>> Is it not possible to support updates through the updateStoragePool
> >> API
> >>>>>> call?
> >>>>>>
> >>>>>
> >>>>> Edison - if you're reading these - do you know if the plug-ins are
> >>>> informed
> >>>>> when the updateStoragePool API command is issued?
> >>>>
> >>>> The way I am thinking about these thresholds is that they are part of
> >> the
> >>>> storage engine itself. Why would plugin need to know that the value
> >>>> changed? When the plugin needs the value, it will be retrieved from
> the
> >>>> database ...
> >>>>
> >>>
> >>> You know, when I wrote what I wrote above, I was thinking the data was
> >>> coming in with the url parameter, which has to be parsed by the plug-in
> >>> (and it's stored in the storage_pool_details table).
> >>>
> >>> Since this is a new, standard parameter for the API (stored in the
> >>> storage_pool table), it should work just fine without plug-in
> >> interaction.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> Every time a volume is created or deleted for the SolidFire
> plug-in,
> >>>> the
> >>>>>>> Current IOPS value (sum of all volumes' Min IOPS that are
> associated
> >>>> with
> >>>>>>> the plug-in) is updated.
> >>>>>>
> >>>>>> I don't think we need another column for this computed value. It
> will
> >>>> be
> >>>>>> simpler to simply ask the database to sum the values as needed.
> >>>>>>
> >>>>>
> >>>>> I'm thinking now that we should follow the pattern established by the
> >>>>> "size" field. There are two related size fields for a storage pool:
> >>>>> capacity_bytes and available bytes.
> >>>>>
> >>>>> We should just follow this pattern and create two new fields:
> >>>> capacity_iops
> >>>>> and available_iops. The plug-in can increment or decrement
> >> available_iops
> >>>>> each time a volume is created or deleted, respectively.
> >>>>
> >>>> I would rather see us do away with available_bytes. My experience
> with
> >>>> storing computed fields is that often get out of sync and represent
> >>>> premature optimization. I missed that part of the throttled I/O patch
> >> ...
> >>>>
> >>>
> >>> Fair enough. Agreed - one new field in the storage_pool table:
> >>> capacity_iops.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> The allocator can use these fields to determine if it can fit in a
> >> new
> >>>>>>> volume.
> >>>>>>
> >>>>>> Exactly.
> >>>>>>
> >>>>>>>
> >>>>>>> Does it look like my understanding is OK?
> >>>>>>
> >>>>>> Yes, looks good to me.
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> I apologize for not being clear -- this conversation has been
> >>>> admittedly
> >>>>>>>> disjoint. I think we should allow the maximum IOPS and
> >> overcommitment
> >>>>>>>> values to be updated though I would recommend restricting updates
> to
> >>>> to
> >>>>>> be
> >>>>>>>> an increasing value for 4.2 (e.g. users can increase the number of
> >>>> total
> >>>>>>>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to
> >>>> 200,000).
> >>>>>>>> While not ideal, given the amount of time we have left for 4.2, it
> >>>> will
> >>>>>>>> cover most cases, and we can address the implications of reducing
> >>>>>> resource
> >>>>>>>> capacity in 4.3. This approach addresses both of your concerns.
> >>>>>> First, it
> >>>>>>>> allows the administrator/operator to determine what portion of the
> >>>>>> device
> >>>>>>>> they wish to dedicate. For example, if the device has a total
> >>>> capacity
> >>>>>> of
> >>>>>>>> 200,000 IOPS, and they only want CS to use 25% of the device then
> >> they
> >>>>>> set
> >>>>>>>> the maximum total IOPS to 50,000. Second, as they grow capacity,
> >> they
> >>>>>> can
> >>>>>>>> update the DataStore to increase the number of IOPS they want to
> >>>>>> dedicate
> >>>>>>>> to CS' use. I would imagine expansion of capacity happens
> >>>> infrequently
> >>>>>>>> enough that increasing the maximum IOPS value would not be a
> >>>> significant
> >>>>>>>> burden.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> "the administrator/operator simply needs to tell us the total
> >> number
> >>>> of
> >>>>>>>>> IOPS that can be committed to it and an overcommitment factor."
> >>>>>>>>>
> >>>>>>>>> Are you thinking when we create a plug-in as primary storage that
> >> we
> >>>>>> say
> >>>>>>>> -
> >>>>>>>>> up front - how many IOPS the SAN can handle?
> >>>>>>>>>
> >>>>>>>>> That is not a good move, in my opinion. Our SAN is designed to
> >> start
> >>>>>>>> small
> >>>>>>>>> and grow to PBs. As the need arises for more storage, the admin
> >>>>>> purchases
> >>>>>>>>> additional storage nodes that join the cluster and the
> performance
> >>>> and
> >>>>>>>>> capacity go up.
> >>>>>>>>>
> >>>>>>>>> We need to know how many IOPS total the SAN can handle and what
> it
> >> is
> >>>>>>>>> committed to currently (the sum of the number of volumes' min
> >> IOPS).
> >>>>>>>>>
> >>>>>>>>> We also cannot assume the SAN is dedicated to CS.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <
> jburwell@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Simon,
> >>>>>>>>>>
> >>>>>>>>>> Yes, it is CloudStack's job to protect, as best it can, from
> >>>>>>>>>> oversubscribing resources. I would argue that resource
> management
> >>>> is
> >>>>>>>> one,
> >>>>>>>>>> if not the most, important functions of the system. It is no
> >>>>>> different
> >>>>>>>>>> than the allocation/planning performed for hosts relative to
> cores
> >>>> and
> >>>>>>>>>> memory. We can still oversubscribe resources, but we have
> rails +
> >>>>>> knobs
> >>>>>>>>>> and dials to avoid it. Without these controls in place, we
> could
> >>>>>> easily
> >>>>>>>>>> allow users to deploy workloads that overrun resources harming
> all
> >>>>>>>> tenants.
> >>>>>>>>>>
> >>>>>>>>>> I also think that we are over thinking this issue for
> provisioned
> >>>>>> IOPS.
> >>>>>>>>>> When the DataStore is configured, the administrator/operator
> >> simply
> >>>>>>>> needs
> >>>>>>>>>> to tell us the total number of IOPS that can be committed to it
> >> and
> >>>> an
> >>>>>>>>>> overcommitment factor. As we allocate volumes to that
> DataStore,
> >> we
> >>>>>>>> sum up
> >>>>>>>>>> the committed IOPS of the existing Volumes attached to the
> >>>> DataStore,
> >>>>>>>> apply
> >>>>>>>>>> the overcommitment factor, and determine whether or not the
> >>>> requested
> >>>>>>>>>> minimum IOPS for the new volume can be fulfilled. We can
> provide
> >>>> both
> >>>>>>>>>> general and vendor specific documentation for determining these
> >>>> values
> >>>>>>>> --
> >>>>>>>>>> be they to consume the entire device or a portion of it.
> >>>>>>>>>>
> >>>>>>>>>> Querying the device is unnecessary and deceptive. CloudStack
> >>>> resource
> >>>>>>>>>> management is not interested in the current state of the device
> >>>> which
> >>>>>>>> could
> >>>>>>>>>> be anywhere from extremely heavy to extremely light at any given
> >>>> time.
> >>>>>>>> We
> >>>>>>>>>> are interested in the worst case load that is anticipated for
> >>>>>> resource.
> >>>>>>>> In
> >>>>>>>>>> my view, it is up to administrators/operators to instrument
> their
> >>>>>>>>>> environment to understand usage patterns and capacity. We
> should
> >>>>>>>> provide
> >>>>>>>>>> information that will help determine what should be
> >>>>>>>> instrumented/monitored,
> >>>>>>>>>> but that function should be performed outside of CloudStack.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com>
> >> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I'd like to comment on this briefly.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I think an assumption is being made that the SAN is being
> >> dedicated
> >>>>>> to
> >>>>>>>> a
> >>>>>>>>>> CS instance.
> >>>>>>>>>>>
> >>>>>>>>>>> My person opinion that this whole IOPS calculation is getting
> >>>> rather
> >>>>>>>>>> complicated, and could probably be much simpler than this. Over
> >>>>>>>>>> subscription is a fact of life on virtually all storage, and is
> >>>> really
> >>>>>>>> no
> >>>>>>>>>> different in concept than multiple virt instances on a single
> >> piece
> >>>> of
> >>>>>>>>>> hardware. All decent SANs offer many management options for the
> >>>>>> storage
> >>>>>>>>>> engineers to keep track of IOPS utilization, and plan for
> spindle
> >>>>>>>>>> augmentation as required.
> >>>>>>>>>>> Is it really the job of CS to become yet another management
> layer
> >>>> on
> >>>>>>>> top
> >>>>>>>>>> of this?
> >>>>>>>>>>>
> >>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>
> >>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
> >>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> 1) We want number of IOPS currently supported by the SAN.
> >>>>>>>>>>>
> >>>>>>>>>>> 2) We want the number of IOPS that are committed (sum of min
> IOPS
> >>>> for
> >>>>>>>>>> each
> >>>>>>>>>>> volume).
> >>>>>>>>>>>
> >>>>>>>>>>> We could do the following to keep track of IOPS:
> >>>>>>>>>>>
> >>>>>>>>>>> The plug-in could have a timer thread that goes off every,
> say, 1
> >>>>>>>> minute.
> >>>>>>>>>>>
> >>>>>>>>>>> It could query the SAN for the number of nodes that make up the
> >> SAN
> >>>>>> and
> >>>>>>>>>>> multiple this by 50,000. This is essentially the number of
> >>>> supported
> >>>>>>>> IOPS
> >>>>>>>>>>> of the SAN.
> >>>>>>>>>>>
> >>>>>>>>>>> The next API call could be to get all of the volumes on the
> SAN.
> >>>>>>>> Iterate
> >>>>>>>>>>> through them all and add up their min IOPS values. This is the
> >>>> number
> >>>>>>>> of
> >>>>>>>>>>> IOPS the SAN is committed to.
> >>>>>>>>>>>
> >>>>>>>>>>> These two numbers can then be updated in the storage_pool table
> >> (a
> >>>>>>>> column
> >>>>>>>>>>> for each value).
> >>>>>>>>>>>
> >>>>>>>>>>> The allocators can get these values as needed (and they would
> be
> >> as
> >>>>>>>>>>> accurate as the last time the thread asked the SAN for this
> >> info).
> >>>>>>>>>>>
> >>>>>>>>>>> These two fields, the min IOPS of the volume to create, and the
> >>>>>>>>>> overcommit
> >>>>>>>>>>> ratio of the plug-in would tell the allocator if it can select
> >> the
> >>>>>>>> given
> >>>>>>>>>>> storage pool.
> >>>>>>>>>>>
> >>>>>>>>>>> What do you think?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> "As I mentioned previously, I am very reluctant for any
> feature
> >> to
> >>>>>>>> come
> >>>>>>>>>>>> into master that can exhaust resources."
> >>>>>>>>>>>>
> >>>>>>>>>>>> Just wanted to mention that, worst case, the SAN would fail
> >>>> creation
> >>>>>>>> of
> >>>>>>>>>>>> the volume before allowing a new volume to break the system.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Are you thinking we add a column on to the storage pool
> table,
> >>>>>>>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> That is easy enough.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> How do you want to determine what the SAN is capable of
> >>>> supporting
> >>>>>>>> IOPS
> >>>>>>>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you
> >> add
> >>>>>>>>>> storage
> >>>>>>>>>>>>> nodes to the cluster, the number of IOPS increases. Do we
> have
> >> a
> >>>>>>>>>> thread we
> >>>>>>>>>>>>> can use to query external devices like this SAN to update the
> >>>>>>>> supported
> >>>>>>>>>>>>> number of IOPS?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass
> in
> >> an
> >>>>>>>>>>>>> overcommit ration to the plug-in when it's created? We would
> >> need
> >>>>>> to
> >>>>>>>>>> store
> >>>>>>>>>>>>> this in the storage_pool table, as well, I believe.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We should also get Wei involved in this as his feature will
> >> need
> >>>>>>>>>> similar
> >>>>>>>>>>>>> functionality.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, we should do this FAST as we have only two weeks left
> and
> >>>>>> many
> >>>>>>>> of
> >>>>>>>>>>>>> us will be out for several days for the CS Collab Conference.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <
> >>>> jburwell@basho.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Querying the SAN only indicates the number of IOPS currently
> >> in
> >>>>>> use.
> >>>>>>>>>>>>>> The allocator needs to check the number of IOPS committed
> >> which
> >>>> is
> >>>>>>>>>> tracked
> >>>>>>>>>>>>>> by CloudStack. For 4.2, we should be able to query the
> number
> >> of
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> committed to a DataStore, and determine whether or not the
> >>>> number
> >>>>>>>>>> requested
> >>>>>>>>>>>>>> can be fulfilled by that device. It seems to be that a
> >>>>>>>>>>>>>> DataStore#getCommittedIOPS() : Long method would be
> >> sufficient.
> >>>>>>>>>>>>>> DataStore's that don't support provisioned IOPS would return
> >>>> null.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As I mentioned previously, I am very reluctant for any
> feature
> >>>> to
> >>>>>>>> come
> >>>>>>>>>>>>>> into master that can exhaust resources.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yeah, I'm not sure I could come up with anything near an
> >>>> accurate
> >>>>>>>>>>>>>>> assessment of how many IOPS are currently available on the
> >> SAN
> >>>>>> (or
> >>>>>>>>>>>>>> even a
> >>>>>>>>>>>>>>> total number that are available for volumes). Not sure if
> >>>> there's
> >>>>>>>> yet
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>> API call for that.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If I did know this number (total number of IOPS supported
> by
> >>>> the
> >>>>>>>>>> SAN),
> >>>>>>>>>>>>>> we'd
> >>>>>>>>>>>>>>> still have to keep track of the total number of volumes
> we've
> >>>>>>>> created
> >>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin
> >> issues
> >>>> an
> >>>>>>>> API
> >>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given
> >>>> volume
> >>>>>>>> or
> >>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>> volumes (not supported from CS, but supported via the
> >> SolidFire
> >>>>>>>> API),
> >>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>> numbers in CS would be off.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a
> really
> >>>> good
> >>>>>>>>>> idea
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> a future release.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> For 4.2 I think we should stick to having the allocator
> >> detect
> >>>> if
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>> QoS is desired and if the storage pool in question supports
> >>>> that
> >>>>>>>>>>>>>> feature.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If you really are over provisioned on your SAN in terms of
> >> IOPS
> >>>>>> or
> >>>>>>>>>>>>>>> capacity, the SAN can let the admin know in several
> different
> >>>>>> ways
> >>>>>>>>>>>>>> (e-mail,
> >>>>>>>>>>>>>>> SNMP, GUI).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Please see my comments in-line below.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2.
> However,
> >>>> we
> >>>>>>>> may
> >>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> throw an exception if we can not fulfill a requested
> QoS.
> >> If
> >>>>>> the
> >>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> expecting that the hypervisor will provide a particular
> >> QoS,
> >>>>>> and
> >>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>> not possible, it seems like we should inform them rather
> >>>> than
> >>>>>>>>>>>>>> silently
> >>>>>>>>>>>>>>>>>> ignoring their request.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Sure, that sounds reasonable.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We'd have to come up with some way for the allocators to
> >> know
> >>>>>>>> about
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> requested storage QoS and then query the candidate
> drivers.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Any thoughts on how we might do that?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> To collect my thoughts from previous parts of the
> thread,
> >> I
> >>>> am
> >>>>>>>>>>>>>>>>>> uncomfortable with the idea that the management server
> can
> >>>>>>>>>>>>>> overcommit a
> >>>>>>>>>>>>>>>>>> resource. You had mentioned querying the device for
> >>>> available
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>> While
> >>>>>>>>>>>>>>>>>> that would be nice, it seems like we could fall back to
> a
> >>>> max
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> overcommit factor manually calculated and entered by the
> >>>>>>>>>>>>>>>>>> administrator/operator. I think such threshold and
> >>>> allocation
> >>>>>>>>>> rails
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O --
> it
> >>>> is
> >>>>>> a
> >>>>>>>>>>>>>> basic
> >>>>>>>>>>>>>>>>>> feature of any cloud orchestration platform.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Are you thinking this ability would make it into 4.2?
> Just
> >>>>>>>> curious
> >>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>> release we're talking about here. For the SolidFire SAN,
> >> you
> >>>>>>>> might
> >>>>>>>>>>>>>> have,
> >>>>>>>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS)
> >> and
> >>>>>> then
> >>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to
> >>>> have
> >>>>>> a
> >>>>>>>>>> way
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> know that the number of supported IOPS has increased.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in
> 4.2.
> >>>> For
> >>>>>>>>>>>>>> example,
> >>>>>>>>>>>>>>>> we may only support expanding capacity in 4.2, and not
> >> handle
> >>>>>> any
> >>>>>>>>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
> >>>>>> Extrapolating
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a
> >>>> similar
> >>>>>>>> set
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> rails defined per host.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be
> >> expressed
> >>>>>> in
> >>>>>>>> a
> >>>>>>>>>>>>>>>>>> implementation specific manner. I think we need to
> arrive
> >>>> at a
> >>>>>>>>>>>>>> general
> >>>>>>>>>>>>>>>>>> model that can be exploited by the allocators and
> >> planners.
> >>>> We
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> restrict implementation specific key-value pairs (call
> >> them
> >>>>>>>>>> details,
> >>>>>>>>>>>>>>>>>> extended data, whatever) to information that is unique
> to
> >>>> the
> >>>>>>>>>>>>>> driver and
> >>>>>>>>>>>>>>>>>> would provide no useful information to the management
> >>>> server's
> >>>>>>>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to
> >> fit
> >>>>>>>> those
> >>>>>>>>>>>>>>>>>> criteria.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I wonder if this would be a good discussion topic for
> >>>> Sunday's
> >>>>>> CS
> >>>>>>>>>>>>>> Collab
> >>>>>>>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yes, it would -- I will put something in the wiki topic.
> It
> >>>> will
> >>>>>>>>>>>>>> also be
> >>>>>>>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
> >>>>>> include
> >>>>>>>>>>>>>> some of
> >>>>>>>>>>>>>>>> my opinions on the topic.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> So, here's my suggestion for 4.2:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Accept the values as they are currently required (four
> >> new
> >>>>>>>> fields
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>>>>>> feature or two new fields for mine).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio
> >> buttons:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1) No QoS
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2) Hypervisor QoS
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 3) Storage Qos
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The admin needs to specify storage tags that only map
> to
> >>>>>>>> storage
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> supports Storage QoS.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that
> >> unless
> >>>>>> all
> >>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>> in use support the new fields, they may not be
> enforced.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Post 4.3:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Come up with a way to more generally enter these
> >> parameters
> >>>>>>>>>>>>>> (probably
> >>>>>>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>> key/value pairs sent to the drivers).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Have the drivers expose their feature set so the
> >> allocators
> >>>>>> can
> >>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>> them more fully and throw an exception if there is not
> a
> >>>>>>>>>> sufficient
> >>>>>>>>>>>>>>>>>> match.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need
> >> to
> >>>>>> put
> >>>>>>>>>> some
> >>>>>>>>>>>>>>>>>> burden
> >>>>>>>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way
> that
> >>>>>> makes
> >>>>>>>>>>>>>> sense.
> >>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported
> >> Min
> >>>>>> and
> >>>>>>>>>> Max
> >>>>>>>>>>>>>>>>>> values,
> >>>>>>>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>>> primary
> >>>>>>>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that
> >>>> doesn't,
> >>>>>>>>>> then
> >>>>>>>>>>>>>> he's
> >>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>> going to get the Min and Max feature. We could add
> help
> >>>> text
> >>>>>>>> to
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> pop-up
> >>>>>>>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and
> >> Max
> >>>>>> text
> >>>>>>>>>>>>>> fields.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Same idea for Wei's feature.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
> >>>>>>>>>> comprehensive
> >>>>>>>>>>>>>>>>>>>> approach for post 4.2.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Maybe in the future we could have the drivers
> advertise
> >>>>>> their
> >>>>>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>>>> and if the allocator feels a request is not being
> >>>> satisfied
> >>>>>>>> (say
> >>>>>>>>>>>>>> Min
> >>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>> entered, but it not's supported by any storage
> plug-in)
> >> it
> >>>>>> can
> >>>>>>>>>>>>>> throw
> >>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>> exception.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Please see my comment in-line below.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I've put comments below in red.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
> >>>>>> features
> >>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>> mutually
> >>>>>>>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a
> >>>> configuration
> >>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> contains both
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads
> to
> >>>>>>>>>>>>>>>>>> non-deterministic
> >>>>>>>>>>>>>>>>>>>>>>>> operations.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor
> >> or
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> oriented
> >>>>>>>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
> >>>>>> storage
> >>>>>>>>>>>>>> tags,
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> have no
> >>>>>>>>>>>>>>>>>>>>>>>> means of expressing this type of mutual
> exclusion. I
> >>>>>>>> wasn't
> >>>>>>>>>>>>>>>>>>>>>> necessarily
> >>>>>>>>>>>>>>>>>>>>>>>> intending that we implement this allocation model
> in
> >>>>>> 4.3,
> >>>>>>>>>> but
> >>>>>>>>>>>>>>>>>> instead,
> >>>>>>>>>>>>>>>>>>>>>>>> determine if this type model would be one we would
> >>>> want
> >>>>>> to
> >>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that
> >> the
> >>>>>>>> data
> >>>>>>>>>>>>>> model
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> current implementation would not preclude
> evolution
> >> in
> >>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> direction. My
> >>>>>>>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
> >>>>>> user's
> >>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> "cloud"
> >>>>>>>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource
> set
> >>>> to
> >>>>>>>>>>>>>> fulfill a
> >>>>>>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>>>>>> system requirements.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in
> that
> >>>> what
> >>>>>>>>>> we've
> >>>>>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>> should not make refinement difficult in the future.
> >> If
> >>>> we
> >>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>> modify
> >>>>>>>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
> >>>>>>>> relatively
> >>>>>>>>>>>>>> simple
> >>>>>>>>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
> >>>>>>>>>> aggregated
> >>>>>>>>>>>>>>>> result
> >>>>>>>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes
> >> for
> >>>>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>>> I/O and
> >>>>>>>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping.
> My
> >>>>>>>> concern
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At
> >>>> minimum,
> >>>>>>>>>>>>>> Javadoc
> >>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and
> >> their
> >>>>>>>>>> mutual
> >>>>>>>>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their
> >> mutual
> >>>>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The other part that is interesting is that throttled
> >> I/O
> >>>>>>>>>>>>>> provides
> >>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where
> >>>> provisioned
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> range. In order to select the best available
> resource
> >> to
> >>>>>>>>>>>>>> fulfill a
> >>>>>>>>>>>>>>>>>> QoS, we
> >>>>>>>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in
> >>>> terms
> >>>>>> of
> >>>>>>>>>>>>>> units
> >>>>>>>>>>>>>>>>>> (IOPS or
> >>>>>>>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we
> >> want
> >>>>>> to
> >>>>>>>>>>>>>>>> achieve a
> >>>>>>>>>>>>>>>>>>>>>> model like I described, I think we would need to
> >> resolve
> >>>>>>>> this
> >>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>> in 4.2
> >>>>>>>>>>>>>>>>>>>>>> to ensure a viable migration path.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the
> >>>> input
> >>>>>>>> for
> >>>>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>> Plus
> >>>>>>>>>>>>>>>>>>>>> people probably want to input the data in terms
> they're
> >>>>>>>>>> familiar
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> the products in question.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging
> and
> >>>> let
> >>>>>>>> the
> >>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected
> >> due
> >>>>>> to
> >>>>>>>> a
> >>>>>>>>>>>>>> given
> >>>>>>>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it
> >>>> because
> >>>>>>>>>> what
> >>>>>>>>>>>>>>>>>> happens if
> >>>>>>>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage
> >>>> after
> >>>>>>>>>>>>>> having
> >>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>> a Disk Offering?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we
> >> should
> >>>>>>>>>>>>>> re-think
> >>>>>>>>>>>>>>>> it.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose
> >>>> their
> >>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> As I think through the implications of these
> >>>>>> requirements
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> reflect
> >>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't
> >>>> already
> >>>>>>>>>>>>>> impacted
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
> >>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> has no
> >>>>>>>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
> >>>>>> devices
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore,
> >> as I
> >>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources
> >> --
> >>>>>>>>>>>>>> potentially
> >>>>>>>>>>>>>>>>>>>>>> causing
> >>>>>>>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled.
> It
> >>>>>> seems
> >>>>>>>>>>>>>> to me
> >>>>>>>>>>>>>>>>>>>>>> that a
> >>>>>>>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should
> express
> >>>> the
> >>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment
> >> factor.
> >>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>>>>>>> should be used by the storage allocators to
> >> determine
> >>>>>> the
> >>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>> best
> >>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It
> seems
> >>>>>> that
> >>>>>>>> a
> >>>>>>>>>>>>>>>> similar
> >>>>>>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>>>>>> considerations would need to be added to the
> >>>> Hypervisor
> >>>>>>>>>> layer
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent
> >> oversubscription.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the
> >> path
> >>>>>>>> that
> >>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>> previously
> >>>>>>>>>>>>>>>>>>>>>>> established for other properties you see on dialogs
> >> in
> >>>>>> CS:
> >>>>>>>>>> Just
> >>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is
> >>>> deployed
> >>>>>>>> to
> >>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs,
> >>>> say, a
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> confines the deployment only to storage that
> supports
> >>>> the
> >>>>>>>>>>>>>> selected
> >>>>>>>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the
> way
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>> works
> >>>>>>>>>>>>>>>>>>>>>>> today.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for
> >> the
> >>>>>>>>>>>>>>>> administrator
> >>>>>>>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is
> >>>> that
> >>>>>> we
> >>>>>>>>>>>>>> over
> >>>>>>>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the
> >> purposes
> >>>> of
> >>>>>>>>>>>>>>>>>> illustration,
> >>>>>>>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for
> >> that
> >>>>>>>>>> device
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> 100000.
> >>>>>>>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by
> 50%.
> >>>>>>>>>>>>>> Therefore,
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than
> >> 150,000
> >>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the
> >>>> DataStore
> >>>>>>>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
> >>>>>> factor.
> >>>>>>>>>>>>>> As we
> >>>>>>>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure
> >> that
> >>>>>>>>>>>>>> guarantee
> >>>>>>>>>>>>>>>>>> more IOPS
> >>>>>>>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does
> >> that
> >>>>>>>> make
> >>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement.
> >> I'm
> >>>>>> not
> >>>>>>>>>> even
> >>>>>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely
> >>>> remain.
> >>>>>>>> I'd
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>> all of the min values for all of the volumes on the
> SAN
> >>>> and
> >>>>>>>>>> total
> >>>>>>>>>>>>>>>> them
> >>>>>>>>>>>>>>>>>> up,
> >>>>>>>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user
> facing)
> >>>>>>>>>> supported
> >>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> the system.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow
> >>>> non-QoS
> >>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS?
> >> For
> >>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>> IOPS, I
> >>>>>>>>>>>>>>>>>>>>>>>> think a side effect of the current implementation
> is
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor
> throttled
> >>>>>> I/O,
> >>>>>>>> it
> >>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
> >>>>>>>> side-by-side
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
> >>>>>> potentially
> >>>>>>>>>>>>>> starve
> >>>>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes
> >> defined
> >>>>>> the
> >>>>>>>>>>>>>>>> collocated
> >>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>> from being fulfilled.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside
> of
> >>>> CS)
> >>>>>>>> and
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>> specify
> >>>>>>>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will
> >> be
> >>>> at
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>> defaults
> >>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a
> SolidFire
> >>>>>> SAN,
> >>>>>>>>>> you
> >>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>> your Mins met.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage
> >>>> tags
> >>>>>>>>>>>>>> exclusions
> >>>>>>>>>>>>>>>>>>>>>> list,
> >>>>>>>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e.
> don't
> >>>>>>>> deploy
> >>>>>>>>>> my
> >>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> storage with these following tags).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion
> list
> >>>> as
> >>>>>> it
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> complicate component assembly. It would require a
> >>>> storage
> >>>>>>>>>>>>>> plugin to
> >>>>>>>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies
> >> and
> >>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express
> >>>> their
> >>>>>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>>>>>> which can be matched to a set of requested
> >> requirements.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
> >>>>>> inclusion
> >>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
> >>>>>> functionality
> >>>>>>>>>> in
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> allocator understands.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS.
> Since
> >>>> it
> >>>>>>>> only
> >>>>>>>>>>>>>>>> limits
> >>>>>>>>>>>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only
> a
> >>>>>> system
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>>>>>>>> minimum IOPS does this.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this
> problem
> >>>> does
> >>>>>>>> not
> >>>>>>>>>>>>>>>> exist.
> >>>>>>>>>>>>>>>>>>>>>> However, I am concerned with the more general case
> as
> >> we
> >>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>> devices with provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
> >>>>>> vendor-specific
> >>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
> >>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>> IOPS, but
> >>>>>>>>>>>>>>>>>>>>> what if you break them out by read and write or do
> >>>>>> something
> >>>>>>>>>>>>>> else? We
> >>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>> a more general mechanism.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
> >>>>>> throttled
> >>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive
> per
> >>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> We also need to understand the implications of
> these
> >>>> QoS
> >>>>>>>>>>>>>>>> guarantees
> >>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> operation of the system to ensure that the
> >> underlying
> >>>>>>>>>> hardware
> >>>>>>>>>>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will
> >> likely
> >>>>>> be
> >>>>>>>>>>>>>> forced
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
> >>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> 4.3.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with
> some
> >>>>>> great
> >>>>>>>>>>>>>> ideas
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> 4.3. :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were
> planning
> >>>>>> (with
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> tags).
> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that -
> >> in
> >>>>>>>>>>>>>> addition to
> >>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators
> (storage
> >>>> QoS
> >>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>> preferred,
> >>>>>>>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not
> >> available,
> >>>>>> but
> >>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>>>>>>>> is).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in
> >>>> values
> >>>>>>>> for
> >>>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at
> >>>> least
> >>>>>>>>>>>>>> yet), so
> >>>>>>>>>>>>>>>> he
> >>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
> >>>>>> allocators
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>> prefer
> >>>>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it
> >> stands
> >>>>>>>> today
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
> >>>>>> features
> >>>>>>>>>>>>>> (fields
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors
> support).
> >>>>>>>> Perhaps
> >>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is
> >> how
> >>>>>>>>>>>>>> OpenStack
> >>>>>>>>>>>>>>>>>>>>>> addresses
> >>>>>>>>>>>>>>>>>>>>>>>>> the issue).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two
> >>>> weeks
> >>>>>>>>>> until
> >>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>> freeze)
> >>>>>>>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 -
> >> keep
> >>>>>>>> Wei's
> >>>>>>>>>>>>>> fields
> >>>>>>>>>>>>>>>>>>>>>> and my
> >>>>>>>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other
> >> feature
> >>>>>> has
> >>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>> entered
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
> >>>>>> general-purpose
> >>>>>>>>>>>>>> design
> >>>>>>>>>>>>>>>>>>>>>> that can
> >>>>>>>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have
> >> these
> >>>>>>>>>>>>>> situations.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I just published my review @
> >>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before
> >>>> coming
> >>>>>> up
> >>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> file.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's
> changes
> >>>> to
> >>>>>> be
> >>>>>>>>>>>>>> checked
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new
> >> patch
> >>>>>>>> file?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell
> <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my
> >>>> review.
> >>>>>> I
> >>>>>>>> am
> >>>>>>>>>>>>>>>>>> offline
> >>>>>>>>>>>>>>>>>>>>>> ATM,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of
> >> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
> >>>>>> Board?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
> >>>>>> reading
> >>>>>>>>>>>>>> this :)
> >>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented
> >> all
> >>>>>> the
> >>>>>>>>>>>>>> areas
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> wanted
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow
> >> morning
> >>>> or
> >>>>>>>>>>>>>>>> afternoon,
> >>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike
> >> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
> >>>>>> conflicts
> >>>>>>>>>> in
> >>>>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>>>>>>> code,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
> >>>>>> disk_io_throttling
> >>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>>>>>>>>>>>> object_store
> >>>>>>>>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if
> we
> >>>>>> merge
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before
> object_store
> >>>> goes
> >>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling
> fits
> >>>>>> into
> >>>>>>>>>> this
> >>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to
> perform.
> >>>> The
> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to
> have
> >> a
> >>>>>>>> number
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies
> >>>> between
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire ->
> >>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management
> >>>> aspects
> >>>>>>>> of
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
> >>>>>> circular
> >>>>>>>>>>>>>>>> dependency
> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the
> dependency
> >>>>>>>> graph,
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in
> >> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike
> >> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging
> >> my
> >>>>>> code
> >>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>> his
> >>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the
> StoragePoolType.Dynamic
> >>>> code
> >>>>>>>>>>>>>> after his
> >>>>>>>>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
> >>>>>> Tutkowski
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're
> >> going
> >>>>>>>> here:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
> >>>>>> hypervisor
> >>>>>>>>>>>>>> code in
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the
> default
> >>>>>>>> storage
> >>>>>>>>>>>>>>>> plug-in,
> >>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the
> hypervisor
> >> in
> >>>>>> use
> >>>>>>>>>>>>>>>> (although
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i),
> >> etc.)
> >>>> is
> >>>>>>>>>>>>>> actually
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
> >>>>>> placed
> >>>>>>>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> url
> >>>>>>>>>>>>>>>>>>>>>> field
> >>>>>>>>>>>>>>>>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This
> info
> >> is
> >>>>>>>>>> stored
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to
> >> the
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this
> >> takes
> >>>>>> the
> >>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> checks
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
> >>>>>> necessary
> >>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach
> >> command
> >>>>>>>>>>>>>> executes to
> >>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
> >>>>>> invoked
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> detach a
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
> >>>>>> hypervisor
> >>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support
> >>>> Burst
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and
> disk_offerings
> >>>>>> table.
> >>>>>>>>>> If
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
> >>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented
> >> while
> >>>>>>>>>>>>>> avoiding
> >>>>>>>>>>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about
> the
> >>>>>>>> details
> >>>>>>>>>>>>>>>> field.
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the
> >>>> documentation
> >>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that
> the
> >>>>>>>> details
> >>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool,
> or
> >>>>>>>>>>>>>>>> listStoragePool
> >>>>>>>>>>>>>>>>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it
> >> would
> >>>>>> be
> >>>>>>>>>>>>>> useful
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the
> >>>> details
> >>>>>>>>>>>>>> field."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing
> >> Burst
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a
> Disk
> >>>>>>>>>>>>>>>>>> Offering-by-Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created,
> >> you
> >>>>>>>> have
> >>>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>>>>>>>>>>>> disk_offering_details
> >>>>>>>>>>>>>>>>>> table.
> >>>>>>>>>>>>>>>>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept
> >> the
> >>>>>>>> Burst
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields
> >> are
> >>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>>>> DB.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
It is important to note that we need a similar rail for throttled I/O as well. Hopefully, we can work that out early next week.
Thanks,
-John
On Jun 14, 2013, at 6:01 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> We should be on our way to getting a really good 1.0 release out for this,
> so that's cool. :)
>
> Thanks for the time...I should be able to find time this weekend to work on
> this.
>
> Next week I can look into the GUI and API changes for mutual exclusion of
> mine and Wei's features.
>
>
> On Fri, Jun 14, 2013 at 3:56 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Awesome. +1.
>>
>> Thanks for your patience with the back and forth,
>> -John
>>
>> On Jun 14, 2013, at 5:50 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Quick comments below in red, but I think we're good to go.
>>>
>>>
>>> On Fri, Jun 14, 2013 at 3:42 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> Please see my comments in-line below.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 14, 2013, at 5:37 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> Comments below in red (but I believe we are reaching a consensus here).
>>>> :)
>>>>>
>>>>>
>>>>> On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Please see my comments in-line below.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I am OK with that approach, John.
>>>>>>>
>>>>>>> So, let me review to make sure I follow you correctly:
>>>>>>>
>>>>>>> We introduce two new parameters to the plug-in: Number of Total IOPS
>>>> for
>>>>>>> the SAN and an overcommit ratio. (On a side note, if we are just
>>>>>>> multiplying the two numbers, why don't we have the user just input
>>>> their
>>>>>>> product?)
>>>>>>
>>>>>> It is a straw man suggestion to allow some fine tuning of the
>> allocation
>>>>>> algorithm. I defer to you to determine whether or not such a tunable
>>>> would
>>>>>> be valuable in this use case. My thinking was that it is tunable to
>>>>>> support operators as they more deeply understand/observe the CS
>> workload
>>>>>> while allowing them to think about IOPS absolutely. For example, I
>>>> start
>>>>>> out at say 50% overcommit and realize that the load is actually
>> heavier
>>>>>> than expected, I can reduce it or vice versa.
>>>>>>
>>>>>
>>>>> Using your example of support people changing the value after seeing
>> how
>>>>> the system is performing:
>>>>>
>>>>> If we had them just pass in a new Total IOPS value, that should cover
>> it.
>>>>
>>>> That is fine answer to me. I expect that we will be refining this
>>>> capability over the next few releases. Therefore, I am cool with
>> starting
>>>> as simple as possible as it will ease/simplify future evolution.
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Are both of these new parameters to the create storage pool API
>> command
>>>>>> or
>>>>>>> are they passed into the create storage pool API command through its
>>>> url
>>>>>>> parameter?
>>>>>>>
>>>>>>> If they are new parameters, we should make two new columns in the
>>>>>>> storage_pool table.
>>>>>>>
>>>>>>> If they are passed in via the url parameter, they should go in the
>>>>>>> storage_pool_details table.
>>>>>>
>>>>>> I think these allocation parameters implementation agnostic, and
>> should
>>>> be
>>>>>> columns on the storage_pool table and first class properties on the
>>>>>> StoragePool class.
>>>>>>
>>>>>
>>>>> Per my most recent e-mail, I agree.
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> For 4.2, if someone wants to change these values, they must update
>> the
>>>> DB
>>>>>>> manually.
>>>>>>
>>>>>> Is it not possible to support updates through the updateStoragePool
>> API
>>>>>> call?
>>>>>>
>>>>>
>>>>> Edison - if you're reading these - do you know if the plug-ins are
>>>> informed
>>>>> when the updateStoragePool API command is issued?
>>>>
>>>> The way I am thinking about these thresholds is that they are part of
>> the
>>>> storage engine itself. Why would plugin need to know that the value
>>>> changed? When the plugin needs the value, it will be retrieved from the
>>>> database ...
>>>>
>>>
>>> You know, when I wrote what I wrote above, I was thinking the data was
>>> coming in with the url parameter, which has to be parsed by the plug-in
>>> (and it's stored in the storage_pool_details table).
>>>
>>> Since this is a new, standard parameter for the API (stored in the
>>> storage_pool table), it should work just fine without plug-in
>> interaction.
>>>
>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Every time a volume is created or deleted for the SolidFire plug-in,
>>>> the
>>>>>>> Current IOPS value (sum of all volumes' Min IOPS that are associated
>>>> with
>>>>>>> the plug-in) is updated.
>>>>>>
>>>>>> I don't think we need another column for this computed value. It will
>>>> be
>>>>>> simpler to simply ask the database to sum the values as needed.
>>>>>>
>>>>>
>>>>> I'm thinking now that we should follow the pattern established by the
>>>>> "size" field. There are two related size fields for a storage pool:
>>>>> capacity_bytes and available bytes.
>>>>>
>>>>> We should just follow this pattern and create two new fields:
>>>> capacity_iops
>>>>> and available_iops. The plug-in can increment or decrement
>> available_iops
>>>>> each time a volume is created or deleted, respectively.
>>>>
>>>> I would rather see us do away with available_bytes. My experience with
>>>> storing computed fields is that often get out of sync and represent
>>>> premature optimization. I missed that part of the throttled I/O patch
>> ...
>>>>
>>>
>>> Fair enough. Agreed - one new field in the storage_pool table:
>>> capacity_iops.
>>>
>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> The allocator can use these fields to determine if it can fit in a
>> new
>>>>>>> volume.
>>>>>>
>>>>>> Exactly.
>>>>>>
>>>>>>>
>>>>>>> Does it look like my understanding is OK?
>>>>>>
>>>>>> Yes, looks good to me.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> I apologize for not being clear -- this conversation has been
>>>> admittedly
>>>>>>>> disjoint. I think we should allow the maximum IOPS and
>> overcommitment
>>>>>>>> values to be updated though I would recommend restricting updates to
>>>> to
>>>>>> be
>>>>>>>> an increasing value for 4.2 (e.g. users can increase the number of
>>>> total
>>>>>>>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to
>>>> 200,000).
>>>>>>>> While not ideal, given the amount of time we have left for 4.2, it
>>>> will
>>>>>>>> cover most cases, and we can address the implications of reducing
>>>>>> resource
>>>>>>>> capacity in 4.3. This approach addresses both of your concerns.
>>>>>> First, it
>>>>>>>> allows the administrator/operator to determine what portion of the
>>>>>> device
>>>>>>>> they wish to dedicate. For example, if the device has a total
>>>> capacity
>>>>>> of
>>>>>>>> 200,000 IOPS, and they only want CS to use 25% of the device then
>> they
>>>>>> set
>>>>>>>> the maximum total IOPS to 50,000. Second, as they grow capacity,
>> they
>>>>>> can
>>>>>>>> update the DataStore to increase the number of IOPS they want to
>>>>>> dedicate
>>>>>>>> to CS' use. I would imagine expansion of capacity happens
>>>> infrequently
>>>>>>>> enough that increasing the maximum IOPS value would not be a
>>>> significant
>>>>>>>> burden.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> "the administrator/operator simply needs to tell us the total
>> number
>>>> of
>>>>>>>>> IOPS that can be committed to it and an overcommitment factor."
>>>>>>>>>
>>>>>>>>> Are you thinking when we create a plug-in as primary storage that
>> we
>>>>>> say
>>>>>>>> -
>>>>>>>>> up front - how many IOPS the SAN can handle?
>>>>>>>>>
>>>>>>>>> That is not a good move, in my opinion. Our SAN is designed to
>> start
>>>>>>>> small
>>>>>>>>> and grow to PBs. As the need arises for more storage, the admin
>>>>>> purchases
>>>>>>>>> additional storage nodes that join the cluster and the performance
>>>> and
>>>>>>>>> capacity go up.
>>>>>>>>>
>>>>>>>>> We need to know how many IOPS total the SAN can handle and what it
>> is
>>>>>>>>> committed to currently (the sum of the number of volumes' min
>> IOPS).
>>>>>>>>>
>>>>>>>>> We also cannot assume the SAN is dedicated to CS.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Simon,
>>>>>>>>>>
>>>>>>>>>> Yes, it is CloudStack's job to protect, as best it can, from
>>>>>>>>>> oversubscribing resources. I would argue that resource management
>>>> is
>>>>>>>> one,
>>>>>>>>>> if not the most, important functions of the system. It is no
>>>>>> different
>>>>>>>>>> than the allocation/planning performed for hosts relative to cores
>>>> and
>>>>>>>>>> memory. We can still oversubscribe resources, but we have rails +
>>>>>> knobs
>>>>>>>>>> and dials to avoid it. Without these controls in place, we could
>>>>>> easily
>>>>>>>>>> allow users to deploy workloads that overrun resources harming all
>>>>>>>> tenants.
>>>>>>>>>>
>>>>>>>>>> I also think that we are over thinking this issue for provisioned
>>>>>> IOPS.
>>>>>>>>>> When the DataStore is configured, the administrator/operator
>> simply
>>>>>>>> needs
>>>>>>>>>> to tell us the total number of IOPS that can be committed to it
>> and
>>>> an
>>>>>>>>>> overcommitment factor. As we allocate volumes to that DataStore,
>> we
>>>>>>>> sum up
>>>>>>>>>> the committed IOPS of the existing Volumes attached to the
>>>> DataStore,
>>>>>>>> apply
>>>>>>>>>> the overcommitment factor, and determine whether or not the
>>>> requested
>>>>>>>>>> minimum IOPS for the new volume can be fulfilled. We can provide
>>>> both
>>>>>>>>>> general and vendor specific documentation for determining these
>>>> values
>>>>>>>> --
>>>>>>>>>> be they to consume the entire device or a portion of it.
>>>>>>>>>>
>>>>>>>>>> Querying the device is unnecessary and deceptive. CloudStack
>>>> resource
>>>>>>>>>> management is not interested in the current state of the device
>>>> which
>>>>>>>> could
>>>>>>>>>> be anywhere from extremely heavy to extremely light at any given
>>>> time.
>>>>>>>> We
>>>>>>>>>> are interested in the worst case load that is anticipated for
>>>>>> resource.
>>>>>>>> In
>>>>>>>>>> my view, it is up to administrators/operators to instrument their
>>>>>>>>>> environment to understand usage patterns and capacity. We should
>>>>>>>> provide
>>>>>>>>>> information that will help determine what should be
>>>>>>>> instrumented/monitored,
>>>>>>>>>> but that function should be performed outside of CloudStack.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com>
>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I'd like to comment on this briefly.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I think an assumption is being made that the SAN is being
>> dedicated
>>>>>> to
>>>>>>>> a
>>>>>>>>>> CS instance.
>>>>>>>>>>>
>>>>>>>>>>> My person opinion that this whole IOPS calculation is getting
>>>> rather
>>>>>>>>>> complicated, and could probably be much simpler than this. Over
>>>>>>>>>> subscription is a fact of life on virtually all storage, and is
>>>> really
>>>>>>>> no
>>>>>>>>>> different in concept than multiple virt instances on a single
>> piece
>>>> of
>>>>>>>>>> hardware. All decent SANs offer many management options for the
>>>>>> storage
>>>>>>>>>> engineers to keep track of IOPS utilization, and plan for spindle
>>>>>>>>>> augmentation as required.
>>>>>>>>>>> Is it really the job of CS to become yet another management layer
>>>> on
>>>>>>>> top
>>>>>>>>>> of this?
>>>>>>>>>>>
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>
>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>
>>>>>>>>>>> 1) We want number of IOPS currently supported by the SAN.
>>>>>>>>>>>
>>>>>>>>>>> 2) We want the number of IOPS that are committed (sum of min IOPS
>>>> for
>>>>>>>>>> each
>>>>>>>>>>> volume).
>>>>>>>>>>>
>>>>>>>>>>> We could do the following to keep track of IOPS:
>>>>>>>>>>>
>>>>>>>>>>> The plug-in could have a timer thread that goes off every, say, 1
>>>>>>>> minute.
>>>>>>>>>>>
>>>>>>>>>>> It could query the SAN for the number of nodes that make up the
>> SAN
>>>>>> and
>>>>>>>>>>> multiple this by 50,000. This is essentially the number of
>>>> supported
>>>>>>>> IOPS
>>>>>>>>>>> of the SAN.
>>>>>>>>>>>
>>>>>>>>>>> The next API call could be to get all of the volumes on the SAN.
>>>>>>>> Iterate
>>>>>>>>>>> through them all and add up their min IOPS values. This is the
>>>> number
>>>>>>>> of
>>>>>>>>>>> IOPS the SAN is committed to.
>>>>>>>>>>>
>>>>>>>>>>> These two numbers can then be updated in the storage_pool table
>> (a
>>>>>>>> column
>>>>>>>>>>> for each value).
>>>>>>>>>>>
>>>>>>>>>>> The allocators can get these values as needed (and they would be
>> as
>>>>>>>>>>> accurate as the last time the thread asked the SAN for this
>> info).
>>>>>>>>>>>
>>>>>>>>>>> These two fields, the min IOPS of the volume to create, and the
>>>>>>>>>> overcommit
>>>>>>>>>>> ratio of the plug-in would tell the allocator if it can select
>> the
>>>>>>>> given
>>>>>>>>>>> storage pool.
>>>>>>>>>>>
>>>>>>>>>>> What do you think?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> "As I mentioned previously, I am very reluctant for any feature
>> to
>>>>>>>> come
>>>>>>>>>>>> into master that can exhaust resources."
>>>>>>>>>>>>
>>>>>>>>>>>> Just wanted to mention that, worst case, the SAN would fail
>>>> creation
>>>>>>>> of
>>>>>>>>>>>> the volume before allowing a new volume to break the system.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Are you thinking we add a column on to the storage pool table,
>>>>>>>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
>>>>>>>>>>>>>
>>>>>>>>>>>>> That is easy enough.
>>>>>>>>>>>>>
>>>>>>>>>>>>> How do you want to determine what the SAN is capable of
>>>> supporting
>>>>>>>> IOPS
>>>>>>>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you
>> add
>>>>>>>>>> storage
>>>>>>>>>>>>> nodes to the cluster, the number of IOPS increases. Do we have
>> a
>>>>>>>>>> thread we
>>>>>>>>>>>>> can use to query external devices like this SAN to update the
>>>>>>>> supported
>>>>>>>>>>>>> number of IOPS?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in
>> an
>>>>>>>>>>>>> overcommit ration to the plug-in when it's created? We would
>> need
>>>>>> to
>>>>>>>>>> store
>>>>>>>>>>>>> this in the storage_pool table, as well, I believe.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We should also get Wei involved in this as his feature will
>> need
>>>>>>>>>> similar
>>>>>>>>>>>>> functionality.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, we should do this FAST as we have only two weeks left and
>>>>>> many
>>>>>>>> of
>>>>>>>>>>>>> us will be out for several days for the CS Collab Conference.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <
>>>> jburwell@basho.com
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Querying the SAN only indicates the number of IOPS currently
>> in
>>>>>> use.
>>>>>>>>>>>>>> The allocator needs to check the number of IOPS committed
>> which
>>>> is
>>>>>>>>>> tracked
>>>>>>>>>>>>>> by CloudStack. For 4.2, we should be able to query the number
>> of
>>>>>>>> IOPS
>>>>>>>>>>>>>> committed to a DataStore, and determine whether or not the
>>>> number
>>>>>>>>>> requested
>>>>>>>>>>>>>> can be fulfilled by that device. It seems to be that a
>>>>>>>>>>>>>> DataStore#getCommittedIOPS() : Long method would be
>> sufficient.
>>>>>>>>>>>>>> DataStore's that don't support provisioned IOPS would return
>>>> null.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I mentioned previously, I am very reluctant for any feature
>>>> to
>>>>>>>> come
>>>>>>>>>>>>>> into master that can exhaust resources.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, I'm not sure I could come up with anything near an
>>>> accurate
>>>>>>>>>>>>>>> assessment of how many IOPS are currently available on the
>> SAN
>>>>>> (or
>>>>>>>>>>>>>> even a
>>>>>>>>>>>>>>> total number that are available for volumes). Not sure if
>>>> there's
>>>>>>>> yet
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>> API call for that.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If I did know this number (total number of IOPS supported by
>>>> the
>>>>>>>>>> SAN),
>>>>>>>>>>>>>> we'd
>>>>>>>>>>>>>>> still have to keep track of the total number of volumes we've
>>>>>>>> created
>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin
>> issues
>>>> an
>>>>>>>> API
>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given
>>>> volume
>>>>>>>> or
>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>> volumes (not supported from CS, but supported via the
>> SolidFire
>>>>>>>> API),
>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>> numbers in CS would be off.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a really
>>>> good
>>>>>>>>>> idea
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> a future release.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> For 4.2 I think we should stick to having the allocator
>> detect
>>>> if
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>> QoS is desired and if the storage pool in question supports
>>>> that
>>>>>>>>>>>>>> feature.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you really are over provisioned on your SAN in terms of
>> IOPS
>>>>>> or
>>>>>>>>>>>>>>> capacity, the SAN can let the admin know in several different
>>>>>> ways
>>>>>>>>>>>>>> (e-mail,
>>>>>>>>>>>>>>> SNMP, GUI).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please see my comments in-line below.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However,
>>>> we
>>>>>>>> may
>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> throw an exception if we can not fulfill a requested QoS.
>> If
>>>>>> the
>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> expecting that the hypervisor will provide a particular
>> QoS,
>>>>>> and
>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>> not possible, it seems like we should inform them rather
>>>> than
>>>>>>>>>>>>>> silently
>>>>>>>>>>>>>>>>>> ignoring their request.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sure, that sounds reasonable.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We'd have to come up with some way for the allocators to
>> know
>>>>>>>> about
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Any thoughts on how we might do that?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To collect my thoughts from previous parts of the thread,
>> I
>>>> am
>>>>>>>>>>>>>>>>>> uncomfortable with the idea that the management server can
>>>>>>>>>>>>>> overcommit a
>>>>>>>>>>>>>>>>>> resource. You had mentioned querying the device for
>>>> available
>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>> While
>>>>>>>>>>>>>>>>>> that would be nice, it seems like we could fall back to a
>>>> max
>>>>>>>> IOPS
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>>>>>>>>>>>> administrator/operator. I think such threshold and
>>>> allocation
>>>>>>>>>> rails
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it
>>>> is
>>>>>> a
>>>>>>>>>>>>>> basic
>>>>>>>>>>>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Are you thinking this ability would make it into 4.2? Just
>>>>>>>> curious
>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>> release we're talking about here. For the SolidFire SAN,
>> you
>>>>>>>> might
>>>>>>>>>>>>>> have,
>>>>>>>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS)
>> and
>>>>>> then
>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to
>>>> have
>>>>>> a
>>>>>>>>>> way
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> know that the number of supported IOPS has increased.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2.
>>>> For
>>>>>>>>>>>>>> example,
>>>>>>>>>>>>>>>> we may only support expanding capacity in 4.2, and not
>> handle
>>>>>> any
>>>>>>>>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
>>>>>> Extrapolating
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a
>>>> similar
>>>>>>>> set
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> rails defined per host.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be
>> expressed
>>>>>> in
>>>>>>>> a
>>>>>>>>>>>>>>>>>> implementation specific manner. I think we need to arrive
>>>> at a
>>>>>>>>>>>>>> general
>>>>>>>>>>>>>>>>>> model that can be exploited by the allocators and
>> planners.
>>>> We
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> restrict implementation specific key-value pairs (call
>> them
>>>>>>>>>> details,
>>>>>>>>>>>>>>>>>> extended data, whatever) to information that is unique to
>>>> the
>>>>>>>>>>>>>> driver and
>>>>>>>>>>>>>>>>>> would provide no useful information to the management
>>>> server's
>>>>>>>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to
>> fit
>>>>>>>> those
>>>>>>>>>>>>>>>>>> criteria.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I wonder if this would be a good discussion topic for
>>>> Sunday's
>>>>>> CS
>>>>>>>>>>>>>> Collab
>>>>>>>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes, it would -- I will put something in the wiki topic. It
>>>> will
>>>>>>>>>>>>>> also be
>>>>>>>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
>>>>>> include
>>>>>>>>>>>>>> some of
>>>>>>>>>>>>>>>> my opinions on the topic.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Accept the values as they are currently required (four
>> new
>>>>>>>> fields
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio
>> buttons:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1) No QoS
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 3) Storage Qos
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The admin needs to specify storage tags that only map to
>>>>>>>> storage
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> supports Storage QoS.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that
>> unless
>>>>>> all
>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Post 4.3:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Come up with a way to more generally enter these
>> parameters
>>>>>>>>>>>>>> (probably
>>>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Have the drivers expose their feature set so the
>> allocators
>>>>>> can
>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>> them more fully and throw an exception if there is not a
>>>>>>>>>> sufficient
>>>>>>>>>>>>>>>>>> match.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need
>> to
>>>>>> put
>>>>>>>>>> some
>>>>>>>>>>>>>>>>>> burden
>>>>>>>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that
>>>>>> makes
>>>>>>>>>>>>>> sense.
>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported
>> Min
>>>>>> and
>>>>>>>>>> Max
>>>>>>>>>>>>>>>>>> values,
>>>>>>>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
>>>>>>>> SolidFire
>>>>>>>>>>>>>>>> primary
>>>>>>>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that
>>>> doesn't,
>>>>>>>>>> then
>>>>>>>>>>>>>> he's
>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>> going to get the Min and Max feature. We could add help
>>>> text
>>>>>>>> to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> pop-up
>>>>>>>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and
>> Max
>>>>>> text
>>>>>>>>>>>>>> fields.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
>>>>>>>>>> comprehensive
>>>>>>>>>>>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Maybe in the future we could have the drivers advertise
>>>>>> their
>>>>>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>>>> and if the allocator feels a request is not being
>>>> satisfied
>>>>>>>> (say
>>>>>>>>>>>>>> Min
>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>> entered, but it not's supported by any storage plug-in)
>> it
>>>>>> can
>>>>>>>>>>>>>> throw
>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>> exception.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
>>>>>> features
>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>> mutually
>>>>>>>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a
>>>> configuration
>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> contains both
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>>>>>>>>>>>> non-deterministic
>>>>>>>>>>>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor
>> or
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> oriented
>>>>>>>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
>>>>>> storage
>>>>>>>>>>>>>> tags,
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> have no
>>>>>>>>>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
>>>>>>>> wasn't
>>>>>>>>>>>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>>>>>>>>>>>> intending that we implement this allocation model in
>>>>>> 4.3,
>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> instead,
>>>>>>>>>>>>>>>>>>>>>>>> determine if this type model would be one we would
>>>> want
>>>>>> to
>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that
>> the
>>>>>>>> data
>>>>>>>>>>>>>> model
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> current implementation would not preclude evolution
>> in
>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
>>>>>> user's
>>>>>>>>>>>>>> expect
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource set
>>>> to
>>>>>>>>>>>>>> fulfill a
>>>>>>>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in that
>>>> what
>>>>>>>>>> we've
>>>>>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>>>>>> should not make refinement difficult in the future.
>> If
>>>> we
>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>> modify
>>>>>>>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
>>>>>>>> relatively
>>>>>>>>>>>>>> simple
>>>>>>>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
>>>>>>>>>> aggregated
>>>>>>>>>>>>>>>> result
>>>>>>>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes
>> for
>>>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>>> I/O and
>>>>>>>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
>>>>>>>> concern
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At
>>>> minimum,
>>>>>>>>>>>>>> Javadoc
>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and
>> their
>>>>>>>>>> mutual
>>>>>>>>>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their
>> mutual
>>>>>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The other part that is interesting is that throttled
>> I/O
>>>>>>>>>>>>>> provides
>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where
>>>> provisioned
>>>>>>>> IOPS
>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> range. In order to select the best available resource
>> to
>>>>>>>>>>>>>> fulfill a
>>>>>>>>>>>>>>>>>> QoS, we
>>>>>>>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in
>>>> terms
>>>>>> of
>>>>>>>>>>>>>> units
>>>>>>>>>>>>>>>>>> (IOPS or
>>>>>>>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we
>> want
>>>>>> to
>>>>>>>>>>>>>>>> achieve a
>>>>>>>>>>>>>>>>>>>>>> model like I described, I think we would need to
>> resolve
>>>>>>>> this
>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>> in 4.2
>>>>>>>>>>>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the
>>>> input
>>>>>>>> for
>>>>>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>> Plus
>>>>>>>>>>>>>>>>>>>>> people probably want to input the data in terms they're
>>>>>>>>>> familiar
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> the products in question.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and
>>>> let
>>>>>>>> the
>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected
>> due
>>>>>> to
>>>>>>>> a
>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it
>>>> because
>>>>>>>>>> what
>>>>>>>>>>>>>>>>>> happens if
>>>>>>>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage
>>>> after
>>>>>>>>>>>>>> having
>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we
>> should
>>>>>>>>>>>>>> re-think
>>>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose
>>>> their
>>>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> As I think through the implications of these
>>>>>> requirements
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> reflect
>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't
>>>> already
>>>>>>>>>>>>>> impacted
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> has no
>>>>>>>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
>>>>>> devices
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore,
>> as I
>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources
>> --
>>>>>>>>>>>>>> potentially
>>>>>>>>>>>>>>>>>>>>>> causing
>>>>>>>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It
>>>>>> seems
>>>>>>>>>>>>>> to me
>>>>>>>>>>>>>>>>>>>>>> that a
>>>>>>>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express
>>>> the
>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment
>> factor.
>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>>>> should be used by the storage allocators to
>> determine
>>>>>> the
>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>> best
>>>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems
>>>>>> that
>>>>>>>> a
>>>>>>>>>>>>>>>> similar
>>>>>>>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>>>>>>>> considerations would need to be added to the
>>>> Hypervisor
>>>>>>>>>> layer
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent
>> oversubscription.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the
>> path
>>>>>>>> that
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>> previously
>>>>>>>>>>>>>>>>>>>>>>> established for other properties you see on dialogs
>> in
>>>>>> CS:
>>>>>>>>>> Just
>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is
>>>> deployed
>>>>>>>> to
>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs,
>>>> say, a
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> confines the deployment only to storage that supports
>>>> the
>>>>>>>>>>>>>> selected
>>>>>>>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>> works
>>>>>>>>>>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for
>> the
>>>>>>>>>>>>>>>> administrator
>>>>>>>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is
>>>> that
>>>>>> we
>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the
>> purposes
>>>> of
>>>>>>>>>>>>>>>>>> illustration,
>>>>>>>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for
>> that
>>>>>>>>>> device
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> 100000.
>>>>>>>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>>>>>>>>>>>> Therefore,
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than
>> 150,000
>>>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the
>>>> DataStore
>>>>>>>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
>>>>>> factor.
>>>>>>>>>>>>>> As we
>>>>>>>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure
>> that
>>>>>>>>>>>>>> guarantee
>>>>>>>>>>>>>>>>>> more IOPS
>>>>>>>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does
>> that
>>>>>>>> make
>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement.
>> I'm
>>>>>> not
>>>>>>>>>> even
>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely
>>>> remain.
>>>>>>>> I'd
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN
>>>> and
>>>>>>>>>> total
>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>>> up,
>>>>>>>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
>>>>>>>>>> supported
>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> the system.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow
>>>> non-QoS
>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS?
>> For
>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>>>>>>>>>>>> think a side effect of the current implementation is
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled
>>>>>> I/O,
>>>>>>>> it
>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
>>>>>>>> side-by-side
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
>>>>>> potentially
>>>>>>>>>>>>>> starve
>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes
>> defined
>>>>>> the
>>>>>>>>>>>>>>>> collocated
>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of
>>>> CS)
>>>>>>>> and
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>> specify
>>>>>>>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will
>> be
>>>> at
>>>>>>>> the
>>>>>>>>>>>>>>>>>> defaults
>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire
>>>>>> SAN,
>>>>>>>>>> you
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage
>>>> tags
>>>>>>>>>>>>>> exclusions
>>>>>>>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
>>>>>>>> deploy
>>>>>>>>>> my
>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list
>>>> as
>>>>>> it
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> complicate component assembly. It would require a
>>>> storage
>>>>>>>>>>>>>> plugin to
>>>>>>>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies
>> and
>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express
>>>> their
>>>>>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>>>>>> which can be matched to a set of requested
>> requirements.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
>>>>>> inclusion
>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
>>>>>> functionality
>>>>>>>>>> in
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> allocator understands.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since
>>>> it
>>>>>>>> only
>>>>>>>>>>>>>>>> limits
>>>>>>>>>>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a
>>>>>> system
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem
>>>> does
>>>>>>>> not
>>>>>>>>>>>>>>>> exist.
>>>>>>>>>>>>>>>>>>>>>> However, I am concerned with the more general case as
>> we
>>>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
>>>>>> vendor-specific
>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>> IOPS, but
>>>>>>>>>>>>>>>>>>>>> what if you break them out by read and write or do
>>>>>> something
>>>>>>>>>>>>>> else? We
>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
>>>>>> throttled
>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> We also need to understand the implications of these
>>>> QoS
>>>>>>>>>>>>>>>> guarantees
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> operation of the system to ensure that the
>> underlying
>>>>>>>>>> hardware
>>>>>>>>>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will
>> likely
>>>>>> be
>>>>>>>>>>>>>> forced
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some
>>>>>> great
>>>>>>>>>>>>>> ideas
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning
>>>>>> (with
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> tags).
>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that -
>> in
>>>>>>>>>>>>>> addition to
>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage
>>>> QoS
>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not
>> available,
>>>>>> but
>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in
>>>> values
>>>>>>>> for
>>>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at
>>>> least
>>>>>>>>>>>>>> yet), so
>>>>>>>>>>>>>>>> he
>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
>>>>>> allocators
>>>>>>>>>> to
>>>>>>>>>>>>>>>> prefer
>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it
>> stands
>>>>>>>> today
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
>>>>>> features
>>>>>>>>>>>>>> (fields
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
>>>>>>>> Perhaps
>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is
>> how
>>>>>>>>>>>>>> OpenStack
>>>>>>>>>>>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two
>>>> weeks
>>>>>>>>>> until
>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 -
>> keep
>>>>>>>> Wei's
>>>>>>>>>>>>>> fields
>>>>>>>>>>>>>>>>>>>>>> and my
>>>>>>>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other
>> feature
>>>>>> has
>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>> entered
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
>>>>>> general-purpose
>>>>>>>>>>>>>> design
>>>>>>>>>>>>>>>>>>>>>> that can
>>>>>>>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have
>> these
>>>>>>>>>>>>>> situations.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I just published my review @
>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before
>>>> coming
>>>>>> up
>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes
>>>> to
>>>>>> be
>>>>>>>>>>>>>> checked
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new
>> patch
>>>>>>>> file?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my
>>>> review.
>>>>>> I
>>>>>>>> am
>>>>>>>>>>>>>>>>>> offline
>>>>>>>>>>>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of
>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
>>>>>> Board?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
>>>>>> reading
>>>>>>>>>>>>>> this :)
>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented
>> all
>>>>>> the
>>>>>>>>>>>>>> areas
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow
>> morning
>>>> or
>>>>>>>>>>>>>>>> afternoon,
>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike
>> Tutkowski
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
>>>>>> conflicts
>>>>>>>>>> in
>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
>>>>>> disk_io_throttling
>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>>>>>>>>>>> object_store
>>>>>>>>>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we
>>>>>> merge
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store
>>>> goes
>>>>>>>>>> into
>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits
>>>>>> into
>>>>>>>>>> this
>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John
>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform.
>>>> The
>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have
>> a
>>>>>>>> number
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies
>>>> between
>>>>>>>> the
>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire ->
>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management
>>>> aspects
>>>>>>>> of
>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
>>>>>> circular
>>>>>>>>>>>>>>>> dependency
>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
>>>>>>>> graph,
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in
>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike
>> Tutkowski
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging
>> my
>>>>>> code
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>> his
>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic
>>>> code
>>>>>>>>>>>>>> after his
>>>>>>>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
>>>>>> Tutkowski
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're
>> going
>>>>>>>> here:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
>>>>>> hypervisor
>>>>>>>>>>>>>> code in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
>>>>>>>> storage
>>>>>>>>>>>>>>>> plug-in,
>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor
>> in
>>>>>> use
>>>>>>>>>>>>>>>> (although
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i),
>> etc.)
>>>> is
>>>>>>>>>>>>>> actually
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
>>>>>> placed
>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> url
>>>>>>>>>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info
>> is
>>>>>>>>>> stored
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to
>> the
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this
>> takes
>>>>>> the
>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> checks
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
>>>>>> necessary
>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach
>> command
>>>>>>>>>>>>>> executes to
>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
>>>>>> invoked
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> detach a
>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
>>>>>> hypervisor
>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support
>>>> Burst
>>>>>>>>>> IOPS
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
>>>>>> table.
>>>>>>>>>> If
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented
>> while
>>>>>>>>>>>>>> avoiding
>>>>>>>>>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
>>>>>>>> details
>>>>>>>>>>>>>>>> field.
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the
>>>> documentation
>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>>>>>>>> details
>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>>>>>>>>>>>> listStoragePool
>>>>>>>>>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it
>> would
>>>>>> be
>>>>>>>>>>>>>> useful
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the
>>>> details
>>>>>>>>>>>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing
>> Burst
>>>>>>>> IOPS
>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>>>>>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created,
>> you
>>>>>>>> have
>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>>>>>>>>>>> disk_offering_details
>>>>>>>>>>>>>>>>>> table.
>>>>>>>>>>>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept
>> the
>>>>>>>> Burst
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields
>> are
>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
We should be on our way to getting a really good 1.0 release out for this,
so that's cool. :)
Thanks for the time...I should be able to find time this weekend to work on
this.
Next week I can look into the GUI and API changes for mutual exclusion of
mine and Wei's features.
On Fri, Jun 14, 2013 at 3:56 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Awesome. +1.
>
> Thanks for your patience with the back and forth,
> -John
>
> On Jun 14, 2013, at 5:50 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Quick comments below in red, but I think we're good to go.
> >
> >
> > On Fri, Jun 14, 2013 at 3:42 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Please see my comments in-line below.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 14, 2013, at 5:37 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Comments below in red (but I believe we are reaching a consensus here).
> >> :)
> >>>
> >>>
> >>> On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Please see my comments in-line below.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> I am OK with that approach, John.
> >>>>>
> >>>>> So, let me review to make sure I follow you correctly:
> >>>>>
> >>>>> We introduce two new parameters to the plug-in: Number of Total IOPS
> >> for
> >>>>> the SAN and an overcommit ratio. (On a side note, if we are just
> >>>>> multiplying the two numbers, why don't we have the user just input
> >> their
> >>>>> product?)
> >>>>
> >>>> It is a straw man suggestion to allow some fine tuning of the
> allocation
> >>>> algorithm. I defer to you to determine whether or not such a tunable
> >> would
> >>>> be valuable in this use case. My thinking was that it is tunable to
> >>>> support operators as they more deeply understand/observe the CS
> workload
> >>>> while allowing them to think about IOPS absolutely. For example, I
> >> start
> >>>> out at say 50% overcommit and realize that the load is actually
> heavier
> >>>> than expected, I can reduce it or vice versa.
> >>>>
> >>>
> >>> Using your example of support people changing the value after seeing
> how
> >>> the system is performing:
> >>>
> >>> If we had them just pass in a new Total IOPS value, that should cover
> it.
> >>
> >> That is fine answer to me. I expect that we will be refining this
> >> capability over the next few releases. Therefore, I am cool with
> starting
> >> as simple as possible as it will ease/simplify future evolution.
> >>
> >>>
> >>>
> >>>>
> >>>>>
> >>>>> Are both of these new parameters to the create storage pool API
> command
> >>>> or
> >>>>> are they passed into the create storage pool API command through its
> >> url
> >>>>> parameter?
> >>>>>
> >>>>> If they are new parameters, we should make two new columns in the
> >>>>> storage_pool table.
> >>>>>
> >>>>> If they are passed in via the url parameter, they should go in the
> >>>>> storage_pool_details table.
> >>>>
> >>>> I think these allocation parameters implementation agnostic, and
> should
> >> be
> >>>> columns on the storage_pool table and first class properties on the
> >>>> StoragePool class.
> >>>>
> >>>
> >>> Per my most recent e-mail, I agree.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>> For 4.2, if someone wants to change these values, they must update
> the
> >> DB
> >>>>> manually.
> >>>>
> >>>> Is it not possible to support updates through the updateStoragePool
> API
> >>>> call?
> >>>>
> >>>
> >>> Edison - if you're reading these - do you know if the plug-ins are
> >> informed
> >>> when the updateStoragePool API command is issued?
> >>
> >> The way I am thinking about these thresholds is that they are part of
> the
> >> storage engine itself. Why would plugin need to know that the value
> >> changed? When the plugin needs the value, it will be retrieved from the
> >> database ...
> >>
> >
> > You know, when I wrote what I wrote above, I was thinking the data was
> > coming in with the url parameter, which has to be parsed by the plug-in
> > (and it's stored in the storage_pool_details table).
> >
> > Since this is a new, standard parameter for the API (stored in the
> > storage_pool table), it should work just fine without plug-in
> interaction.
> >
> >
> >>
> >>>
> >>>
> >>>>
> >>>>>
> >>>>> Every time a volume is created or deleted for the SolidFire plug-in,
> >> the
> >>>>> Current IOPS value (sum of all volumes' Min IOPS that are associated
> >> with
> >>>>> the plug-in) is updated.
> >>>>
> >>>> I don't think we need another column for this computed value. It will
> >> be
> >>>> simpler to simply ask the database to sum the values as needed.
> >>>>
> >>>
> >>> I'm thinking now that we should follow the pattern established by the
> >>> "size" field. There are two related size fields for a storage pool:
> >>> capacity_bytes and available bytes.
> >>>
> >>> We should just follow this pattern and create two new fields:
> >> capacity_iops
> >>> and available_iops. The plug-in can increment or decrement
> available_iops
> >>> each time a volume is created or deleted, respectively.
> >>
> >> I would rather see us do away with available_bytes. My experience with
> >> storing computed fields is that often get out of sync and represent
> >> premature optimization. I missed that part of the throttled I/O patch
> ...
> >>
> >
> > Fair enough. Agreed - one new field in the storage_pool table:
> > capacity_iops.
> >
> >
> >>
> >>>
> >>>
> >>>>
> >>>>>
> >>>>> The allocator can use these fields to determine if it can fit in a
> new
> >>>>> volume.
> >>>>
> >>>> Exactly.
> >>>>
> >>>>>
> >>>>> Does it look like my understanding is OK?
> >>>>
> >>>> Yes, looks good to me.
> >>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> I apologize for not being clear -- this conversation has been
> >> admittedly
> >>>>>> disjoint. I think we should allow the maximum IOPS and
> overcommitment
> >>>>>> values to be updated though I would recommend restricting updates to
> >> to
> >>>> be
> >>>>>> an increasing value for 4.2 (e.g. users can increase the number of
> >> total
> >>>>>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to
> >> 200,000).
> >>>>>> While not ideal, given the amount of time we have left for 4.2, it
> >> will
> >>>>>> cover most cases, and we can address the implications of reducing
> >>>> resource
> >>>>>> capacity in 4.3. This approach addresses both of your concerns.
> >>>> First, it
> >>>>>> allows the administrator/operator to determine what portion of the
> >>>> device
> >>>>>> they wish to dedicate. For example, if the device has a total
> >> capacity
> >>>> of
> >>>>>> 200,000 IOPS, and they only want CS to use 25% of the device then
> they
> >>>> set
> >>>>>> the maximum total IOPS to 50,000. Second, as they grow capacity,
> they
> >>>> can
> >>>>>> update the DataStore to increase the number of IOPS they want to
> >>>> dedicate
> >>>>>> to CS' use. I would imagine expansion of capacity happens
> >> infrequently
> >>>>>> enough that increasing the maximum IOPS value would not be a
> >> significant
> >>>>>> burden.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> "the administrator/operator simply needs to tell us the total
> number
> >> of
> >>>>>>> IOPS that can be committed to it and an overcommitment factor."
> >>>>>>>
> >>>>>>> Are you thinking when we create a plug-in as primary storage that
> we
> >>>> say
> >>>>>> -
> >>>>>>> up front - how many IOPS the SAN can handle?
> >>>>>>>
> >>>>>>> That is not a good move, in my opinion. Our SAN is designed to
> start
> >>>>>> small
> >>>>>>> and grow to PBs. As the need arises for more storage, the admin
> >>>> purchases
> >>>>>>> additional storage nodes that join the cluster and the performance
> >> and
> >>>>>>> capacity go up.
> >>>>>>>
> >>>>>>> We need to know how many IOPS total the SAN can handle and what it
> is
> >>>>>>> committed to currently (the sum of the number of volumes' min
> IOPS).
> >>>>>>>
> >>>>>>> We also cannot assume the SAN is dedicated to CS.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Simon,
> >>>>>>>>
> >>>>>>>> Yes, it is CloudStack's job to protect, as best it can, from
> >>>>>>>> oversubscribing resources. I would argue that resource management
> >> is
> >>>>>> one,
> >>>>>>>> if not the most, important functions of the system. It is no
> >>>> different
> >>>>>>>> than the allocation/planning performed for hosts relative to cores
> >> and
> >>>>>>>> memory. We can still oversubscribe resources, but we have rails +
> >>>> knobs
> >>>>>>>> and dials to avoid it. Without these controls in place, we could
> >>>> easily
> >>>>>>>> allow users to deploy workloads that overrun resources harming all
> >>>>>> tenants.
> >>>>>>>>
> >>>>>>>> I also think that we are over thinking this issue for provisioned
> >>>> IOPS.
> >>>>>>>> When the DataStore is configured, the administrator/operator
> simply
> >>>>>> needs
> >>>>>>>> to tell us the total number of IOPS that can be committed to it
> and
> >> an
> >>>>>>>> overcommitment factor. As we allocate volumes to that DataStore,
> we
> >>>>>> sum up
> >>>>>>>> the committed IOPS of the existing Volumes attached to the
> >> DataStore,
> >>>>>> apply
> >>>>>>>> the overcommitment factor, and determine whether or not the
> >> requested
> >>>>>>>> minimum IOPS for the new volume can be fulfilled. We can provide
> >> both
> >>>>>>>> general and vendor specific documentation for determining these
> >> values
> >>>>>> --
> >>>>>>>> be they to consume the entire device or a portion of it.
> >>>>>>>>
> >>>>>>>> Querying the device is unnecessary and deceptive. CloudStack
> >> resource
> >>>>>>>> management is not interested in the current state of the device
> >> which
> >>>>>> could
> >>>>>>>> be anywhere from extremely heavy to extremely light at any given
> >> time.
> >>>>>> We
> >>>>>>>> are interested in the worst case load that is anticipated for
> >>>> resource.
> >>>>>> In
> >>>>>>>> my view, it is up to administrators/operators to instrument their
> >>>>>>>> environment to understand usage patterns and capacity. We should
> >>>>>> provide
> >>>>>>>> information that will help determine what should be
> >>>>>> instrumented/monitored,
> >>>>>>>> but that function should be performed outside of CloudStack.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com>
> wrote:
> >>>>>>>>
> >>>>>>>>> I'd like to comment on this briefly.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I think an assumption is being made that the SAN is being
> dedicated
> >>>> to
> >>>>>> a
> >>>>>>>> CS instance.
> >>>>>>>>>
> >>>>>>>>> My person opinion that this whole IOPS calculation is getting
> >> rather
> >>>>>>>> complicated, and could probably be much simpler than this. Over
> >>>>>>>> subscription is a fact of life on virtually all storage, and is
> >> really
> >>>>>> no
> >>>>>>>> different in concept than multiple virt instances on a single
> piece
> >> of
> >>>>>>>> hardware. All decent SANs offer many management options for the
> >>>> storage
> >>>>>>>> engineers to keep track of IOPS utilization, and plan for spindle
> >>>>>>>> augmentation as required.
> >>>>>>>>> Is it really the job of CS to become yet another management layer
> >> on
> >>>>>> top
> >>>>>>>> of this?
> >>>>>>>>>
> >>>>>>>>> ----- Original Message -----
> >>>>>>>>>
> >>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
> >>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
> >>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>
> >>>>>>>>> 1) We want number of IOPS currently supported by the SAN.
> >>>>>>>>>
> >>>>>>>>> 2) We want the number of IOPS that are committed (sum of min IOPS
> >> for
> >>>>>>>> each
> >>>>>>>>> volume).
> >>>>>>>>>
> >>>>>>>>> We could do the following to keep track of IOPS:
> >>>>>>>>>
> >>>>>>>>> The plug-in could have a timer thread that goes off every, say, 1
> >>>>>> minute.
> >>>>>>>>>
> >>>>>>>>> It could query the SAN for the number of nodes that make up the
> SAN
> >>>> and
> >>>>>>>>> multiple this by 50,000. This is essentially the number of
> >> supported
> >>>>>> IOPS
> >>>>>>>>> of the SAN.
> >>>>>>>>>
> >>>>>>>>> The next API call could be to get all of the volumes on the SAN.
> >>>>>> Iterate
> >>>>>>>>> through them all and add up their min IOPS values. This is the
> >> number
> >>>>>> of
> >>>>>>>>> IOPS the SAN is committed to.
> >>>>>>>>>
> >>>>>>>>> These two numbers can then be updated in the storage_pool table
> (a
> >>>>>> column
> >>>>>>>>> for each value).
> >>>>>>>>>
> >>>>>>>>> The allocators can get these values as needed (and they would be
> as
> >>>>>>>>> accurate as the last time the thread asked the SAN for this
> info).
> >>>>>>>>>
> >>>>>>>>> These two fields, the min IOPS of the volume to create, and the
> >>>>>>>> overcommit
> >>>>>>>>> ratio of the plug-in would tell the allocator if it can select
> the
> >>>>>> given
> >>>>>>>>> storage pool.
> >>>>>>>>>
> >>>>>>>>> What do you think?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> "As I mentioned previously, I am very reluctant for any feature
> to
> >>>>>> come
> >>>>>>>>>> into master that can exhaust resources."
> >>>>>>>>>>
> >>>>>>>>>> Just wanted to mention that, worst case, the SAN would fail
> >> creation
> >>>>>> of
> >>>>>>>>>> the volume before allowing a new volume to break the system.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi John,
> >>>>>>>>>>>
> >>>>>>>>>>> Are you thinking we add a column on to the storage pool table,
> >>>>>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
> >>>>>>>>>>>
> >>>>>>>>>>> That is easy enough.
> >>>>>>>>>>>
> >>>>>>>>>>> How do you want to determine what the SAN is capable of
> >> supporting
> >>>>>> IOPS
> >>>>>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you
> add
> >>>>>>>> storage
> >>>>>>>>>>> nodes to the cluster, the number of IOPS increases. Do we have
> a
> >>>>>>>> thread we
> >>>>>>>>>>> can use to query external devices like this SAN to update the
> >>>>>> supported
> >>>>>>>>>>> number of IOPS?
> >>>>>>>>>>>
> >>>>>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in
> an
> >>>>>>>>>>> overcommit ration to the plug-in when it's created? We would
> need
> >>>> to
> >>>>>>>> store
> >>>>>>>>>>> this in the storage_pool table, as well, I believe.
> >>>>>>>>>>>
> >>>>>>>>>>> We should also get Wei involved in this as his feature will
> need
> >>>>>>>> similar
> >>>>>>>>>>> functionality.
> >>>>>>>>>>>
> >>>>>>>>>>> Also, we should do this FAST as we have only two weeks left and
> >>>> many
> >>>>>> of
> >>>>>>>>>>> us will be out for several days for the CS Collab Conference.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <
> >> jburwell@basho.com
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Querying the SAN only indicates the number of IOPS currently
> in
> >>>> use.
> >>>>>>>>>>>> The allocator needs to check the number of IOPS committed
> which
> >> is
> >>>>>>>> tracked
> >>>>>>>>>>>> by CloudStack. For 4.2, we should be able to query the number
> of
> >>>>>> IOPS
> >>>>>>>>>>>> committed to a DataStore, and determine whether or not the
> >> number
> >>>>>>>> requested
> >>>>>>>>>>>> can be fulfilled by that device. It seems to be that a
> >>>>>>>>>>>> DataStore#getCommittedIOPS() : Long method would be
> sufficient.
> >>>>>>>>>>>> DataStore's that don't support provisioned IOPS would return
> >> null.
> >>>>>>>>>>>>
> >>>>>>>>>>>> As I mentioned previously, I am very reluctant for any feature
> >> to
> >>>>>> come
> >>>>>>>>>>>> into master that can exhaust resources.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Yeah, I'm not sure I could come up with anything near an
> >> accurate
> >>>>>>>>>>>>> assessment of how many IOPS are currently available on the
> SAN
> >>>> (or
> >>>>>>>>>>>> even a
> >>>>>>>>>>>>> total number that are available for volumes). Not sure if
> >> there's
> >>>>>> yet
> >>>>>>>>>>>> an
> >>>>>>>>>>>>> API call for that.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If I did know this number (total number of IOPS supported by
> >> the
> >>>>>>>> SAN),
> >>>>>>>>>>>> we'd
> >>>>>>>>>>>>> still have to keep track of the total number of volumes we've
> >>>>>> created
> >>>>>>>>>>>> from
> >>>>>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin
> issues
> >> an
> >>>>>> API
> >>>>>>>>>>>> call
> >>>>>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given
> >> volume
> >>>>>> or
> >>>>>>>>>>>> set of
> >>>>>>>>>>>>> volumes (not supported from CS, but supported via the
> SolidFire
> >>>>>> API),
> >>>>>>>>>>>> our
> >>>>>>>>>>>>> numbers in CS would be off.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a really
> >> good
> >>>>>>>> idea
> >>>>>>>>>>>> for
> >>>>>>>>>>>>> a future release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> For 4.2 I think we should stick to having the allocator
> detect
> >> if
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>> QoS is desired and if the storage pool in question supports
> >> that
> >>>>>>>>>>>> feature.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If you really are over provisioned on your SAN in terms of
> IOPS
> >>>> or
> >>>>>>>>>>>>> capacity, the SAN can let the admin know in several different
> >>>> ways
> >>>>>>>>>>>> (e-mail,
> >>>>>>>>>>>>> SNMP, GUI).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
> >>>> jburwell@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please see my comments in-line below.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However,
> >> we
> >>>>>> may
> >>>>>>>>>>>> want
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> throw an exception if we can not fulfill a requested QoS.
> If
> >>>> the
> >>>>>>>>>>>> user
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> expecting that the hypervisor will provide a particular
> QoS,
> >>>> and
> >>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>> not possible, it seems like we should inform them rather
> >> than
> >>>>>>>>>>>> silently
> >>>>>>>>>>>>>>>> ignoring their request.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sure, that sounds reasonable.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> We'd have to come up with some way for the allocators to
> know
> >>>>>> about
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> requested storage QoS and then query the candidate drivers.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Any thoughts on how we might do that?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> To collect my thoughts from previous parts of the thread,
> I
> >> am
> >>>>>>>>>>>>>>>> uncomfortable with the idea that the management server can
> >>>>>>>>>>>> overcommit a
> >>>>>>>>>>>>>>>> resource. You had mentioned querying the device for
> >> available
> >>>>>>>> IOPS.
> >>>>>>>>>>>>>> While
> >>>>>>>>>>>>>>>> that would be nice, it seems like we could fall back to a
> >> max
> >>>>>> IOPS
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> overcommit factor manually calculated and entered by the
> >>>>>>>>>>>>>>>> administrator/operator. I think such threshold and
> >> allocation
> >>>>>>>> rails
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it
> >> is
> >>>> a
> >>>>>>>>>>>> basic
> >>>>>>>>>>>>>>>> feature of any cloud orchestration platform.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Are you thinking this ability would make it into 4.2? Just
> >>>>>> curious
> >>>>>>>>>>>> what
> >>>>>>>>>>>>>>> release we're talking about here. For the SolidFire SAN,
> you
> >>>>>> might
> >>>>>>>>>>>> have,
> >>>>>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS)
> and
> >>>> then
> >>>>>>>>>>>> later
> >>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to
> >> have
> >>>> a
> >>>>>>>> way
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> know that the number of supported IOPS has increased.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2.
> >> For
> >>>>>>>>>>>> example,
> >>>>>>>>>>>>>> we may only support expanding capacity in 4.2, and not
> handle
> >>>> any
> >>>>>>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
> >>>> Extrapolating
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a
> >> similar
> >>>>>> set
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>> rails defined per host.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be
> expressed
> >>>> in
> >>>>>> a
> >>>>>>>>>>>>>>>> implementation specific manner. I think we need to arrive
> >> at a
> >>>>>>>>>>>> general
> >>>>>>>>>>>>>>>> model that can be exploited by the allocators and
> planners.
> >> We
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> restrict implementation specific key-value pairs (call
> them
> >>>>>>>> details,
> >>>>>>>>>>>>>>>> extended data, whatever) to information that is unique to
> >> the
> >>>>>>>>>>>> driver and
> >>>>>>>>>>>>>>>> would provide no useful information to the management
> >> server's
> >>>>>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to
> fit
> >>>>>> those
> >>>>>>>>>>>>>>>> criteria.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I wonder if this would be a good discussion topic for
> >> Sunday's
> >>>> CS
> >>>>>>>>>>>> Collab
> >>>>>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes, it would -- I will put something in the wiki topic. It
> >> will
> >>>>>>>>>>>> also be
> >>>>>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
> >>>> include
> >>>>>>>>>>>> some of
> >>>>>>>>>>>>>> my opinions on the topic.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> So, here's my suggestion for 4.2:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Accept the values as they are currently required (four
> new
> >>>>>> fields
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>>>> feature or two new fields for mine).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio
> buttons:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1) No QoS
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2) Hypervisor QoS
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 3) Storage Qos
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The admin needs to specify storage tags that only map to
> >>>>>> storage
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> supports Storage QoS.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that
> unless
> >>>> all
> >>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>> in use support the new fields, they may not be enforced.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Post 4.3:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Come up with a way to more generally enter these
> parameters
> >>>>>>>>>>>> (probably
> >>>>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>> key/value pairs sent to the drivers).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Have the drivers expose their feature set so the
> allocators
> >>>> can
> >>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>> them more fully and throw an exception if there is not a
> >>>>>>>> sufficient
> >>>>>>>>>>>>>>>> match.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need
> to
> >>>> put
> >>>>>>>> some
> >>>>>>>>>>>>>>>> burden
> >>>>>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that
> >>>> makes
> >>>>>>>>>>>> sense.
> >>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported
> Min
> >>>> and
> >>>>>>>> Max
> >>>>>>>>>>>>>>>> values,
> >>>>>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
> >>>>>> SolidFire
> >>>>>>>>>>>>>> primary
> >>>>>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that
> >> doesn't,
> >>>>>>>> then
> >>>>>>>>>>>> he's
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>> going to get the Min and Max feature. We could add help
> >> text
> >>>>>> to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> pop-up
> >>>>>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and
> Max
> >>>> text
> >>>>>>>>>>>> fields.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Same idea for Wei's feature.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
> >>>>>>>> comprehensive
> >>>>>>>>>>>>>>>>>> approach for post 4.2.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Maybe in the future we could have the drivers advertise
> >>>> their
> >>>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>> and if the allocator feels a request is not being
> >> satisfied
> >>>>>> (say
> >>>>>>>>>>>> Min
> >>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>> entered, but it not's supported by any storage plug-in)
> it
> >>>> can
> >>>>>>>>>>>> throw
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>> exception.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Please see my comment in-line below.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I've put comments below in red.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
> >>>> features
> >>>>>>>> are
> >>>>>>>>>>>>>>>>>>>> mutually
> >>>>>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a
> >> configuration
> >>>>>> that
> >>>>>>>>>>>>>>>>>>>> contains both
> >>>>>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >>>>>>>>>>>>>>>> non-deterministic
> >>>>>>>>>>>>>>>>>>>>>> operations.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor
> or
> >>>>>>>> storage
> >>>>>>>>>>>>>>>> oriented
> >>>>>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
> >>>> storage
> >>>>>>>>>>>> tags,
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>> have no
> >>>>>>>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
> >>>>>> wasn't
> >>>>>>>>>>>>>>>>>>>> necessarily
> >>>>>>>>>>>>>>>>>>>>>> intending that we implement this allocation model in
> >>>> 4.3,
> >>>>>>>> but
> >>>>>>>>>>>>>>>> instead,
> >>>>>>>>>>>>>>>>>>>>>> determine if this type model would be one we would
> >> want
> >>>> to
> >>>>>>>>>>>> support
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that
> the
> >>>>>> data
> >>>>>>>>>>>> model
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> current implementation would not preclude evolution
> in
> >>>>>> that
> >>>>>>>>>>>>>>>>>>>> direction. My
> >>>>>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
> >>>> user's
> >>>>>>>>>>>> expect
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> "cloud"
> >>>>>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource set
> >> to
> >>>>>>>>>>>> fulfill a
> >>>>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>>>> system requirements.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in that
> >> what
> >>>>>>>> we've
> >>>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>> should not make refinement difficult in the future.
> If
> >> we
> >>>>>>>> don't
> >>>>>>>>>>>>>>>> modify
> >>>>>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
> >>>>>> relatively
> >>>>>>>>>>>> simple
> >>>>>>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
> >>>>>>>> aggregated
> >>>>>>>>>>>>>> result
> >>>>>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes
> for
> >>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>> I/O and
> >>>>>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
> >>>>>> concern
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At
> >> minimum,
> >>>>>>>>>>>> Javadoc
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and
> their
> >>>>>>>> mutual
> >>>>>>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their
> mutual
> >>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The other part that is interesting is that throttled
> I/O
> >>>>>>>>>>>> provides
> >>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where
> >> provisioned
> >>>>>> IOPS
> >>>>>>>>>>>> uses
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>> range. In order to select the best available resource
> to
> >>>>>>>>>>>> fulfill a
> >>>>>>>>>>>>>>>> QoS, we
> >>>>>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in
> >> terms
> >>>> of
> >>>>>>>>>>>> units
> >>>>>>>>>>>>>>>> (IOPS or
> >>>>>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we
> want
> >>>> to
> >>>>>>>>>>>>>> achieve a
> >>>>>>>>>>>>>>>>>>>> model like I described, I think we would need to
> resolve
> >>>>>> this
> >>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>> in 4.2
> >>>>>>>>>>>>>>>>>>>> to ensure a viable migration path.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the
> >> input
> >>>>>> for
> >>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>> Plus
> >>>>>>>>>>>>>>>>>>> people probably want to input the data in terms they're
> >>>>>>>> familiar
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> the products in question.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and
> >> let
> >>>>>> the
> >>>>>>>>>>>> user
> >>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected
> due
> >>>> to
> >>>>>> a
> >>>>>>>>>>>> given
> >>>>>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it
> >> because
> >>>>>>>> what
> >>>>>>>>>>>>>>>> happens if
> >>>>>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage
> >> after
> >>>>>>>>>>>> having
> >>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>> a Disk Offering?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we
> should
> >>>>>>>>>>>> re-think
> >>>>>>>>>>>>>> it.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose
> >> their
> >>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> As I think through the implications of these
> >>>> requirements
> >>>>>>>> and
> >>>>>>>>>>>>>>>> reflect
> >>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't
> >> already
> >>>>>>>>>>>> impacted
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
> >>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>> has no
> >>>>>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
> >>>> devices
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore,
> as I
> >>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources
> --
> >>>>>>>>>>>> potentially
> >>>>>>>>>>>>>>>>>>>> causing
> >>>>>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It
> >>>> seems
> >>>>>>>>>>>> to me
> >>>>>>>>>>>>>>>>>>>> that a
> >>>>>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express
> >> the
> >>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment
> factor.
> >>>>>> This
> >>>>>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>>>>> should be used by the storage allocators to
> determine
> >>>> the
> >>>>>>>>>>>> device
> >>>>>>>>>>>>>>>> best
> >>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems
> >>>> that
> >>>>>> a
> >>>>>>>>>>>>>> similar
> >>>>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>>>> considerations would need to be added to the
> >> Hypervisor
> >>>>>>>> layer
> >>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent
> oversubscription.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the
> path
> >>>>>> that
> >>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>> previously
> >>>>>>>>>>>>>>>>>>>>> established for other properties you see on dialogs
> in
> >>>> CS:
> >>>>>>>> Just
> >>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is
> >> deployed
> >>>>>> to
> >>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs,
> >> say, a
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> confines the deployment only to storage that supports
> >> the
> >>>>>>>>>>>> selected
> >>>>>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
> >>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>> works
> >>>>>>>>>>>>>>>>>>>>> today.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for
> the
> >>>>>>>>>>>>>> administrator
> >>>>>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is
> >> that
> >>>> we
> >>>>>>>>>>>> over
> >>>>>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the
> purposes
> >> of
> >>>>>>>>>>>>>>>> illustration,
> >>>>>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for
> that
> >>>>>>>> device
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> 100000.
> >>>>>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
> >>>>>>>>>>>> Therefore,
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than
> 150,000
> >>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the
> >> DataStore
> >>>>>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
> >>>> factor.
> >>>>>>>>>>>> As we
> >>>>>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure
> that
> >>>>>>>>>>>> guarantee
> >>>>>>>>>>>>>>>> more IOPS
> >>>>>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does
> that
> >>>>>> make
> >>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement.
> I'm
> >>>> not
> >>>>>>>> even
> >>>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely
> >> remain.
> >>>>>> I'd
> >>>>>>>>>>>> have
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN
> >> and
> >>>>>>>> total
> >>>>>>>>>>>>>> them
> >>>>>>>>>>>>>>>> up,
> >>>>>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
> >>>>>>>> supported
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> the system.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow
> >> non-QoS
> >>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS?
> For
> >>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>> IOPS, I
> >>>>>>>>>>>>>>>>>>>>>> think a side effect of the current implementation is
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled
> >>>> I/O,
> >>>>>> it
> >>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
> >>>>>> side-by-side
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
> >>>> potentially
> >>>>>>>>>>>> starve
> >>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes
> defined
> >>>> the
> >>>>>>>>>>>>>> collocated
> >>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>> from being fulfilled.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of
> >> CS)
> >>>>>> and
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>> specify
> >>>>>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will
> be
> >> at
> >>>>>> the
> >>>>>>>>>>>>>>>> defaults
> >>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire
> >>>> SAN,
> >>>>>>>> you
> >>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>> your Mins met.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage
> >> tags
> >>>>>>>>>>>> exclusions
> >>>>>>>>>>>>>>>>>>>> list,
> >>>>>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
> >>>>>> deploy
> >>>>>>>> my
> >>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> storage with these following tags).
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list
> >> as
> >>>> it
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>> complicate component assembly. It would require a
> >> storage
> >>>>>>>>>>>> plugin to
> >>>>>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies
> and
> >>>>>>>>>>>> determine
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express
> >> their
> >>>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>>>> which can be matched to a set of requested
> requirements.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
> >>>> inclusion
> >>>>>>>> or
> >>>>>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
> >>>> functionality
> >>>>>>>> in
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> allocator understands.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since
> >> it
> >>>>>> only
> >>>>>>>>>>>>>> limits
> >>>>>>>>>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a
> >>>> system
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>>>>>> minimum IOPS does this.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem
> >> does
> >>>>>> not
> >>>>>>>>>>>>>> exist.
> >>>>>>>>>>>>>>>>>>>> However, I am concerned with the more general case as
> we
> >>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>> devices with provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
> >>>> vendor-specific
> >>>>>>>>>>>> values
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
> >>>>>>>> provisioned
> >>>>>>>>>>>>>>>> IOPS, but
> >>>>>>>>>>>>>>>>>>> what if you break them out by read and write or do
> >>>> something
> >>>>>>>>>>>> else? We
> >>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>> a more general mechanism.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
> >>>> throttled
> >>>>>>>>>>>> I/O
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
> >>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> We also need to understand the implications of these
> >> QoS
> >>>>>>>>>>>>>> guarantees
> >>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>> operation of the system to ensure that the
> underlying
> >>>>>>>> hardware
> >>>>>>>>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will
> likely
> >>>> be
> >>>>>>>>>>>> forced
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
> >>>>>>>>>>>> implementation
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> 4.3.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some
> >>>> great
> >>>>>>>>>>>> ideas
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>> 4.3. :)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning
> >>>> (with
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> tags).
> >>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that -
> in
> >>>>>>>>>>>> addition to
> >>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage
> >> QoS
> >>>>>>>> being
> >>>>>>>>>>>>>>>>>>>> preferred,
> >>>>>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not
> available,
> >>>> but
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>>>>>> is).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in
> >> values
> >>>>>> for
> >>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at
> >> least
> >>>>>>>>>>>> yet), so
> >>>>>>>>>>>>>> he
> >>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
> >>>> allocators
> >>>>>>>> to
> >>>>>>>>>>>>>> prefer
> >>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it
> stands
> >>>>>> today
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
> >>>> features
> >>>>>>>>>>>> (fields
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
> >>>>>> Perhaps
> >>>>>>>>>>>> post
> >>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is
> how
> >>>>>>>>>>>> OpenStack
> >>>>>>>>>>>>>>>>>>>> addresses
> >>>>>>>>>>>>>>>>>>>>>>> the issue).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two
> >> weeks
> >>>>>>>> until
> >>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>> freeze)
> >>>>>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 -
> keep
> >>>>>> Wei's
> >>>>>>>>>>>> fields
> >>>>>>>>>>>>>>>>>>>> and my
> >>>>>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other
> feature
> >>>> has
> >>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>> entered
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
> >>>> general-purpose
> >>>>>>>>>>>> design
> >>>>>>>>>>>>>>>>>>>> that can
> >>>>>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have
> these
> >>>>>>>>>>>> situations.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I just published my review @
> >>>>>>>>>>>>>> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before
> >> coming
> >>>> up
> >>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>> file.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes
> >> to
> >>>> be
> >>>>>>>>>>>> checked
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new
> patch
> >>>>>> file?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my
> >> review.
> >>>> I
> >>>>>> am
> >>>>>>>>>>>>>>>> offline
> >>>>>>>>>>>>>>>>>>>> ATM,
> >>>>>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of
> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
> >>>> Board?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
> >>>> reading
> >>>>>>>>>>>> this :)
> >>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented
> all
> >>>> the
> >>>>>>>>>>>> areas
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>> wanted
> >>>>>>>>>>>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow
> morning
> >> or
> >>>>>>>>>>>>>> afternoon,
> >>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike
> Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
> >>>> conflicts
> >>>>>>>> in
> >>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>>>>> code,
> >>>>>>>>>>>>>>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
> >>>> disk_io_throttling
> >>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>>>>>>>>>> object_store
> >>>>>>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we
> >>>> merge
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store
> >> goes
> >>>>>>>> into
> >>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits
> >>>> into
> >>>>>>>> this
> >>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John
> >> Burwell <
> >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform.
> >> The
> >>>>>>>>>>>>>>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have
> a
> >>>>>> number
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies
> >> between
> >>>>>> the
> >>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire ->
> >> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management
> >> aspects
> >>>>>> of
> >>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
> >>>> circular
> >>>>>>>>>>>>>> dependency
> >>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
> >>>>>> graph,
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in
> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike
> Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging
> my
> >>>> code
> >>>>>>>>>>>> into
> >>>>>>>>>>>>>> his
> >>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic
> >> code
> >>>>>>>>>>>> after his
> >>>>>>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
> >>>> Tutkowski
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're
> going
> >>>>>> here:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
> >>>> hypervisor
> >>>>>>>>>>>> code in
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
> >>>>>> storage
> >>>>>>>>>>>>>> plug-in,
> >>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor
> in
> >>>> use
> >>>>>>>>>>>>>> (although
> >>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i),
> etc.)
> >> is
> >>>>>>>>>>>> actually
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
> >>>> placed
> >>>>>> in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> url
> >>>>>>>>>>>>>>>>>>>> field
> >>>>>>>>>>>>>>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info
> is
> >>>>>>>> stored
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to
> the
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this
> takes
> >>>> the
> >>>>>>>>>>>> place
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
> >>>>>> hypervisor
> >>>>>>>>>>>>>> checks
> >>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
> >>>> necessary
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach
> command
> >>>>>>>>>>>> executes to
> >>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
> >>>> invoked
> >>>>>>>> to
> >>>>>>>>>>>>>>>> detach a
> >>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
> >>>> hypervisor
> >>>>>>>> data
> >>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support
> >> Burst
> >>>>>>>> IOPS
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
> >>>> table.
> >>>>>>>> If
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented
> while
> >>>>>>>>>>>> avoiding
> >>>>>>>>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
> >>>>>> details
> >>>>>>>>>>>>>> field.
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the
> >> documentation
> >>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
> >>>>>> details
> >>>>>>>>>>>> are
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >>>>>>>>>>>>>> listStoragePool
> >>>>>>>>>>>>>>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it
> would
> >>>> be
> >>>>>>>>>>>> useful
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the
> >> details
> >>>>>>>>>>>> field."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing
> Burst
> >>>>>> IOPS
> >>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>>>>>>>>>>>>>>> Offering-by-Disk
> >>>>>>>>>>>>>>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created,
> you
> >>>>>> have
> >>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>>>>>>>>>> disk_offering_details
> >>>>>>>>>>>>>>>> table.
> >>>>>>>>>>>>>>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept
> the
> >>>>>> Burst
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields
> are
> >>>> in
> >>>>>>>> the
> >>>>>>>>>>>> DB.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Awesome. +1.
Thanks for your patience with the back and forth,
-John
On Jun 14, 2013, at 5:50 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Quick comments below in red, but I think we're good to go.
>
>
> On Fri, Jun 14, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Please see my comments in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 14, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Comments below in red (but I believe we are reaching a consensus here).
>> :)
>>>
>>>
>>> On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> Please see my comments in-line below.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> I am OK with that approach, John.
>>>>>
>>>>> So, let me review to make sure I follow you correctly:
>>>>>
>>>>> We introduce two new parameters to the plug-in: Number of Total IOPS
>> for
>>>>> the SAN and an overcommit ratio. (On a side note, if we are just
>>>>> multiplying the two numbers, why don't we have the user just input
>> their
>>>>> product?)
>>>>
>>>> It is a straw man suggestion to allow some fine tuning of the allocation
>>>> algorithm. I defer to you to determine whether or not such a tunable
>> would
>>>> be valuable in this use case. My thinking was that it is tunable to
>>>> support operators as they more deeply understand/observe the CS workload
>>>> while allowing them to think about IOPS absolutely. For example, I
>> start
>>>> out at say 50% overcommit and realize that the load is actually heavier
>>>> than expected, I can reduce it or vice versa.
>>>>
>>>
>>> Using your example of support people changing the value after seeing how
>>> the system is performing:
>>>
>>> If we had them just pass in a new Total IOPS value, that should cover it.
>>
>> That is fine answer to me. I expect that we will be refining this
>> capability over the next few releases. Therefore, I am cool with starting
>> as simple as possible as it will ease/simplify future evolution.
>>
>>>
>>>
>>>>
>>>>>
>>>>> Are both of these new parameters to the create storage pool API command
>>>> or
>>>>> are they passed into the create storage pool API command through its
>> url
>>>>> parameter?
>>>>>
>>>>> If they are new parameters, we should make two new columns in the
>>>>> storage_pool table.
>>>>>
>>>>> If they are passed in via the url parameter, they should go in the
>>>>> storage_pool_details table.
>>>>
>>>> I think these allocation parameters implementation agnostic, and should
>> be
>>>> columns on the storage_pool table and first class properties on the
>>>> StoragePool class.
>>>>
>>>
>>> Per my most recent e-mail, I agree.
>>>
>>>
>>>>
>>>>>
>>>>> For 4.2, if someone wants to change these values, they must update the
>> DB
>>>>> manually.
>>>>
>>>> Is it not possible to support updates through the updateStoragePool API
>>>> call?
>>>>
>>>
>>> Edison - if you're reading these - do you know if the plug-ins are
>> informed
>>> when the updateStoragePool API command is issued?
>>
>> The way I am thinking about these thresholds is that they are part of the
>> storage engine itself. Why would plugin need to know that the value
>> changed? When the plugin needs the value, it will be retrieved from the
>> database ...
>>
>
> You know, when I wrote what I wrote above, I was thinking the data was
> coming in with the url parameter, which has to be parsed by the plug-in
> (and it's stored in the storage_pool_details table).
>
> Since this is a new, standard parameter for the API (stored in the
> storage_pool table), it should work just fine without plug-in interaction.
>
>
>>
>>>
>>>
>>>>
>>>>>
>>>>> Every time a volume is created or deleted for the SolidFire plug-in,
>> the
>>>>> Current IOPS value (sum of all volumes' Min IOPS that are associated
>> with
>>>>> the plug-in) is updated.
>>>>
>>>> I don't think we need another column for this computed value. It will
>> be
>>>> simpler to simply ask the database to sum the values as needed.
>>>>
>>>
>>> I'm thinking now that we should follow the pattern established by the
>>> "size" field. There are two related size fields for a storage pool:
>>> capacity_bytes and available bytes.
>>>
>>> We should just follow this pattern and create two new fields:
>> capacity_iops
>>> and available_iops. The plug-in can increment or decrement available_iops
>>> each time a volume is created or deleted, respectively.
>>
>> I would rather see us do away with available_bytes. My experience with
>> storing computed fields is that often get out of sync and represent
>> premature optimization. I missed that part of the throttled I/O patch ...
>>
>
> Fair enough. Agreed - one new field in the storage_pool table:
> capacity_iops.
>
>
>>
>>>
>>>
>>>>
>>>>>
>>>>> The allocator can use these fields to determine if it can fit in a new
>>>>> volume.
>>>>
>>>> Exactly.
>>>>
>>>>>
>>>>> Does it look like my understanding is OK?
>>>>
>>>> Yes, looks good to me.
>>>>
>>>>>
>>>>>
>>>>> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> I apologize for not being clear -- this conversation has been
>> admittedly
>>>>>> disjoint. I think we should allow the maximum IOPS and overcommitment
>>>>>> values to be updated though I would recommend restricting updates to
>> to
>>>> be
>>>>>> an increasing value for 4.2 (e.g. users can increase the number of
>> total
>>>>>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to
>> 200,000).
>>>>>> While not ideal, given the amount of time we have left for 4.2, it
>> will
>>>>>> cover most cases, and we can address the implications of reducing
>>>> resource
>>>>>> capacity in 4.3. This approach addresses both of your concerns.
>>>> First, it
>>>>>> allows the administrator/operator to determine what portion of the
>>>> device
>>>>>> they wish to dedicate. For example, if the device has a total
>> capacity
>>>> of
>>>>>> 200,000 IOPS, and they only want CS to use 25% of the device then they
>>>> set
>>>>>> the maximum total IOPS to 50,000. Second, as they grow capacity, they
>>>> can
>>>>>> update the DataStore to increase the number of IOPS they want to
>>>> dedicate
>>>>>> to CS' use. I would imagine expansion of capacity happens
>> infrequently
>>>>>> enough that increasing the maximum IOPS value would not be a
>> significant
>>>>>> burden.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> "the administrator/operator simply needs to tell us the total number
>> of
>>>>>>> IOPS that can be committed to it and an overcommitment factor."
>>>>>>>
>>>>>>> Are you thinking when we create a plug-in as primary storage that we
>>>> say
>>>>>> -
>>>>>>> up front - how many IOPS the SAN can handle?
>>>>>>>
>>>>>>> That is not a good move, in my opinion. Our SAN is designed to start
>>>>>> small
>>>>>>> and grow to PBs. As the need arises for more storage, the admin
>>>> purchases
>>>>>>> additional storage nodes that join the cluster and the performance
>> and
>>>>>>> capacity go up.
>>>>>>>
>>>>>>> We need to know how many IOPS total the SAN can handle and what it is
>>>>>>> committed to currently (the sum of the number of volumes' min IOPS).
>>>>>>>
>>>>>>> We also cannot assume the SAN is dedicated to CS.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Simon,
>>>>>>>>
>>>>>>>> Yes, it is CloudStack's job to protect, as best it can, from
>>>>>>>> oversubscribing resources. I would argue that resource management
>> is
>>>>>> one,
>>>>>>>> if not the most, important functions of the system. It is no
>>>> different
>>>>>>>> than the allocation/planning performed for hosts relative to cores
>> and
>>>>>>>> memory. We can still oversubscribe resources, but we have rails +
>>>> knobs
>>>>>>>> and dials to avoid it. Without these controls in place, we could
>>>> easily
>>>>>>>> allow users to deploy workloads that overrun resources harming all
>>>>>> tenants.
>>>>>>>>
>>>>>>>> I also think that we are over thinking this issue for provisioned
>>>> IOPS.
>>>>>>>> When the DataStore is configured, the administrator/operator simply
>>>>>> needs
>>>>>>>> to tell us the total number of IOPS that can be committed to it and
>> an
>>>>>>>> overcommitment factor. As we allocate volumes to that DataStore, we
>>>>>> sum up
>>>>>>>> the committed IOPS of the existing Volumes attached to the
>> DataStore,
>>>>>> apply
>>>>>>>> the overcommitment factor, and determine whether or not the
>> requested
>>>>>>>> minimum IOPS for the new volume can be fulfilled. We can provide
>> both
>>>>>>>> general and vendor specific documentation for determining these
>> values
>>>>>> --
>>>>>>>> be they to consume the entire device or a portion of it.
>>>>>>>>
>>>>>>>> Querying the device is unnecessary and deceptive. CloudStack
>> resource
>>>>>>>> management is not interested in the current state of the device
>> which
>>>>>> could
>>>>>>>> be anywhere from extremely heavy to extremely light at any given
>> time.
>>>>>> We
>>>>>>>> are interested in the worst case load that is anticipated for
>>>> resource.
>>>>>> In
>>>>>>>> my view, it is up to administrators/operators to instrument their
>>>>>>>> environment to understand usage patterns and capacity. We should
>>>>>> provide
>>>>>>>> information that will help determine what should be
>>>>>> instrumented/monitored,
>>>>>>>> but that function should be performed outside of CloudStack.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>>>>>>>>
>>>>>>>>> I'd like to comment on this briefly.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I think an assumption is being made that the SAN is being dedicated
>>>> to
>>>>>> a
>>>>>>>> CS instance.
>>>>>>>>>
>>>>>>>>> My person opinion that this whole IOPS calculation is getting
>> rather
>>>>>>>> complicated, and could probably be much simpler than this. Over
>>>>>>>> subscription is a fact of life on virtually all storage, and is
>> really
>>>>>> no
>>>>>>>> different in concept than multiple virt instances on a single piece
>> of
>>>>>>>> hardware. All decent SANs offer many management options for the
>>>> storage
>>>>>>>> engineers to keep track of IOPS utilization, and plan for spindle
>>>>>>>> augmentation as required.
>>>>>>>>> Is it really the job of CS to become yet another management layer
>> on
>>>>>> top
>>>>>>>> of this?
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>>
>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>
>>>>>>>>> 1) We want number of IOPS currently supported by the SAN.
>>>>>>>>>
>>>>>>>>> 2) We want the number of IOPS that are committed (sum of min IOPS
>> for
>>>>>>>> each
>>>>>>>>> volume).
>>>>>>>>>
>>>>>>>>> We could do the following to keep track of IOPS:
>>>>>>>>>
>>>>>>>>> The plug-in could have a timer thread that goes off every, say, 1
>>>>>> minute.
>>>>>>>>>
>>>>>>>>> It could query the SAN for the number of nodes that make up the SAN
>>>> and
>>>>>>>>> multiple this by 50,000. This is essentially the number of
>> supported
>>>>>> IOPS
>>>>>>>>> of the SAN.
>>>>>>>>>
>>>>>>>>> The next API call could be to get all of the volumes on the SAN.
>>>>>> Iterate
>>>>>>>>> through them all and add up their min IOPS values. This is the
>> number
>>>>>> of
>>>>>>>>> IOPS the SAN is committed to.
>>>>>>>>>
>>>>>>>>> These two numbers can then be updated in the storage_pool table (a
>>>>>> column
>>>>>>>>> for each value).
>>>>>>>>>
>>>>>>>>> The allocators can get these values as needed (and they would be as
>>>>>>>>> accurate as the last time the thread asked the SAN for this info).
>>>>>>>>>
>>>>>>>>> These two fields, the min IOPS of the volume to create, and the
>>>>>>>> overcommit
>>>>>>>>> ratio of the plug-in would tell the allocator if it can select the
>>>>>> given
>>>>>>>>> storage pool.
>>>>>>>>>
>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> "As I mentioned previously, I am very reluctant for any feature to
>>>>>> come
>>>>>>>>>> into master that can exhaust resources."
>>>>>>>>>>
>>>>>>>>>> Just wanted to mention that, worst case, the SAN would fail
>> creation
>>>>>> of
>>>>>>>>>> the volume before allowing a new volume to break the system.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi John,
>>>>>>>>>>>
>>>>>>>>>>> Are you thinking we add a column on to the storage pool table,
>>>>>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
>>>>>>>>>>>
>>>>>>>>>>> That is easy enough.
>>>>>>>>>>>
>>>>>>>>>>> How do you want to determine what the SAN is capable of
>> supporting
>>>>>> IOPS
>>>>>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
>>>>>>>> storage
>>>>>>>>>>> nodes to the cluster, the number of IOPS increases. Do we have a
>>>>>>>> thread we
>>>>>>>>>>> can use to query external devices like this SAN to update the
>>>>>> supported
>>>>>>>>>>> number of IOPS?
>>>>>>>>>>>
>>>>>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>>>>>>>>>>> overcommit ration to the plug-in when it's created? We would need
>>>> to
>>>>>>>> store
>>>>>>>>>>> this in the storage_pool table, as well, I believe.
>>>>>>>>>>>
>>>>>>>>>>> We should also get Wei involved in this as his feature will need
>>>>>>>> similar
>>>>>>>>>>> functionality.
>>>>>>>>>>>
>>>>>>>>>>> Also, we should do this FAST as we have only two weeks left and
>>>> many
>>>>>> of
>>>>>>>>>>> us will be out for several days for the CS Collab Conference.
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <
>> jburwell@basho.com
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Querying the SAN only indicates the number of IOPS currently in
>>>> use.
>>>>>>>>>>>> The allocator needs to check the number of IOPS committed which
>> is
>>>>>>>> tracked
>>>>>>>>>>>> by CloudStack. For 4.2, we should be able to query the number of
>>>>>> IOPS
>>>>>>>>>>>> committed to a DataStore, and determine whether or not the
>> number
>>>>>>>> requested
>>>>>>>>>>>> can be fulfilled by that device. It seems to be that a
>>>>>>>>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>>>>>>>>>>> DataStore's that don't support provisioned IOPS would return
>> null.
>>>>>>>>>>>>
>>>>>>>>>>>> As I mentioned previously, I am very reluctant for any feature
>> to
>>>>>> come
>>>>>>>>>>>> into master that can exhaust resources.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Yeah, I'm not sure I could come up with anything near an
>> accurate
>>>>>>>>>>>>> assessment of how many IOPS are currently available on the SAN
>>>> (or
>>>>>>>>>>>> even a
>>>>>>>>>>>>> total number that are available for volumes). Not sure if
>> there's
>>>>>> yet
>>>>>>>>>>>> an
>>>>>>>>>>>>> API call for that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If I did know this number (total number of IOPS supported by
>> the
>>>>>>>> SAN),
>>>>>>>>>>>> we'd
>>>>>>>>>>>>> still have to keep track of the total number of volumes we've
>>>>>> created
>>>>>>>>>>>> from
>>>>>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues
>> an
>>>>>> API
>>>>>>>>>>>> call
>>>>>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given
>> volume
>>>>>> or
>>>>>>>>>>>> set of
>>>>>>>>>>>>> volumes (not supported from CS, but supported via the SolidFire
>>>>>> API),
>>>>>>>>>>>> our
>>>>>>>>>>>>> numbers in CS would be off.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a really
>> good
>>>>>>>> idea
>>>>>>>>>>>> for
>>>>>>>>>>>>> a future release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For 4.2 I think we should stick to having the allocator detect
>> if
>>>>>>>>>>>> storage
>>>>>>>>>>>>> QoS is desired and if the storage pool in question supports
>> that
>>>>>>>>>>>> feature.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you really are over provisioned on your SAN in terms of IOPS
>>>> or
>>>>>>>>>>>>> capacity, the SAN can let the admin know in several different
>>>> ways
>>>>>>>>>>>> (e-mail,
>>>>>>>>>>>>> SNMP, GUI).
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
>>>> jburwell@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please see my comments in-line below.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However,
>> we
>>>>>> may
>>>>>>>>>>>> want
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If
>>>> the
>>>>>>>>>>>> user
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> expecting that the hypervisor will provide a particular QoS,
>>>> and
>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>> not possible, it seems like we should inform them rather
>> than
>>>>>>>>>>>> silently
>>>>>>>>>>>>>>>> ignoring their request.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sure, that sounds reasonable.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We'd have to come up with some way for the allocators to know
>>>>>> about
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any thoughts on how we might do that?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To collect my thoughts from previous parts of the thread, I
>> am
>>>>>>>>>>>>>>>> uncomfortable with the idea that the management server can
>>>>>>>>>>>> overcommit a
>>>>>>>>>>>>>>>> resource. You had mentioned querying the device for
>> available
>>>>>>>> IOPS.
>>>>>>>>>>>>>> While
>>>>>>>>>>>>>>>> that would be nice, it seems like we could fall back to a
>> max
>>>>>> IOPS
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>>>>>>>>>> administrator/operator. I think such threshold and
>> allocation
>>>>>>>> rails
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it
>> is
>>>> a
>>>>>>>>>>>> basic
>>>>>>>>>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Are you thinking this ability would make it into 4.2? Just
>>>>>> curious
>>>>>>>>>>>> what
>>>>>>>>>>>>>>> release we're talking about here. For the SolidFire SAN, you
>>>>>> might
>>>>>>>>>>>> have,
>>>>>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and
>>>> then
>>>>>>>>>>>> later
>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to
>> have
>>>> a
>>>>>>>> way
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> know that the number of supported IOPS has increased.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2.
>> For
>>>>>>>>>>>> example,
>>>>>>>>>>>>>> we may only support expanding capacity in 4.2, and not handle
>>>> any
>>>>>>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
>>>> Extrapolating
>>>>>>>>>>>> a
>>>>>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a
>> similar
>>>>>> set
>>>>>>>>>>>> of
>>>>>>>>>>>>>> rails defined per host.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed
>>>> in
>>>>>> a
>>>>>>>>>>>>>>>> implementation specific manner. I think we need to arrive
>> at a
>>>>>>>>>>>> general
>>>>>>>>>>>>>>>> model that can be exploited by the allocators and planners.
>> We
>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> restrict implementation specific key-value pairs (call them
>>>>>>>> details,
>>>>>>>>>>>>>>>> extended data, whatever) to information that is unique to
>> the
>>>>>>>>>>>> driver and
>>>>>>>>>>>>>>>> would provide no useful information to the management
>> server's
>>>>>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to fit
>>>>>> those
>>>>>>>>>>>>>>>> criteria.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I wonder if this would be a good discussion topic for
>> Sunday's
>>>> CS
>>>>>>>>>>>> Collab
>>>>>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, it would -- I will put something in the wiki topic. It
>> will
>>>>>>>>>>>> also be
>>>>>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
>>>> include
>>>>>>>>>>>> some of
>>>>>>>>>>>>>> my opinions on the topic.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Accept the values as they are currently required (four new
>>>>>> fields
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) No QoS
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 3) Storage Qos
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The admin needs to specify storage tags that only map to
>>>>>> storage
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> supports Storage QoS.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless
>>>> all
>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Post 4.3:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Come up with a way to more generally enter these parameters
>>>>>>>>>>>> (probably
>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Have the drivers expose their feature set so the allocators
>>>> can
>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>> them more fully and throw an exception if there is not a
>>>>>>>> sufficient
>>>>>>>>>>>>>>>> match.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to
>>>> put
>>>>>>>> some
>>>>>>>>>>>>>>>> burden
>>>>>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that
>>>> makes
>>>>>>>>>>>> sense.
>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported Min
>>>> and
>>>>>>>> Max
>>>>>>>>>>>>>>>> values,
>>>>>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
>>>>>> SolidFire
>>>>>>>>>>>>>> primary
>>>>>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that
>> doesn't,
>>>>>>>> then
>>>>>>>>>>>> he's
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>> going to get the Min and Max feature. We could add help
>> text
>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> pop-up
>>>>>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and Max
>>>> text
>>>>>>>>>>>> fields.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
>>>>>>>> comprehensive
>>>>>>>>>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Maybe in the future we could have the drivers advertise
>>>> their
>>>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>> and if the allocator feels a request is not being
>> satisfied
>>>>>> (say
>>>>>>>>>>>> Min
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it
>>>> can
>>>>>>>>>>>> throw
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>> exception.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
>>>> features
>>>>>>>> are
>>>>>>>>>>>>>>>>>>>> mutually
>>>>>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a
>> configuration
>>>>>> that
>>>>>>>>>>>>>>>>>>>> contains both
>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>>>>>>>>>> non-deterministic
>>>>>>>>>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
>>>>>>>> storage
>>>>>>>>>>>>>>>> oriented
>>>>>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
>>>> storage
>>>>>>>>>>>> tags,
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> have no
>>>>>>>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
>>>>>> wasn't
>>>>>>>>>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>>>>>>>>>> intending that we implement this allocation model in
>>>> 4.3,
>>>>>>>> but
>>>>>>>>>>>>>>>> instead,
>>>>>>>>>>>>>>>>>>>>>> determine if this type model would be one we would
>> want
>>>> to
>>>>>>>>>>>> support
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the
>>>>>> data
>>>>>>>>>>>> model
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> current implementation would not preclude evolution in
>>>>>> that
>>>>>>>>>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
>>>> user's
>>>>>>>>>>>> expect
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource set
>> to
>>>>>>>>>>>> fulfill a
>>>>>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in that
>> what
>>>>>>>> we've
>>>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>>>> should not make refinement difficult in the future. If
>> we
>>>>>>>> don't
>>>>>>>>>>>>>>>> modify
>>>>>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
>>>>>> relatively
>>>>>>>>>>>> simple
>>>>>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
>>>>>>>> aggregated
>>>>>>>>>>>>>> result
>>>>>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>> I/O and
>>>>>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
>>>>>> concern
>>>>>>>>>>>> is
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At
>> minimum,
>>>>>>>>>>>> Javadoc
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and their
>>>>>>>> mutual
>>>>>>>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>>>>>>>>>>>> provides
>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where
>> provisioned
>>>>>> IOPS
>>>>>>>>>>>> uses
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>> range. In order to select the best available resource to
>>>>>>>>>>>> fulfill a
>>>>>>>>>>>>>>>> QoS, we
>>>>>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in
>> terms
>>>> of
>>>>>>>>>>>> units
>>>>>>>>>>>>>>>> (IOPS or
>>>>>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want
>>>> to
>>>>>>>>>>>>>> achieve a
>>>>>>>>>>>>>>>>>>>> model like I described, I think we would need to resolve
>>>>>> this
>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>> in 4.2
>>>>>>>>>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the
>> input
>>>>>> for
>>>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>> Plus
>>>>>>>>>>>>>>>>>>> people probably want to input the data in terms they're
>>>>>>>> familiar
>>>>>>>>>>>> with
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> the products in question.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and
>> let
>>>>>> the
>>>>>>>>>>>> user
>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected due
>>>> to
>>>>>> a
>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it
>> because
>>>>>>>> what
>>>>>>>>>>>>>>>> happens if
>>>>>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage
>> after
>>>>>>>>>>>> having
>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>>>>>>>>>>>> re-think
>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose
>> their
>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> As I think through the implications of these
>>>> requirements
>>>>>>>> and
>>>>>>>>>>>>>>>> reflect
>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't
>> already
>>>>>>>>>>>> impacted
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>> has no
>>>>>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
>>>> devices
>>>>>>>>>>>> that
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>>>>>>>>>>>> potentially
>>>>>>>>>>>>>>>>>>>> causing
>>>>>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It
>>>> seems
>>>>>>>>>>>> to me
>>>>>>>>>>>>>>>>>>>> that a
>>>>>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express
>> the
>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment factor.
>>>>>> This
>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>> should be used by the storage allocators to determine
>>>> the
>>>>>>>>>>>> device
>>>>>>>>>>>>>>>> best
>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems
>>>> that
>>>>>> a
>>>>>>>>>>>>>> similar
>>>>>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>>>>>> considerations would need to be added to the
>> Hypervisor
>>>>>>>> layer
>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path
>>>>>> that
>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>> previously
>>>>>>>>>>>>>>>>>>>>> established for other properties you see on dialogs in
>>>> CS:
>>>>>>>> Just
>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is
>> deployed
>>>>>> to
>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs,
>> say, a
>>>>>>>>>>>> storage
>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> confines the deployment only to storage that supports
>> the
>>>>>>>>>>>> selected
>>>>>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>> works
>>>>>>>>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for the
>>>>>>>>>>>>>> administrator
>>>>>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is
>> that
>>>> we
>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes
>> of
>>>>>>>>>>>>>>>> illustration,
>>>>>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
>>>>>>>> device
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> 100000.
>>>>>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>>>>>>>>>> Therefore,
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the
>> DataStore
>>>>>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
>>>> factor.
>>>>>>>>>>>> As we
>>>>>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>>>>>>>>>>> guarantee
>>>>>>>>>>>>>>>> more IOPS
>>>>>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that
>>>>>> make
>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm
>>>> not
>>>>>>>> even
>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely
>> remain.
>>>>>> I'd
>>>>>>>>>>>> have
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN
>> and
>>>>>>>> total
>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>> up,
>>>>>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
>>>>>>>> supported
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> the system.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow
>> non-QoS
>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>>>>>>>>>> think a side effect of the current implementation is
>>>>>>>> SolidFire
>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled
>>>> I/O,
>>>>>> it
>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
>>>>>> side-by-side
>>>>>>>>>>>> with
>>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
>>>> potentially
>>>>>>>>>>>> starve
>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined
>>>> the
>>>>>>>>>>>>>> collocated
>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of
>> CS)
>>>>>> and
>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>> specify
>>>>>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be
>> at
>>>>>> the
>>>>>>>>>>>>>>>> defaults
>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire
>>>> SAN,
>>>>>>>> you
>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage
>> tags
>>>>>>>>>>>> exclusions
>>>>>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
>>>>>> deploy
>>>>>>>> my
>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list
>> as
>>>> it
>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> complicate component assembly. It would require a
>> storage
>>>>>>>>>>>> plugin to
>>>>>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies and
>>>>>>>>>>>> determine
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express
>> their
>>>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
>>>> inclusion
>>>>>>>> or
>>>>>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
>>>> functionality
>>>>>>>> in
>>>>>>>>>>>> a
>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> allocator understands.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since
>> it
>>>>>> only
>>>>>>>>>>>>>> limits
>>>>>>>>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a
>>>> system
>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem
>> does
>>>>>> not
>>>>>>>>>>>>>> exist.
>>>>>>>>>>>>>>>>>>>> However, I am concerned with the more general case as we
>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
>>>> vendor-specific
>>>>>>>>>>>> values
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
>>>>>>>> provisioned
>>>>>>>>>>>>>>>> IOPS, but
>>>>>>>>>>>>>>>>>>> what if you break them out by read and write or do
>>>> something
>>>>>>>>>>>> else? We
>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
>>>> throttled
>>>>>>>>>>>> I/O
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> We also need to understand the implications of these
>> QoS
>>>>>>>>>>>>>> guarantees
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
>>>>>>>> hardware
>>>>>>>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely
>>>> be
>>>>>>>>>>>> forced
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>>>>>>>>>> implementation
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some
>>>> great
>>>>>>>>>>>> ideas
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning
>>>> (with
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> tags).
>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>>>>>>>>>>>> addition to
>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage
>> QoS
>>>>>>>> being
>>>>>>>>>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available,
>>>> but
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in
>> values
>>>>>> for
>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at
>> least
>>>>>>>>>>>> yet), so
>>>>>>>>>>>>>> he
>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
>>>> allocators
>>>>>>>> to
>>>>>>>>>>>>>> prefer
>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
>>>>>> today
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
>>>> features
>>>>>>>>>>>> (fields
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
>>>>>> Perhaps
>>>>>>>>>>>> post
>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>>>>>>>>>>>> OpenStack
>>>>>>>>>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two
>> weeks
>>>>>>>> until
>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep
>>>>>> Wei's
>>>>>>>>>>>> fields
>>>>>>>>>>>>>>>>>>>> and my
>>>>>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature
>>>> has
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>> entered
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
>>>> general-purpose
>>>>>>>>>>>> design
>>>>>>>>>>>>>>>>>>>> that can
>>>>>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>>>>>>>>>>>> situations.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I just published my review @
>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before
>> coming
>>>> up
>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes
>> to
>>>> be
>>>>>>>>>>>> checked
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
>>>>>> file?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my
>> review.
>>>> I
>>>>>> am
>>>>>>>>>>>>>>>> offline
>>>>>>>>>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
>>>> Board?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
>>>> reading
>>>>>>>>>>>> this :)
>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all
>>>> the
>>>>>>>>>>>> areas
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning
>> or
>>>>>>>>>>>>>> afternoon,
>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
>>>> conflicts
>>>>>>>> in
>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
>>>> disk_io_throttling
>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>>>>>>>>> object_store
>>>>>>>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we
>>>> merge
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store
>> goes
>>>>>>>> into
>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits
>>>> into
>>>>>>>> this
>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John
>> Burwell <
>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform.
>> The
>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
>>>>>> number
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies
>> between
>>>>>> the
>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire ->
>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management
>> aspects
>>>>>> of
>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
>>>> circular
>>>>>>>>>>>>>> dependency
>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
>>>>>> graph,
>>>>>>>>>>>> we
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my
>>>> code
>>>>>>>>>>>> into
>>>>>>>>>>>>>> his
>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic
>> code
>>>>>>>>>>>> after his
>>>>>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
>>>> Tutkowski
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
>>>>>> here:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
>>>> hypervisor
>>>>>>>>>>>> code in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
>>>>>> storage
>>>>>>>>>>>>>> plug-in,
>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in
>>>> use
>>>>>>>>>>>>>> (although
>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.)
>> is
>>>>>>>>>>>> actually
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
>>>> placed
>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>> url
>>>>>>>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>>>>>>>> stored
>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes
>>>> the
>>>>>>>>>>>> place
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
>>>>>> hypervisor
>>>>>>>>>>>>>> checks
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
>>>> necessary
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>>>>>>>>>> executes to
>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
>>>> invoked
>>>>>>>> to
>>>>>>>>>>>>>>>> detach a
>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
>>>> hypervisor
>>>>>>>> data
>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support
>> Burst
>>>>>>>> IOPS
>>>>>>>>>>>> in
>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
>>>> table.
>>>>>>>> If
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>>>>>>>>>> avoiding
>>>>>>>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
>>>>>> details
>>>>>>>>>>>>>> field.
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the
>> documentation
>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>>>>>> details
>>>>>>>>>>>> are
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>>>>>>>>>> listStoragePool
>>>>>>>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would
>>>> be
>>>>>>>>>>>> useful
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the
>> details
>>>>>>>>>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst
>>>>>> IOPS
>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>>>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
>>>>>> have
>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>>>>>>>>> disk_offering_details
>>>>>>>>>>>>>>>> table.
>>>>>>>>>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
>>>>>> Burst
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are
>>>> in
>>>>>>>> the
>>>>>>>>>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Quick comments below in red, but I think we're good to go.
On Fri, Jun 14, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my comments in-line below.
>
> Thanks,
> -John
>
> On Jun 14, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Comments below in red (but I believe we are reaching a consensus here).
> :)
> >
> >
> > On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Please see my comments in-line below.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> I am OK with that approach, John.
> >>>
> >>> So, let me review to make sure I follow you correctly:
> >>>
> >>> We introduce two new parameters to the plug-in: Number of Total IOPS
> for
> >>> the SAN and an overcommit ratio. (On a side note, if we are just
> >>> multiplying the two numbers, why don't we have the user just input
> their
> >>> product?)
> >>
> >> It is a straw man suggestion to allow some fine tuning of the allocation
> >> algorithm. I defer to you to determine whether or not such a tunable
> would
> >> be valuable in this use case. My thinking was that it is tunable to
> >> support operators as they more deeply understand/observe the CS workload
> >> while allowing them to think about IOPS absolutely. For example, I
> start
> >> out at say 50% overcommit and realize that the load is actually heavier
> >> than expected, I can reduce it or vice versa.
> >>
> >
> > Using your example of support people changing the value after seeing how
> > the system is performing:
> >
> > If we had them just pass in a new Total IOPS value, that should cover it.
>
> That is fine answer to me. I expect that we will be refining this
> capability over the next few releases. Therefore, I am cool with starting
> as simple as possible as it will ease/simplify future evolution.
>
> >
> >
> >>
> >>>
> >>> Are both of these new parameters to the create storage pool API command
> >> or
> >>> are they passed into the create storage pool API command through its
> url
> >>> parameter?
> >>>
> >>> If they are new parameters, we should make two new columns in the
> >>> storage_pool table.
> >>>
> >>> If they are passed in via the url parameter, they should go in the
> >>> storage_pool_details table.
> >>
> >> I think these allocation parameters implementation agnostic, and should
> be
> >> columns on the storage_pool table and first class properties on the
> >> StoragePool class.
> >>
> >
> > Per my most recent e-mail, I agree.
> >
> >
> >>
> >>>
> >>> For 4.2, if someone wants to change these values, they must update the
> DB
> >>> manually.
> >>
> >> Is it not possible to support updates through the updateStoragePool API
> >> call?
> >>
> >
> > Edison - if you're reading these - do you know if the plug-ins are
> informed
> > when the updateStoragePool API command is issued?
>
> The way I am thinking about these thresholds is that they are part of the
> storage engine itself. Why would plugin need to know that the value
> changed? When the plugin needs the value, it will be retrieved from the
> database ...
>
You know, when I wrote what I wrote above, I was thinking the data was
coming in with the url parameter, which has to be parsed by the plug-in
(and it's stored in the storage_pool_details table).
Since this is a new, standard parameter for the API (stored in the
storage_pool table), it should work just fine without plug-in interaction.
>
> >
> >
> >>
> >>>
> >>> Every time a volume is created or deleted for the SolidFire plug-in,
> the
> >>> Current IOPS value (sum of all volumes' Min IOPS that are associated
> with
> >>> the plug-in) is updated.
> >>
> >> I don't think we need another column for this computed value. It will
> be
> >> simpler to simply ask the database to sum the values as needed.
> >>
> >
> > I'm thinking now that we should follow the pattern established by the
> > "size" field. There are two related size fields for a storage pool:
> > capacity_bytes and available bytes.
> >
> > We should just follow this pattern and create two new fields:
> capacity_iops
> > and available_iops. The plug-in can increment or decrement available_iops
> > each time a volume is created or deleted, respectively.
>
> I would rather see us do away with available_bytes. My experience with
> storing computed fields is that often get out of sync and represent
> premature optimization. I missed that part of the throttled I/O patch ...
>
Fair enough. Agreed - one new field in the storage_pool table:
capacity_iops.
>
> >
> >
> >>
> >>>
> >>> The allocator can use these fields to determine if it can fit in a new
> >>> volume.
> >>
> >> Exactly.
> >>
> >>>
> >>> Does it look like my understanding is OK?
> >>
> >> Yes, looks good to me.
> >>
> >>>
> >>>
> >>> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> I apologize for not being clear -- this conversation has been
> admittedly
> >>>> disjoint. I think we should allow the maximum IOPS and overcommitment
> >>>> values to be updated though I would recommend restricting updates to
> to
> >> be
> >>>> an increasing value for 4.2 (e.g. users can increase the number of
> total
> >>>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to
> 200,000).
> >>>> While not ideal, given the amount of time we have left for 4.2, it
> will
> >>>> cover most cases, and we can address the implications of reducing
> >> resource
> >>>> capacity in 4.3. This approach addresses both of your concerns.
> >> First, it
> >>>> allows the administrator/operator to determine what portion of the
> >> device
> >>>> they wish to dedicate. For example, if the device has a total
> capacity
> >> of
> >>>> 200,000 IOPS, and they only want CS to use 25% of the device then they
> >> set
> >>>> the maximum total IOPS to 50,000. Second, as they grow capacity, they
> >> can
> >>>> update the DataStore to increase the number of IOPS they want to
> >> dedicate
> >>>> to CS' use. I would imagine expansion of capacity happens
> infrequently
> >>>> enough that increasing the maximum IOPS value would not be a
> significant
> >>>> burden.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> "the administrator/operator simply needs to tell us the total number
> of
> >>>>> IOPS that can be committed to it and an overcommitment factor."
> >>>>>
> >>>>> Are you thinking when we create a plug-in as primary storage that we
> >> say
> >>>> -
> >>>>> up front - how many IOPS the SAN can handle?
> >>>>>
> >>>>> That is not a good move, in my opinion. Our SAN is designed to start
> >>>> small
> >>>>> and grow to PBs. As the need arises for more storage, the admin
> >> purchases
> >>>>> additional storage nodes that join the cluster and the performance
> and
> >>>>> capacity go up.
> >>>>>
> >>>>> We need to know how many IOPS total the SAN can handle and what it is
> >>>>> committed to currently (the sum of the number of volumes' min IOPS).
> >>>>>
> >>>>> We also cannot assume the SAN is dedicated to CS.
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Simon,
> >>>>>>
> >>>>>> Yes, it is CloudStack's job to protect, as best it can, from
> >>>>>> oversubscribing resources. I would argue that resource management
> is
> >>>> one,
> >>>>>> if not the most, important functions of the system. It is no
> >> different
> >>>>>> than the allocation/planning performed for hosts relative to cores
> and
> >>>>>> memory. We can still oversubscribe resources, but we have rails +
> >> knobs
> >>>>>> and dials to avoid it. Without these controls in place, we could
> >> easily
> >>>>>> allow users to deploy workloads that overrun resources harming all
> >>>> tenants.
> >>>>>>
> >>>>>> I also think that we are over thinking this issue for provisioned
> >> IOPS.
> >>>>>> When the DataStore is configured, the administrator/operator simply
> >>>> needs
> >>>>>> to tell us the total number of IOPS that can be committed to it and
> an
> >>>>>> overcommitment factor. As we allocate volumes to that DataStore, we
> >>>> sum up
> >>>>>> the committed IOPS of the existing Volumes attached to the
> DataStore,
> >>>> apply
> >>>>>> the overcommitment factor, and determine whether or not the
> requested
> >>>>>> minimum IOPS for the new volume can be fulfilled. We can provide
> both
> >>>>>> general and vendor specific documentation for determining these
> values
> >>>> --
> >>>>>> be they to consume the entire device or a portion of it.
> >>>>>>
> >>>>>> Querying the device is unnecessary and deceptive. CloudStack
> resource
> >>>>>> management is not interested in the current state of the device
> which
> >>>> could
> >>>>>> be anywhere from extremely heavy to extremely light at any given
> time.
> >>>> We
> >>>>>> are interested in the worst case load that is anticipated for
> >> resource.
> >>>> In
> >>>>>> my view, it is up to administrators/operators to instrument their
> >>>>>> environment to understand usage patterns and capacity. We should
> >>>> provide
> >>>>>> information that will help determine what should be
> >>>> instrumented/monitored,
> >>>>>> but that function should be performed outside of CloudStack.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
> >>>>>>
> >>>>>>> I'd like to comment on this briefly.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I think an assumption is being made that the SAN is being dedicated
> >> to
> >>>> a
> >>>>>> CS instance.
> >>>>>>>
> >>>>>>> My person opinion that this whole IOPS calculation is getting
> rather
> >>>>>> complicated, and could probably be much simpler than this. Over
> >>>>>> subscription is a fact of life on virtually all storage, and is
> really
> >>>> no
> >>>>>> different in concept than multiple virt instances on a single piece
> of
> >>>>>> hardware. All decent SANs offer many management options for the
> >> storage
> >>>>>> engineers to keep track of IOPS utilization, and plan for spindle
> >>>>>> augmentation as required.
> >>>>>>> Is it really the job of CS to become yet another management layer
> on
> >>>> top
> >>>>>> of this?
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>>
> >>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>> To: dev@cloudstack.apache.org
> >>>>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
> >>>>>> ustcweizhou@gmail.com>
> >>>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
> >>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>
> >>>>>>> 1) We want number of IOPS currently supported by the SAN.
> >>>>>>>
> >>>>>>> 2) We want the number of IOPS that are committed (sum of min IOPS
> for
> >>>>>> each
> >>>>>>> volume).
> >>>>>>>
> >>>>>>> We could do the following to keep track of IOPS:
> >>>>>>>
> >>>>>>> The plug-in could have a timer thread that goes off every, say, 1
> >>>> minute.
> >>>>>>>
> >>>>>>> It could query the SAN for the number of nodes that make up the SAN
> >> and
> >>>>>>> multiple this by 50,000. This is essentially the number of
> supported
> >>>> IOPS
> >>>>>>> of the SAN.
> >>>>>>>
> >>>>>>> The next API call could be to get all of the volumes on the SAN.
> >>>> Iterate
> >>>>>>> through them all and add up their min IOPS values. This is the
> number
> >>>> of
> >>>>>>> IOPS the SAN is committed to.
> >>>>>>>
> >>>>>>> These two numbers can then be updated in the storage_pool table (a
> >>>> column
> >>>>>>> for each value).
> >>>>>>>
> >>>>>>> The allocators can get these values as needed (and they would be as
> >>>>>>> accurate as the last time the thread asked the SAN for this info).
> >>>>>>>
> >>>>>>> These two fields, the min IOPS of the volume to create, and the
> >>>>>> overcommit
> >>>>>>> ratio of the plug-in would tell the allocator if it can select the
> >>>> given
> >>>>>>> storage pool.
> >>>>>>>
> >>>>>>> What do you think?
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> "As I mentioned previously, I am very reluctant for any feature to
> >>>> come
> >>>>>>>> into master that can exhaust resources."
> >>>>>>>>
> >>>>>>>> Just wanted to mention that, worst case, the SAN would fail
> creation
> >>>> of
> >>>>>>>> the volume before allowing a new volume to break the system.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi John,
> >>>>>>>>>
> >>>>>>>>> Are you thinking we add a column on to the storage pool table,
> >>>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
> >>>>>>>>>
> >>>>>>>>> That is easy enough.
> >>>>>>>>>
> >>>>>>>>> How do you want to determine what the SAN is capable of
> supporting
> >>>> IOPS
> >>>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
> >>>>>> storage
> >>>>>>>>> nodes to the cluster, the number of IOPS increases. Do we have a
> >>>>>> thread we
> >>>>>>>>> can use to query external devices like this SAN to update the
> >>>> supported
> >>>>>>>>> number of IOPS?
> >>>>>>>>>
> >>>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
> >>>>>>>>> overcommit ration to the plug-in when it's created? We would need
> >> to
> >>>>>> store
> >>>>>>>>> this in the storage_pool table, as well, I believe.
> >>>>>>>>>
> >>>>>>>>> We should also get Wei involved in this as his feature will need
> >>>>>> similar
> >>>>>>>>> functionality.
> >>>>>>>>>
> >>>>>>>>> Also, we should do this FAST as we have only two weeks left and
> >> many
> >>>> of
> >>>>>>>>> us will be out for several days for the CS Collab Conference.
> >>>>>>>>>
> >>>>>>>>> Thanks
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <
> jburwell@basho.com
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> Querying the SAN only indicates the number of IOPS currently in
> >> use.
> >>>>>>>>>> The allocator needs to check the number of IOPS committed which
> is
> >>>>>> tracked
> >>>>>>>>>> by CloudStack. For 4.2, we should be able to query the number of
> >>>> IOPS
> >>>>>>>>>> committed to a DataStore, and determine whether or not the
> number
> >>>>>> requested
> >>>>>>>>>> can be fulfilled by that device. It seems to be that a
> >>>>>>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
> >>>>>>>>>> DataStore's that don't support provisioned IOPS would return
> null.
> >>>>>>>>>>
> >>>>>>>>>> As I mentioned previously, I am very reluctant for any feature
> to
> >>>> come
> >>>>>>>>>> into master that can exhaust resources.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Yeah, I'm not sure I could come up with anything near an
> accurate
> >>>>>>>>>>> assessment of how many IOPS are currently available on the SAN
> >> (or
> >>>>>>>>>> even a
> >>>>>>>>>>> total number that are available for volumes). Not sure if
> there's
> >>>> yet
> >>>>>>>>>> an
> >>>>>>>>>>> API call for that.
> >>>>>>>>>>>
> >>>>>>>>>>> If I did know this number (total number of IOPS supported by
> the
> >>>>>> SAN),
> >>>>>>>>>> we'd
> >>>>>>>>>>> still have to keep track of the total number of volumes we've
> >>>> created
> >>>>>>>>>> from
> >>>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues
> an
> >>>> API
> >>>>>>>>>> call
> >>>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given
> volume
> >>>> or
> >>>>>>>>>> set of
> >>>>>>>>>>> volumes (not supported from CS, but supported via the SolidFire
> >>>> API),
> >>>>>>>>>> our
> >>>>>>>>>>> numbers in CS would be off.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a really
> good
> >>>>>> idea
> >>>>>>>>>> for
> >>>>>>>>>>> a future release.
> >>>>>>>>>>>
> >>>>>>>>>>> For 4.2 I think we should stick to having the allocator detect
> if
> >>>>>>>>>> storage
> >>>>>>>>>>> QoS is desired and if the storage pool in question supports
> that
> >>>>>>>>>> feature.
> >>>>>>>>>>>
> >>>>>>>>>>> If you really are over provisioned on your SAN in terms of IOPS
> >> or
> >>>>>>>>>>> capacity, the SAN can let the admin know in several different
> >> ways
> >>>>>>>>>> (e-mail,
> >>>>>>>>>>> SNMP, GUI).
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
> >> jburwell@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please see my comments in-line below.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
> >>>> jburwell@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However,
> we
> >>>> may
> >>>>>>>>>> want
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If
> >> the
> >>>>>>>>>> user
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>> expecting that the hypervisor will provide a particular QoS,
> >> and
> >>>>>>>>>> that is
> >>>>>>>>>>>>>> not possible, it seems like we should inform them rather
> than
> >>>>>>>>>> silently
> >>>>>>>>>>>>>> ignoring their request.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sure, that sounds reasonable.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We'd have to come up with some way for the allocators to know
> >>>> about
> >>>>>>>>>> the
> >>>>>>>>>>>>> requested storage QoS and then query the candidate drivers.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Any thoughts on how we might do that?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> To collect my thoughts from previous parts of the thread, I
> am
> >>>>>>>>>>>>>> uncomfortable with the idea that the management server can
> >>>>>>>>>> overcommit a
> >>>>>>>>>>>>>> resource. You had mentioned querying the device for
> available
> >>>>>> IOPS.
> >>>>>>>>>>>> While
> >>>>>>>>>>>>>> that would be nice, it seems like we could fall back to a
> max
> >>>> IOPS
> >>>>>>>>>> and
> >>>>>>>>>>>>>> overcommit factor manually calculated and entered by the
> >>>>>>>>>>>>>> administrator/operator. I think such threshold and
> allocation
> >>>>>> rails
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it
> is
> >> a
> >>>>>>>>>> basic
> >>>>>>>>>>>>>> feature of any cloud orchestration platform.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Are you thinking this ability would make it into 4.2? Just
> >>>> curious
> >>>>>>>>>> what
> >>>>>>>>>>>>> release we're talking about here. For the SolidFire SAN, you
> >>>> might
> >>>>>>>>>> have,
> >>>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and
> >> then
> >>>>>>>>>> later
> >>>>>>>>>>>> add
> >>>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to
> have
> >> a
> >>>>>> way
> >>>>>>>>>> to
> >>>>>>>>>>>>> know that the number of supported IOPS has increased.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2.
> For
> >>>>>>>>>> example,
> >>>>>>>>>>>> we may only support expanding capacity in 4.2, and not handle
> >> any
> >>>>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
> >> Extrapolating
> >>>>>>>>>> a
> >>>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a
> similar
> >>>> set
> >>>>>>>>>> of
> >>>>>>>>>>>> rails defined per host.
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed
> >> in
> >>>> a
> >>>>>>>>>>>>>> implementation specific manner. I think we need to arrive
> at a
> >>>>>>>>>> general
> >>>>>>>>>>>>>> model that can be exploited by the allocators and planners.
> We
> >>>>>>>>>> should
> >>>>>>>>>>>>>> restrict implementation specific key-value pairs (call them
> >>>>>> details,
> >>>>>>>>>>>>>> extended data, whatever) to information that is unique to
> the
> >>>>>>>>>> driver and
> >>>>>>>>>>>>>> would provide no useful information to the management
> server's
> >>>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to fit
> >>>> those
> >>>>>>>>>>>>>> criteria.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I wonder if this would be a good discussion topic for
> Sunday's
> >> CS
> >>>>>>>>>> Collab
> >>>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes, it would -- I will put something in the wiki topic. It
> will
> >>>>>>>>>> also be
> >>>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
> >> include
> >>>>>>>>>> some of
> >>>>>>>>>>>> my opinions on the topic.
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> So, here's my suggestion for 4.2:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Accept the values as they are currently required (four new
> >>>> fields
> >>>>>>>>>> for
> >>>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>> feature or two new fields for mine).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1) No QoS
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2) Hypervisor QoS
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 3) Storage Qos
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The admin needs to specify storage tags that only map to
> >>>> storage
> >>>>>>>>>> that
> >>>>>>>>>>>>>>> supports Storage QoS.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless
> >> all
> >>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>> in use support the new fields, they may not be enforced.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Post 4.3:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Come up with a way to more generally enter these parameters
> >>>>>>>>>> (probably
> >>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>> key/value pairs sent to the drivers).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Have the drivers expose their feature set so the allocators
> >> can
> >>>>>>>>>>>> consider
> >>>>>>>>>>>>>>> them more fully and throw an exception if there is not a
> >>>>>> sufficient
> >>>>>>>>>>>>>> match.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to
> >> put
> >>>>>> some
> >>>>>>>>>>>>>> burden
> >>>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that
> >> makes
> >>>>>>>>>> sense.
> >>>>>>>>>>>> For
> >>>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported Min
> >> and
> >>>>>> Max
> >>>>>>>>>>>>>> values,
> >>>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
> >>>> SolidFire
> >>>>>>>>>>>> primary
> >>>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that
> doesn't,
> >>>>>> then
> >>>>>>>>>> he's
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>> going to get the Min and Max feature. We could add help
> text
> >>>> to
> >>>>>>>>>> the
> >>>>>>>>>>>>>> pop-up
> >>>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and Max
> >> text
> >>>>>>>>>> fields.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Same idea for Wei's feature.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
> >>>>>> comprehensive
> >>>>>>>>>>>>>>>> approach for post 4.2.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Maybe in the future we could have the drivers advertise
> >> their
> >>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>> and if the allocator feels a request is not being
> satisfied
> >>>> (say
> >>>>>>>>>> Min
> >>>>>>>>>>>> was
> >>>>>>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it
> >> can
> >>>>>>>>>> throw
> >>>>>>>>>>>> an
> >>>>>>>>>>>>>>>> exception.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Please see my comment in-line below.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I've put comments below in red.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
> >> features
> >>>>>> are
> >>>>>>>>>>>>>>>>>> mutually
> >>>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a
> configuration
> >>>> that
> >>>>>>>>>>>>>>>>>> contains both
> >>>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >>>>>>>>>>>>>> non-deterministic
> >>>>>>>>>>>>>>>>>>>> operations.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
> >>>>>> storage
> >>>>>>>>>>>>>> oriented
> >>>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
> >> storage
> >>>>>>>>>> tags,
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> have no
> >>>>>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
> >>>> wasn't
> >>>>>>>>>>>>>>>>>> necessarily
> >>>>>>>>>>>>>>>>>>>> intending that we implement this allocation model in
> >> 4.3,
> >>>>>> but
> >>>>>>>>>>>>>> instead,
> >>>>>>>>>>>>>>>>>>>> determine if this type model would be one we would
> want
> >> to
> >>>>>>>>>> support
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the
> >>>> data
> >>>>>>>>>> model
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> current implementation would not preclude evolution in
> >>>> that
> >>>>>>>>>>>>>>>>>> direction. My
> >>>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
> >> user's
> >>>>>>>>>> expect
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>> "cloud"
> >>>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource set
> to
> >>>>>>>>>> fulfill a
> >>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>> system requirements.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in that
> what
> >>>>>> we've
> >>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>> should not make refinement difficult in the future. If
> we
> >>>>>> don't
> >>>>>>>>>>>>>> modify
> >>>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
> >>>> relatively
> >>>>>>>>>> simple
> >>>>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
> >>>>>> aggregated
> >>>>>>>>>>>> result
> >>>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
> >>>>>>>>>> throttled
> >>>>>>>>>>>>>> I/O and
> >>>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
> >>>> concern
> >>>>>>>>>> is
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At
> minimum,
> >>>>>>>>>> Javadoc
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and their
> >>>>>> mutual
> >>>>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
> >>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
> >>>>>>>>>> provides
> >>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where
> provisioned
> >>>> IOPS
> >>>>>>>>>> uses
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> range. In order to select the best available resource to
> >>>>>>>>>> fulfill a
> >>>>>>>>>>>>>> QoS, we
> >>>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in
> terms
> >> of
> >>>>>>>>>> units
> >>>>>>>>>>>>>> (IOPS or
> >>>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want
> >> to
> >>>>>>>>>>>> achieve a
> >>>>>>>>>>>>>>>>>> model like I described, I think we would need to resolve
> >>>> this
> >>>>>>>>>> issue
> >>>>>>>>>>>>>> in 4.2
> >>>>>>>>>>>>>>>>>> to ensure a viable migration path.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the
> input
> >>>> for
> >>>>>>>>>> 4.2.
> >>>>>>>>>>>>>> Plus
> >>>>>>>>>>>>>>>>> people probably want to input the data in terms they're
> >>>>>> familiar
> >>>>>>>>>> with
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> the products in question.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and
> let
> >>>> the
> >>>>>>>>>> user
> >>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected due
> >> to
> >>>> a
> >>>>>>>>>> given
> >>>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it
> because
> >>>>>> what
> >>>>>>>>>>>>>> happens if
> >>>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage
> after
> >>>>>>>>>> having
> >>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>> a Disk Offering?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
> >>>>>>>>>> re-think
> >>>>>>>>>>>> it.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose
> their
> >>>>>>>>>> supported
> >>>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> As I think through the implications of these
> >> requirements
> >>>>>> and
> >>>>>>>>>>>>>> reflect
> >>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't
> already
> >>>>>>>>>> impacted
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
> >>>>>>>>>> provisioned
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>> has no
> >>>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
> >> devices
> >>>>>>>>>> that
> >>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
> >>>>>>>>>> understand
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
> >>>>>>>>>> potentially
> >>>>>>>>>>>>>>>>>> causing
> >>>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It
> >> seems
> >>>>>>>>>> to me
> >>>>>>>>>>>>>>>>>> that a
> >>>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express
> the
> >>>>>>>>>> maximum
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment factor.
> >>>> This
> >>>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>>> should be used by the storage allocators to determine
> >> the
> >>>>>>>>>> device
> >>>>>>>>>>>>>> best
> >>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems
> >> that
> >>>> a
> >>>>>>>>>>>> similar
> >>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>> considerations would need to be added to the
> Hypervisor
> >>>>>> layer
> >>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path
> >>>> that
> >>>>>>>>>> was
> >>>>>>>>>>>>>>>>>> previously
> >>>>>>>>>>>>>>>>>>> established for other properties you see on dialogs in
> >> CS:
> >>>>>> Just
> >>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is
> deployed
> >>>> to
> >>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs,
> say, a
> >>>>>>>>>> storage
> >>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> confines the deployment only to storage that supports
> the
> >>>>>>>>>> selected
> >>>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
> >>>>>>>>>> CloudStack
> >>>>>>>>>>>>>> works
> >>>>>>>>>>>>>>>>>>> today.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for the
> >>>>>>>>>>>> administrator
> >>>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is
> that
> >> we
> >>>>>>>>>> over
> >>>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes
> of
> >>>>>>>>>>>>>> illustration,
> >>>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
> >>>>>> device
> >>>>>>>>>> is
> >>>>>>>>>>>>>> 100000.
> >>>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
> >>>>>>>>>> Therefore,
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
> >>>>>>>>>> guaranteed
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the
> DataStore
> >>>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
> >> factor.
> >>>>>>>>>> As we
> >>>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
> >>>>>>>>>> guarantee
> >>>>>>>>>>>>>> more IOPS
> >>>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that
> >>>> make
> >>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm
> >> not
> >>>>>> even
> >>>>>>>>>>>> sure
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely
> remain.
> >>>> I'd
> >>>>>>>>>> have
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN
> and
> >>>>>> total
> >>>>>>>>>>>> them
> >>>>>>>>>>>>>> up,
> >>>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
> >>>>>> supported
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> the system.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow
> non-QoS
> >>>>>>>>>> resources
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
> >>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>> IOPS, I
> >>>>>>>>>>>>>>>>>>>> think a side effect of the current implementation is
> >>>>>> SolidFire
> >>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled
> >> I/O,
> >>>> it
> >>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
> >>>> side-by-side
> >>>>>>>>>> with
> >>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
> >> potentially
> >>>>>>>>>> starve
> >>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined
> >> the
> >>>>>>>>>>>> collocated
> >>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>> from being fulfilled.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of
> CS)
> >>>> and
> >>>>>>>>>> not
> >>>>>>>>>>>>>>>>>> specify
> >>>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be
> at
> >>>> the
> >>>>>>>>>>>>>> defaults
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire
> >> SAN,
> >>>>>> you
> >>>>>>>>>> will
> >>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> your Mins met.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage
> tags
> >>>>>>>>>> exclusions
> >>>>>>>>>>>>>>>>>> list,
> >>>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
> >>>> deploy
> >>>>>> my
> >>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> storage with these following tags).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list
> as
> >> it
> >>>>>>>>>> would
> >>>>>>>>>>>>>>>>>> complicate component assembly. It would require a
> storage
> >>>>>>>>>> plugin to
> >>>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies and
> >>>>>>>>>> determine
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express
> their
> >>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>> which can be matched to a set of requested requirements.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
> >> inclusion
> >>>>>> or
> >>>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
> >> functionality
> >>>>>> in
> >>>>>>>>>> a
> >>>>>>>>>>>> way
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> allocator understands.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since
> it
> >>>> only
> >>>>>>>>>>>> limits
> >>>>>>>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a
> >> system
> >>>>>>>>>> with
> >>>>>>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>>>> minimum IOPS does this.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem
> does
> >>>> not
> >>>>>>>>>>>> exist.
> >>>>>>>>>>>>>>>>>> However, I am concerned with the more general case as we
> >>>>>>>>>> supported
> >>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>> devices with provisioned IOPS.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
> >> vendor-specific
> >>>>>>>>>> values
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
> >>>>>> provisioned
> >>>>>>>>>>>>>> IOPS, but
> >>>>>>>>>>>>>>>>> what if you break them out by read and write or do
> >> something
> >>>>>>>>>> else? We
> >>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>> a more general mechanism.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
> >> throttled
> >>>>>>>>>> I/O
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
> >>>> volume.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> We also need to understand the implications of these
> QoS
> >>>>>>>>>>>> guarantees
> >>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
> >>>>>> hardware
> >>>>>>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely
> >> be
> >>>>>>>>>> forced
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
> >>>>>>>>>> implementation
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> 4.3.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some
> >> great
> >>>>>>>>>> ideas
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> 4.3. :)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning
> >> (with
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> tags).
> >>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
> >>>>>>>>>> addition to
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage
> QoS
> >>>>>> being
> >>>>>>>>>>>>>>>>>> preferred,
> >>>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available,
> >> but
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>>>> is).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in
> values
> >>>> for
> >>>>>>>>>> Wei's
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at
> least
> >>>>>>>>>> yet), so
> >>>>>>>>>>>> he
> >>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
> >> allocators
> >>>>>> to
> >>>>>>>>>>>> prefer
> >>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
> >>>> today
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
> >> features
> >>>>>>>>>> (fields
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
> >>>> Perhaps
> >>>>>>>>>> post
> >>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
> >>>>>>>>>> OpenStack
> >>>>>>>>>>>>>>>>>> addresses
> >>>>>>>>>>>>>>>>>>>>> the issue).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two
> weeks
> >>>>>> until
> >>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>> freeze)
> >>>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep
> >>>> Wei's
> >>>>>>>>>> fields
> >>>>>>>>>>>>>>>>>> and my
> >>>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature
> >> has
> >>>>>>>>>> data
> >>>>>>>>>>>>>>>>>> entered
> >>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
> >> general-purpose
> >>>>>>>>>> design
> >>>>>>>>>>>>>>>>>> that can
> >>>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
> >>>>>>>>>> situations.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I just published my review @
> >>>>>>>>>>>> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before
> coming
> >> up
> >>>>>>>>>> with a
> >>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>> file.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes
> to
> >> be
> >>>>>>>>>> checked
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
> >>>> file?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my
> review.
> >> I
> >>>> am
> >>>>>>>>>>>>>> offline
> >>>>>>>>>>>>>>>>>> ATM,
> >>>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
> >> Board?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
> >> reading
> >>>>>>>>>> this :)
> >>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all
> >> the
> >>>>>>>>>> areas
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> wanted
> >>>>>>>>>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning
> or
> >>>>>>>>>>>> afternoon,
> >>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
> >> conflicts
> >>>>>> in
> >>>>>>>>>> Java
> >>>>>>>>>>>>>>>>>> code,
> >>>>>>>>>>>>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
> >> disk_io_throttling
> >>>>>>>>>> branch
> >>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>>>>>>>> object_store
> >>>>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we
> >> merge
> >>>>>> the
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store
> goes
> >>>>>> into
> >>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits
> >> into
> >>>>>> this
> >>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John
> Burwell <
> >>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform.
> The
> >>>>>>>>>>>>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
> >>>> number
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies
> between
> >>>> the
> >>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire ->
> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management
> aspects
> >>>> of
> >>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
> >> circular
> >>>>>>>>>>>> dependency
> >>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
> >>>> graph,
> >>>>>>>>>> we
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my
> >> code
> >>>>>>>>>> into
> >>>>>>>>>>>> his
> >>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic
> code
> >>>>>>>>>> after his
> >>>>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
> >> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
> >>>> here:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
> >> hypervisor
> >>>>>>>>>> code in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
> >>>> storage
> >>>>>>>>>>>> plug-in,
> >>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in
> >> use
> >>>>>>>>>>>> (although
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.)
> is
> >>>>>>>>>> actually
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
> >> placed
> >>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>> url
> >>>>>>>>>>>>>>>>>> field
> >>>>>>>>>>>>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
> >>>>>> stored
> >>>>>>>>>> in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes
> >> the
> >>>>>>>>>> place
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
> >>>> hypervisor
> >>>>>>>>>>>> checks
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
> >> necessary
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
> >>>>>>>>>> executes to
> >>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
> >> invoked
> >>>>>> to
> >>>>>>>>>>>>>> detach a
> >>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
> >> hypervisor
> >>>>>> data
> >>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support
> Burst
> >>>>>> IOPS
> >>>>>>>>>> in
> >>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
> >> table.
> >>>>>> If
> >>>>>>>>>> we
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
> >>>>>>>>>> avoiding
> >>>>>>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
> >>>> details
> >>>>>>>>>>>> field.
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the
> documentation
> >>>>>>>>>> regarding
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
> >>>> details
> >>>>>>>>>> are
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >>>>>>>>>>>> listStoragePool
> >>>>>>>>>>>>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would
> >> be
> >>>>>>>>>> useful
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the
> details
> >>>>>>>>>> field."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst
> >>>> IOPS
> >>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>>>>>>>>>>>>> Offering-by-Disk
> >>>>>>>>>>>>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
> >>>> have
> >>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>>>>>>>> disk_offering_details
> >>>>>>>>>>>>>> table.
> >>>>>>>>>>>>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
> >>>> Burst
> >>>>>>>>>> IOPS
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are
> >> in
> >>>>>> the
> >>>>>>>>>> DB.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Please see my comments in-line below.
Thanks,
-John
On Jun 14, 2013, at 5:37 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Comments below in red (but I believe we are reaching a consensus here). :)
>
>
> On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Please see my comments in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> I am OK with that approach, John.
>>>
>>> So, let me review to make sure I follow you correctly:
>>>
>>> We introduce two new parameters to the plug-in: Number of Total IOPS for
>>> the SAN and an overcommit ratio. (On a side note, if we are just
>>> multiplying the two numbers, why don't we have the user just input their
>>> product?)
>>
>> It is a straw man suggestion to allow some fine tuning of the allocation
>> algorithm. I defer to you to determine whether or not such a tunable would
>> be valuable in this use case. My thinking was that it is tunable to
>> support operators as they more deeply understand/observe the CS workload
>> while allowing them to think about IOPS absolutely. For example, I start
>> out at say 50% overcommit and realize that the load is actually heavier
>> than expected, I can reduce it or vice versa.
>>
>
> Using your example of support people changing the value after seeing how
> the system is performing:
>
> If we had them just pass in a new Total IOPS value, that should cover it.
That is fine answer to me. I expect that we will be refining this capability over the next few releases. Therefore, I am cool with starting as simple as possible as it will ease/simplify future evolution.
>
>
>>
>>>
>>> Are both of these new parameters to the create storage pool API command
>> or
>>> are they passed into the create storage pool API command through its url
>>> parameter?
>>>
>>> If they are new parameters, we should make two new columns in the
>>> storage_pool table.
>>>
>>> If they are passed in via the url parameter, they should go in the
>>> storage_pool_details table.
>>
>> I think these allocation parameters implementation agnostic, and should be
>> columns on the storage_pool table and first class properties on the
>> StoragePool class.
>>
>
> Per my most recent e-mail, I agree.
>
>
>>
>>>
>>> For 4.2, if someone wants to change these values, they must update the DB
>>> manually.
>>
>> Is it not possible to support updates through the updateStoragePool API
>> call?
>>
>
> Edison - if you're reading these - do you know if the plug-ins are informed
> when the updateStoragePool API command is issued?
The way I am thinking about these thresholds is that they are part of the storage engine itself. Why would plugin need to know that the value changed? When the plugin needs the value, it will be retrieved from the database ...
>
>
>>
>>>
>>> Every time a volume is created or deleted for the SolidFire plug-in, the
>>> Current IOPS value (sum of all volumes' Min IOPS that are associated with
>>> the plug-in) is updated.
>>
>> I don't think we need another column for this computed value. It will be
>> simpler to simply ask the database to sum the values as needed.
>>
>
> I'm thinking now that we should follow the pattern established by the
> "size" field. There are two related size fields for a storage pool:
> capacity_bytes and available bytes.
>
> We should just follow this pattern and create two new fields: capacity_iops
> and available_iops. The plug-in can increment or decrement available_iops
> each time a volume is created or deleted, respectively.
I would rather see us do away with available_bytes. My experience with storing computed fields is that often get out of sync and represent premature optimization. I missed that part of the throttled I/O patch ...
>
>
>>
>>>
>>> The allocator can use these fields to determine if it can fit in a new
>>> volume.
>>
>> Exactly.
>>
>>>
>>> Does it look like my understanding is OK?
>>
>> Yes, looks good to me.
>>
>>>
>>>
>>> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> I apologize for not being clear -- this conversation has been admittedly
>>>> disjoint. I think we should allow the maximum IOPS and overcommitment
>>>> values to be updated though I would recommend restricting updates to to
>> be
>>>> an increasing value for 4.2 (e.g. users can increase the number of total
>>>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to 200,000).
>>>> While not ideal, given the amount of time we have left for 4.2, it will
>>>> cover most cases, and we can address the implications of reducing
>> resource
>>>> capacity in 4.3. This approach addresses both of your concerns.
>> First, it
>>>> allows the administrator/operator to determine what portion of the
>> device
>>>> they wish to dedicate. For example, if the device has a total capacity
>> of
>>>> 200,000 IOPS, and they only want CS to use 25% of the device then they
>> set
>>>> the maximum total IOPS to 50,000. Second, as they grow capacity, they
>> can
>>>> update the DataStore to increase the number of IOPS they want to
>> dedicate
>>>> to CS' use. I would imagine expansion of capacity happens infrequently
>>>> enough that increasing the maximum IOPS value would not be a significant
>>>> burden.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> "the administrator/operator simply needs to tell us the total number of
>>>>> IOPS that can be committed to it and an overcommitment factor."
>>>>>
>>>>> Are you thinking when we create a plug-in as primary storage that we
>> say
>>>> -
>>>>> up front - how many IOPS the SAN can handle?
>>>>>
>>>>> That is not a good move, in my opinion. Our SAN is designed to start
>>>> small
>>>>> and grow to PBs. As the need arises for more storage, the admin
>> purchases
>>>>> additional storage nodes that join the cluster and the performance and
>>>>> capacity go up.
>>>>>
>>>>> We need to know how many IOPS total the SAN can handle and what it is
>>>>> committed to currently (the sum of the number of volumes' min IOPS).
>>>>>
>>>>> We also cannot assume the SAN is dedicated to CS.
>>>>>
>>>>>
>>>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Simon,
>>>>>>
>>>>>> Yes, it is CloudStack's job to protect, as best it can, from
>>>>>> oversubscribing resources. I would argue that resource management is
>>>> one,
>>>>>> if not the most, important functions of the system. It is no
>> different
>>>>>> than the allocation/planning performed for hosts relative to cores and
>>>>>> memory. We can still oversubscribe resources, but we have rails +
>> knobs
>>>>>> and dials to avoid it. Without these controls in place, we could
>> easily
>>>>>> allow users to deploy workloads that overrun resources harming all
>>>> tenants.
>>>>>>
>>>>>> I also think that we are over thinking this issue for provisioned
>> IOPS.
>>>>>> When the DataStore is configured, the administrator/operator simply
>>>> needs
>>>>>> to tell us the total number of IOPS that can be committed to it and an
>>>>>> overcommitment factor. As we allocate volumes to that DataStore, we
>>>> sum up
>>>>>> the committed IOPS of the existing Volumes attached to the DataStore,
>>>> apply
>>>>>> the overcommitment factor, and determine whether or not the requested
>>>>>> minimum IOPS for the new volume can be fulfilled. We can provide both
>>>>>> general and vendor specific documentation for determining these values
>>>> --
>>>>>> be they to consume the entire device or a portion of it.
>>>>>>
>>>>>> Querying the device is unnecessary and deceptive. CloudStack resource
>>>>>> management is not interested in the current state of the device which
>>>> could
>>>>>> be anywhere from extremely heavy to extremely light at any given time.
>>>> We
>>>>>> are interested in the worst case load that is anticipated for
>> resource.
>>>> In
>>>>>> my view, it is up to administrators/operators to instrument their
>>>>>> environment to understand usage patterns and capacity. We should
>>>> provide
>>>>>> information that will help determine what should be
>>>> instrumented/monitored,
>>>>>> but that function should be performed outside of CloudStack.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>>>>>>
>>>>>>> I'd like to comment on this briefly.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I think an assumption is being made that the SAN is being dedicated
>> to
>>>> a
>>>>>> CS instance.
>>>>>>>
>>>>>>> My person opinion that this whole IOPS calculation is getting rather
>>>>>> complicated, and could probably be much simpler than this. Over
>>>>>> subscription is a fact of life on virtually all storage, and is really
>>>> no
>>>>>> different in concept than multiple virt instances on a single piece of
>>>>>> hardware. All decent SANs offer many management options for the
>> storage
>>>>>> engineers to keep track of IOPS utilization, and plan for spindle
>>>>>> augmentation as required.
>>>>>>> Is it really the job of CS to become yet another management layer on
>>>> top
>>>>>> of this?
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>
>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
>>>>>> ustcweizhou@gmail.com>
>>>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>
>>>>>>> 1) We want number of IOPS currently supported by the SAN.
>>>>>>>
>>>>>>> 2) We want the number of IOPS that are committed (sum of min IOPS for
>>>>>> each
>>>>>>> volume).
>>>>>>>
>>>>>>> We could do the following to keep track of IOPS:
>>>>>>>
>>>>>>> The plug-in could have a timer thread that goes off every, say, 1
>>>> minute.
>>>>>>>
>>>>>>> It could query the SAN for the number of nodes that make up the SAN
>> and
>>>>>>> multiple this by 50,000. This is essentially the number of supported
>>>> IOPS
>>>>>>> of the SAN.
>>>>>>>
>>>>>>> The next API call could be to get all of the volumes on the SAN.
>>>> Iterate
>>>>>>> through them all and add up their min IOPS values. This is the number
>>>> of
>>>>>>> IOPS the SAN is committed to.
>>>>>>>
>>>>>>> These two numbers can then be updated in the storage_pool table (a
>>>> column
>>>>>>> for each value).
>>>>>>>
>>>>>>> The allocators can get these values as needed (and they would be as
>>>>>>> accurate as the last time the thread asked the SAN for this info).
>>>>>>>
>>>>>>> These two fields, the min IOPS of the volume to create, and the
>>>>>> overcommit
>>>>>>> ratio of the plug-in would tell the allocator if it can select the
>>>> given
>>>>>>> storage pool.
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> "As I mentioned previously, I am very reluctant for any feature to
>>>> come
>>>>>>>> into master that can exhaust resources."
>>>>>>>>
>>>>>>>> Just wanted to mention that, worst case, the SAN would fail creation
>>>> of
>>>>>>>> the volume before allowing a new volume to break the system.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Hi John,
>>>>>>>>>
>>>>>>>>> Are you thinking we add a column on to the storage pool table,
>>>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
>>>>>>>>>
>>>>>>>>> That is easy enough.
>>>>>>>>>
>>>>>>>>> How do you want to determine what the SAN is capable of supporting
>>>> IOPS
>>>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
>>>>>> storage
>>>>>>>>> nodes to the cluster, the number of IOPS increases. Do we have a
>>>>>> thread we
>>>>>>>>> can use to query external devices like this SAN to update the
>>>> supported
>>>>>>>>> number of IOPS?
>>>>>>>>>
>>>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>>>>>>>>> overcommit ration to the plug-in when it's created? We would need
>> to
>>>>>> store
>>>>>>>>> this in the storage_pool table, as well, I believe.
>>>>>>>>>
>>>>>>>>> We should also get Wei involved in this as his feature will need
>>>>>> similar
>>>>>>>>> functionality.
>>>>>>>>>
>>>>>>>>> Also, we should do this FAST as we have only two weeks left and
>> many
>>>> of
>>>>>>>>> us will be out for several days for the CS Collab Conference.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> Querying the SAN only indicates the number of IOPS currently in
>> use.
>>>>>>>>>> The allocator needs to check the number of IOPS committed which is
>>>>>> tracked
>>>>>>>>>> by CloudStack. For 4.2, we should be able to query the number of
>>>> IOPS
>>>>>>>>>> committed to a DataStore, and determine whether or not the number
>>>>>> requested
>>>>>>>>>> can be fulfilled by that device. It seems to be that a
>>>>>>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>>>>>>>>> DataStore's that don't support provisioned IOPS would return null.
>>>>>>>>>>
>>>>>>>>>> As I mentioned previously, I am very reluctant for any feature to
>>>> come
>>>>>>>>>> into master that can exhaust resources.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yeah, I'm not sure I could come up with anything near an accurate
>>>>>>>>>>> assessment of how many IOPS are currently available on the SAN
>> (or
>>>>>>>>>> even a
>>>>>>>>>>> total number that are available for volumes). Not sure if there's
>>>> yet
>>>>>>>>>> an
>>>>>>>>>>> API call for that.
>>>>>>>>>>>
>>>>>>>>>>> If I did know this number (total number of IOPS supported by the
>>>>>> SAN),
>>>>>>>>>> we'd
>>>>>>>>>>> still have to keep track of the total number of volumes we've
>>>> created
>>>>>>>>>> from
>>>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an
>>>> API
>>>>>>>>>> call
>>>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given volume
>>>> or
>>>>>>>>>> set of
>>>>>>>>>>> volumes (not supported from CS, but supported via the SolidFire
>>>> API),
>>>>>>>>>> our
>>>>>>>>>>> numbers in CS would be off.
>>>>>>>>>>>
>>>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a really good
>>>>>> idea
>>>>>>>>>> for
>>>>>>>>>>> a future release.
>>>>>>>>>>>
>>>>>>>>>>> For 4.2 I think we should stick to having the allocator detect if
>>>>>>>>>> storage
>>>>>>>>>>> QoS is desired and if the storage pool in question supports that
>>>>>>>>>> feature.
>>>>>>>>>>>
>>>>>>>>>>> If you really are over provisioned on your SAN in terms of IOPS
>> or
>>>>>>>>>>> capacity, the SAN can let the admin know in several different
>> ways
>>>>>>>>>> (e-mail,
>>>>>>>>>>> SNMP, GUI).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
>> jburwell@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Please see my comments in-line below.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
>>>> jburwell@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we
>>>> may
>>>>>>>>>> want
>>>>>>>>>>>> to
>>>>>>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If
>> the
>>>>>>>>>> user
>>>>>>>>>>>> is
>>>>>>>>>>>>>> expecting that the hypervisor will provide a particular QoS,
>> and
>>>>>>>>>> that is
>>>>>>>>>>>>>> not possible, it seems like we should inform them rather than
>>>>>>>>>> silently
>>>>>>>>>>>>>> ignoring their request.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sure, that sounds reasonable.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We'd have to come up with some way for the allocators to know
>>>> about
>>>>>>>>>> the
>>>>>>>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any thoughts on how we might do that?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To collect my thoughts from previous parts of the thread, I am
>>>>>>>>>>>>>> uncomfortable with the idea that the management server can
>>>>>>>>>> overcommit a
>>>>>>>>>>>>>> resource. You had mentioned querying the device for available
>>>>>> IOPS.
>>>>>>>>>>>> While
>>>>>>>>>>>>>> that would be nice, it seems like we could fall back to a max
>>>> IOPS
>>>>>>>>>> and
>>>>>>>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>>>>>>>> administrator/operator. I think such threshold and allocation
>>>>>> rails
>>>>>>>>>>>> should
>>>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is
>> a
>>>>>>>>>> basic
>>>>>>>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Are you thinking this ability would make it into 4.2? Just
>>>> curious
>>>>>>>>>> what
>>>>>>>>>>>>> release we're talking about here. For the SolidFire SAN, you
>>>> might
>>>>>>>>>> have,
>>>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and
>> then
>>>>>>>>>> later
>>>>>>>>>>>> add
>>>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have
>> a
>>>>>> way
>>>>>>>>>> to
>>>>>>>>>>>>> know that the number of supported IOPS has increased.
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>>>>>>>>> example,
>>>>>>>>>>>> we may only support expanding capacity in 4.2, and not handle
>> any
>>>>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
>> Extrapolating
>>>>>>>>>> a
>>>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a similar
>>>> set
>>>>>>>>>> of
>>>>>>>>>>>> rails defined per host.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed
>> in
>>>> a
>>>>>>>>>>>>>> implementation specific manner. I think we need to arrive at a
>>>>>>>>>> general
>>>>>>>>>>>>>> model that can be exploited by the allocators and planners. We
>>>>>>>>>> should
>>>>>>>>>>>>>> restrict implementation specific key-value pairs (call them
>>>>>> details,
>>>>>>>>>>>>>> extended data, whatever) to information that is unique to the
>>>>>>>>>> driver and
>>>>>>>>>>>>>> would provide no useful information to the management server's
>>>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to fit
>>>> those
>>>>>>>>>>>>>> criteria.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I wonder if this would be a good discussion topic for Sunday's
>> CS
>>>>>>>>>> Collab
>>>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, it would -- I will put something in the wiki topic. It will
>>>>>>>>>> also be
>>>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
>> include
>>>>>>>>>> some of
>>>>>>>>>>>> my opinions on the topic.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Accept the values as they are currently required (four new
>>>> fields
>>>>>>>>>> for
>>>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) No QoS
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3) Storage Qos
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The admin needs to specify storage tags that only map to
>>>> storage
>>>>>>>>>> that
>>>>>>>>>>>>>>> supports Storage QoS.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless
>> all
>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Post 4.3:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Come up with a way to more generally enter these parameters
>>>>>>>>>> (probably
>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Have the drivers expose their feature set so the allocators
>> can
>>>>>>>>>>>> consider
>>>>>>>>>>>>>>> them more fully and throw an exception if there is not a
>>>>>> sufficient
>>>>>>>>>>>>>> match.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to
>> put
>>>>>> some
>>>>>>>>>>>>>> burden
>>>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that
>> makes
>>>>>>>>>> sense.
>>>>>>>>>>>> For
>>>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported Min
>> and
>>>>>> Max
>>>>>>>>>>>>>> values,
>>>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
>>>> SolidFire
>>>>>>>>>>>> primary
>>>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
>>>>>> then
>>>>>>>>>> he's
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>> going to get the Min and Max feature. We could add help text
>>>> to
>>>>>>>>>> the
>>>>>>>>>>>>>> pop-up
>>>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and Max
>> text
>>>>>>>>>> fields.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
>>>>>> comprehensive
>>>>>>>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Maybe in the future we could have the drivers advertise
>> their
>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>> and if the allocator feels a request is not being satisfied
>>>> (say
>>>>>>>>>> Min
>>>>>>>>>>>> was
>>>>>>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it
>> can
>>>>>>>>>> throw
>>>>>>>>>>>> an
>>>>>>>>>>>>>>>> exception.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
>> features
>>>>>> are
>>>>>>>>>>>>>>>>>> mutually
>>>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration
>>>> that
>>>>>>>>>>>>>>>>>> contains both
>>>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>>>>>>>> non-deterministic
>>>>>>>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
>>>>>> storage
>>>>>>>>>>>>>> oriented
>>>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
>> storage
>>>>>>>>>> tags,
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> have no
>>>>>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
>>>> wasn't
>>>>>>>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>>>>>>>> intending that we implement this allocation model in
>> 4.3,
>>>>>> but
>>>>>>>>>>>>>> instead,
>>>>>>>>>>>>>>>>>>>> determine if this type model would be one we would want
>> to
>>>>>>>>>> support
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the
>>>> data
>>>>>>>>>> model
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> current implementation would not preclude evolution in
>>>> that
>>>>>>>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
>> user's
>>>>>>>>>> expect
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource set to
>>>>>>>>>> fulfill a
>>>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in that what
>>>>>> we've
>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>> should not make refinement difficult in the future. If we
>>>>>> don't
>>>>>>>>>>>>>> modify
>>>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
>>>> relatively
>>>>>>>>>> simple
>>>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
>>>>>> aggregated
>>>>>>>>>>>> result
>>>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>>>>>>>>>> throttled
>>>>>>>>>>>>>> I/O and
>>>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
>>>> concern
>>>>>>>>>> is
>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>>>>>>>>>> Javadoc
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and their
>>>>>> mutual
>>>>>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>>>>>>>>>> provides
>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned
>>>> IOPS
>>>>>>>>>> uses
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> range. In order to select the best available resource to
>>>>>>>>>> fulfill a
>>>>>>>>>>>>>> QoS, we
>>>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in terms
>> of
>>>>>>>>>> units
>>>>>>>>>>>>>> (IOPS or
>>>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want
>> to
>>>>>>>>>>>> achieve a
>>>>>>>>>>>>>>>>>> model like I described, I think we would need to resolve
>>>> this
>>>>>>>>>> issue
>>>>>>>>>>>>>> in 4.2
>>>>>>>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the input
>>>> for
>>>>>>>>>> 4.2.
>>>>>>>>>>>>>> Plus
>>>>>>>>>>>>>>>>> people probably want to input the data in terms they're
>>>>>> familiar
>>>>>>>>>> with
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> the products in question.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let
>>>> the
>>>>>>>>>> user
>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected due
>> to
>>>> a
>>>>>>>>>> given
>>>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it because
>>>>>> what
>>>>>>>>>>>>>> happens if
>>>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage after
>>>>>>>>>> having
>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>>>>>>>>>> re-think
>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>>>>>>>>>> supported
>>>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> As I think through the implications of these
>> requirements
>>>>>> and
>>>>>>>>>>>>>> reflect
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>>>>>>>>>> impacted
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>>>>>>>> provisioned
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>> has no
>>>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
>> devices
>>>>>>>>>> that
>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>>>>>>>>> understand
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>>>>>>>>>> potentially
>>>>>>>>>>>>>>>>>> causing
>>>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It
>> seems
>>>>>>>>>> to me
>>>>>>>>>>>>>>>>>> that a
>>>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>>>>>>>>> maximum
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment factor.
>>>> This
>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>> should be used by the storage allocators to determine
>> the
>>>>>>>>>> device
>>>>>>>>>>>>>> best
>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems
>> that
>>>> a
>>>>>>>>>>>> similar
>>>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
>>>>>> layer
>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path
>>>> that
>>>>>>>>>> was
>>>>>>>>>>>>>>>>>> previously
>>>>>>>>>>>>>>>>>>> established for other properties you see on dialogs in
>> CS:
>>>>>> Just
>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed
>>>> to
>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>>>>>>>>> storage
>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> confines the deployment only to storage that supports the
>>>>>>>>>> selected
>>>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>>>>>>>> CloudStack
>>>>>>>>>>>>>> works
>>>>>>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for the
>>>>>>>>>>>> administrator
>>>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is that
>> we
>>>>>>>>>> over
>>>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>>>>>>>>>>>>> illustration,
>>>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
>>>>>> device
>>>>>>>>>> is
>>>>>>>>>>>>>> 100000.
>>>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>>>>>>>> Therefore,
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>>>>>>>>>> guaranteed
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>>>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
>> factor.
>>>>>>>>>> As we
>>>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>>>>>>>>> guarantee
>>>>>>>>>>>>>> more IOPS
>>>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that
>>>> make
>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm
>> not
>>>>>> even
>>>>>>>>>>>> sure
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain.
>>>> I'd
>>>>>>>>>> have
>>>>>>>>>>>> to
>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN and
>>>>>> total
>>>>>>>>>>>> them
>>>>>>>>>>>>>> up,
>>>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
>>>>>> supported
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> the system.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>>>>>>>>> resources
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>>>>>>>> think a side effect of the current implementation is
>>>>>> SolidFire
>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled
>> I/O,
>>>> it
>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
>>>> side-by-side
>>>>>>>>>> with
>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
>> potentially
>>>>>>>>>> starve
>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined
>> the
>>>>>>>>>>>> collocated
>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS)
>>>> and
>>>>>>>>>> not
>>>>>>>>>>>>>>>>>> specify
>>>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at
>>>> the
>>>>>>>>>>>>>> defaults
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire
>> SAN,
>>>>>> you
>>>>>>>>>> will
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>>>>>>>>> exclusions
>>>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
>>>> deploy
>>>>>> my
>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as
>> it
>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> complicate component assembly. It would require a storage
>>>>>>>>>> plugin to
>>>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies and
>>>>>>>>>> determine
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
>> inclusion
>>>>>> or
>>>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
>> functionality
>>>>>> in
>>>>>>>>>> a
>>>>>>>>>>>> way
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> allocator understands.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it
>>>> only
>>>>>>>>>>>> limits
>>>>>>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a
>> system
>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does
>>>> not
>>>>>>>>>>>> exist.
>>>>>>>>>>>>>>>>>> However, I am concerned with the more general case as we
>>>>>>>>>> supported
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
>> vendor-specific
>>>>>>>>>> values
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
>>>>>> provisioned
>>>>>>>>>>>>>> IOPS, but
>>>>>>>>>>>>>>>>> what if you break them out by read and write or do
>> something
>>>>>>>>>> else? We
>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
>> throttled
>>>>>>>>>> I/O
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
>>>> volume.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> We also need to understand the implications of these QoS
>>>>>>>>>>>> guarantees
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
>>>>>> hardware
>>>>>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely
>> be
>>>>>>>>>> forced
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>>>>>>>> implementation
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some
>> great
>>>>>>>>>> ideas
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning
>> (with
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> tags).
>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>>>>>>>>>> addition to
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
>>>>>> being
>>>>>>>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available,
>> but
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in values
>>>> for
>>>>>>>>>> Wei's
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>>>>>>>>>> yet), so
>>>>>>>>>>>> he
>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
>> allocators
>>>>>> to
>>>>>>>>>>>> prefer
>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
>>>> today
>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
>> features
>>>>>>>>>> (fields
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
>>>> Perhaps
>>>>>>>>>> post
>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>>>>>>>>>> OpenStack
>>>>>>>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
>>>>>> until
>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep
>>>> Wei's
>>>>>>>>>> fields
>>>>>>>>>>>>>>>>>> and my
>>>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature
>> has
>>>>>>>>>> data
>>>>>>>>>>>>>>>>>> entered
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
>> general-purpose
>>>>>>>>>> design
>>>>>>>>>>>>>>>>>> that can
>>>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>>>>>>>>>> situations.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I just published my review @
>>>>>>>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming
>> up
>>>>>>>>>> with a
>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to
>> be
>>>>>>>>>> checked
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
>>>> file?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review.
>> I
>>>> am
>>>>>>>>>>>>>> offline
>>>>>>>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
>> Board?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
>> reading
>>>>>>>>>> this :)
>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all
>> the
>>>>>>>>>> areas
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>>>>>>>>> afternoon,
>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
>> conflicts
>>>>>> in
>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
>> disk_io_throttling
>>>>>>>>>> branch
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>>>>>>> object_store
>>>>>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we
>> merge
>>>>>> the
>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
>>>>>> into
>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits
>> into
>>>>>> this
>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
>>>> number
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between
>>>> the
>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects
>>>> of
>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
>> circular
>>>>>>>>>>>> dependency
>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
>>>> graph,
>>>>>>>>>> we
>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my
>> code
>>>>>>>>>> into
>>>>>>>>>>>> his
>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>>>>>>>> after his
>>>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
>> Tutkowski
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
>>>> here:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
>> hypervisor
>>>>>>>>>> code in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
>>>> storage
>>>>>>>>>>>> plug-in,
>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in
>> use
>>>>>>>>>>>> (although
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>>>>>>>> actually
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
>> placed
>>>> in
>>>>>>>>>> the
>>>>>>>>>>>> url
>>>>>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>>>>>> stored
>>>>>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>>>>>>>> hypervisor
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes
>> the
>>>>>>>>>> place
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
>>>> hypervisor
>>>>>>>>>>>> checks
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
>> necessary
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>>>>>>>> executes to
>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
>> invoked
>>>>>> to
>>>>>>>>>>>>>> detach a
>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
>> hypervisor
>>>>>> data
>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>>>>>> IOPS
>>>>>>>>>> in
>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
>> table.
>>>>>> If
>>>>>>>>>> we
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>>>>>>>> avoiding
>>>>>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
>>>> details
>>>>>>>>>>>> field.
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>>>>>>>> regarding
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>>>> details
>>>>>>>>>> are
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>>>>>>>> listStoragePool
>>>>>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would
>> be
>>>>>>>>>> useful
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>>>>>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst
>>>> IOPS
>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
>>>> have
>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>>>>>>> disk_offering_details
>>>>>>>>>>>>>> table.
>>>>>>>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
>>>> Burst
>>>>>>>>>> IOPS
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are
>> in
>>>>>> the
>>>>>>>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Comments below in red (but I believe we are reaching a consensus here). :)
On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my comments in-line below.
>
> Thanks,
> -John
>
> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > I am OK with that approach, John.
> >
> > So, let me review to make sure I follow you correctly:
> >
> > We introduce two new parameters to the plug-in: Number of Total IOPS for
> > the SAN and an overcommit ratio. (On a side note, if we are just
> > multiplying the two numbers, why don't we have the user just input their
> > product?)
>
> It is a straw man suggestion to allow some fine tuning of the allocation
> algorithm. I defer to you to determine whether or not such a tunable would
> be valuable in this use case. My thinking was that it is tunable to
> support operators as they more deeply understand/observe the CS workload
> while allowing them to think about IOPS absolutely. For example, I start
> out at say 50% overcommit and realize that the load is actually heavier
> than expected, I can reduce it or vice versa.
>
Using your example of support people changing the value after seeing how
the system is performing:
If we had them just pass in a new Total IOPS value, that should cover it.
>
> >
> > Are both of these new parameters to the create storage pool API command
> or
> > are they passed into the create storage pool API command through its url
> > parameter?
> >
> > If they are new parameters, we should make two new columns in the
> > storage_pool table.
> >
> > If they are passed in via the url parameter, they should go in the
> > storage_pool_details table.
>
> I think these allocation parameters implementation agnostic, and should be
> columns on the storage_pool table and first class properties on the
> StoragePool class.
>
Per my most recent e-mail, I agree.
>
> >
> > For 4.2, if someone wants to change these values, they must update the DB
> > manually.
>
> Is it not possible to support updates through the updateStoragePool API
> call?
>
Edison - if you're reading these - do you know if the plug-ins are informed
when the updateStoragePool API command is issued?
>
> >
> > Every time a volume is created or deleted for the SolidFire plug-in, the
> > Current IOPS value (sum of all volumes' Min IOPS that are associated with
> > the plug-in) is updated.
>
> I don't think we need another column for this computed value. It will be
> simpler to simply ask the database to sum the values as needed.
>
I'm thinking now that we should follow the pattern established by the
"size" field. There are two related size fields for a storage pool:
capacity_bytes and available bytes.
We should just follow this pattern and create two new fields: capacity_iops
and available_iops. The plug-in can increment or decrement available_iops
each time a volume is created or deleted, respectively.
>
> >
> > The allocator can use these fields to determine if it can fit in a new
> > volume.
>
> Exactly.
>
> >
> > Does it look like my understanding is OK?
>
> Yes, looks good to me.
>
> >
> >
> > On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> I apologize for not being clear -- this conversation has been admittedly
> >> disjoint. I think we should allow the maximum IOPS and overcommitment
> >> values to be updated though I would recommend restricting updates to to
> be
> >> an increasing value for 4.2 (e.g. users can increase the number of total
> >> IOPS from 200,000 to 250,000, but not decrease from 250,000 to 200,000).
> >> While not ideal, given the amount of time we have left for 4.2, it will
> >> cover most cases, and we can address the implications of reducing
> resource
> >> capacity in 4.3. This approach addresses both of your concerns.
> First, it
> >> allows the administrator/operator to determine what portion of the
> device
> >> they wish to dedicate. For example, if the device has a total capacity
> of
> >> 200,000 IOPS, and they only want CS to use 25% of the device then they
> set
> >> the maximum total IOPS to 50,000. Second, as they grow capacity, they
> can
> >> update the DataStore to increase the number of IOPS they want to
> dedicate
> >> to CS' use. I would imagine expansion of capacity happens infrequently
> >> enough that increasing the maximum IOPS value would not be a significant
> >> burden.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> "the administrator/operator simply needs to tell us the total number of
> >>> IOPS that can be committed to it and an overcommitment factor."
> >>>
> >>> Are you thinking when we create a plug-in as primary storage that we
> say
> >> -
> >>> up front - how many IOPS the SAN can handle?
> >>>
> >>> That is not a good move, in my opinion. Our SAN is designed to start
> >> small
> >>> and grow to PBs. As the need arises for more storage, the admin
> purchases
> >>> additional storage nodes that join the cluster and the performance and
> >>> capacity go up.
> >>>
> >>> We need to know how many IOPS total the SAN can handle and what it is
> >>> committed to currently (the sum of the number of volumes' min IOPS).
> >>>
> >>> We also cannot assume the SAN is dedicated to CS.
> >>>
> >>>
> >>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Simon,
> >>>>
> >>>> Yes, it is CloudStack's job to protect, as best it can, from
> >>>> oversubscribing resources. I would argue that resource management is
> >> one,
> >>>> if not the most, important functions of the system. It is no
> different
> >>>> than the allocation/planning performed for hosts relative to cores and
> >>>> memory. We can still oversubscribe resources, but we have rails +
> knobs
> >>>> and dials to avoid it. Without these controls in place, we could
> easily
> >>>> allow users to deploy workloads that overrun resources harming all
> >> tenants.
> >>>>
> >>>> I also think that we are over thinking this issue for provisioned
> IOPS.
> >>>> When the DataStore is configured, the administrator/operator simply
> >> needs
> >>>> to tell us the total number of IOPS that can be committed to it and an
> >>>> overcommitment factor. As we allocate volumes to that DataStore, we
> >> sum up
> >>>> the committed IOPS of the existing Volumes attached to the DataStore,
> >> apply
> >>>> the overcommitment factor, and determine whether or not the requested
> >>>> minimum IOPS for the new volume can be fulfilled. We can provide both
> >>>> general and vendor specific documentation for determining these values
> >> --
> >>>> be they to consume the entire device or a portion of it.
> >>>>
> >>>> Querying the device is unnecessary and deceptive. CloudStack resource
> >>>> management is not interested in the current state of the device which
> >> could
> >>>> be anywhere from extremely heavy to extremely light at any given time.
> >> We
> >>>> are interested in the worst case load that is anticipated for
> resource.
> >> In
> >>>> my view, it is up to administrators/operators to instrument their
> >>>> environment to understand usage patterns and capacity. We should
> >> provide
> >>>> information that will help determine what should be
> >> instrumented/monitored,
> >>>> but that function should be performed outside of CloudStack.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
> >>>>
> >>>>> I'd like to comment on this briefly.
> >>>>>
> >>>>>
> >>>>>
> >>>>> I think an assumption is being made that the SAN is being dedicated
> to
> >> a
> >>>> CS instance.
> >>>>>
> >>>>> My person opinion that this whole IOPS calculation is getting rather
> >>>> complicated, and could probably be much simpler than this. Over
> >>>> subscription is a fact of life on virtually all storage, and is really
> >> no
> >>>> different in concept than multiple virt instances on a single piece of
> >>>> hardware. All decent SANs offer many management options for the
> storage
> >>>> engineers to keep track of IOPS utilization, and plan for spindle
> >>>> augmentation as required.
> >>>>> Is it really the job of CS to become yet another management layer on
> >> top
> >>>> of this?
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>
> >>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>> To: dev@cloudstack.apache.org
> >>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
> >>>> ustcweizhou@gmail.com>
> >>>>> Sent: Friday, June 14, 2013 1:00:26 PM
> >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>
> >>>>> 1) We want number of IOPS currently supported by the SAN.
> >>>>>
> >>>>> 2) We want the number of IOPS that are committed (sum of min IOPS for
> >>>> each
> >>>>> volume).
> >>>>>
> >>>>> We could do the following to keep track of IOPS:
> >>>>>
> >>>>> The plug-in could have a timer thread that goes off every, say, 1
> >> minute.
> >>>>>
> >>>>> It could query the SAN for the number of nodes that make up the SAN
> and
> >>>>> multiple this by 50,000. This is essentially the number of supported
> >> IOPS
> >>>>> of the SAN.
> >>>>>
> >>>>> The next API call could be to get all of the volumes on the SAN.
> >> Iterate
> >>>>> through them all and add up their min IOPS values. This is the number
> >> of
> >>>>> IOPS the SAN is committed to.
> >>>>>
> >>>>> These two numbers can then be updated in the storage_pool table (a
> >> column
> >>>>> for each value).
> >>>>>
> >>>>> The allocators can get these values as needed (and they would be as
> >>>>> accurate as the last time the thread asked the SAN for this info).
> >>>>>
> >>>>> These two fields, the min IOPS of the volume to create, and the
> >>>> overcommit
> >>>>> ratio of the plug-in would tell the allocator if it can select the
> >> given
> >>>>> storage pool.
> >>>>>
> >>>>> What do you think?
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>>> "As I mentioned previously, I am very reluctant for any feature to
> >> come
> >>>>>> into master that can exhaust resources."
> >>>>>>
> >>>>>> Just wanted to mention that, worst case, the SAN would fail creation
> >> of
> >>>>>> the volume before allowing a new volume to break the system.
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Hi John,
> >>>>>>>
> >>>>>>> Are you thinking we add a column on to the storage pool table,
> >>>>>>> IOPS_Count, where we add and subtract committed IOPS?
> >>>>>>>
> >>>>>>> That is easy enough.
> >>>>>>>
> >>>>>>> How do you want to determine what the SAN is capable of supporting
> >> IOPS
> >>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
> >>>> storage
> >>>>>>> nodes to the cluster, the number of IOPS increases. Do we have a
> >>>> thread we
> >>>>>>> can use to query external devices like this SAN to update the
> >> supported
> >>>>>>> number of IOPS?
> >>>>>>>
> >>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
> >>>>>>> overcommit ration to the plug-in when it's created? We would need
> to
> >>>> store
> >>>>>>> this in the storage_pool table, as well, I believe.
> >>>>>>>
> >>>>>>> We should also get Wei involved in this as his feature will need
> >>>> similar
> >>>>>>> functionality.
> >>>>>>>
> >>>>>>> Also, we should do this FAST as we have only two weeks left and
> many
> >> of
> >>>>>>> us will be out for several days for the CS Collab Conference.
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> Querying the SAN only indicates the number of IOPS currently in
> use.
> >>>>>>>> The allocator needs to check the number of IOPS committed which is
> >>>> tracked
> >>>>>>>> by CloudStack. For 4.2, we should be able to query the number of
> >> IOPS
> >>>>>>>> committed to a DataStore, and determine whether or not the number
> >>>> requested
> >>>>>>>> can be fulfilled by that device. It seems to be that a
> >>>>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
> >>>>>>>> DataStore's that don't support provisioned IOPS would return null.
> >>>>>>>>
> >>>>>>>> As I mentioned previously, I am very reluctant for any feature to
> >> come
> >>>>>>>> into master that can exhaust resources.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Yeah, I'm not sure I could come up with anything near an accurate
> >>>>>>>>> assessment of how many IOPS are currently available on the SAN
> (or
> >>>>>>>> even a
> >>>>>>>>> total number that are available for volumes). Not sure if there's
> >> yet
> >>>>>>>> an
> >>>>>>>>> API call for that.
> >>>>>>>>>
> >>>>>>>>> If I did know this number (total number of IOPS supported by the
> >>>> SAN),
> >>>>>>>> we'd
> >>>>>>>>> still have to keep track of the total number of volumes we've
> >> created
> >>>>>>>> from
> >>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an
> >> API
> >>>>>>>> call
> >>>>>>>>> directly to the SAN to tweak the number of IOPS on a given volume
> >> or
> >>>>>>>> set of
> >>>>>>>>> volumes (not supported from CS, but supported via the SolidFire
> >> API),
> >>>>>>>> our
> >>>>>>>>> numbers in CS would be off.
> >>>>>>>>>
> >>>>>>>>> I'm thinking verifying sufficient number of IOPS is a really good
> >>>> idea
> >>>>>>>> for
> >>>>>>>>> a future release.
> >>>>>>>>>
> >>>>>>>>> For 4.2 I think we should stick to having the allocator detect if
> >>>>>>>> storage
> >>>>>>>>> QoS is desired and if the storage pool in question supports that
> >>>>>>>> feature.
> >>>>>>>>>
> >>>>>>>>> If you really are over provisioned on your SAN in terms of IOPS
> or
> >>>>>>>>> capacity, the SAN can let the admin know in several different
> ways
> >>>>>>>> (e-mail,
> >>>>>>>>> SNMP, GUI).
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
> jburwell@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> Please see my comments in-line below.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
> >> jburwell@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we
> >> may
> >>>>>>>> want
> >>>>>>>>>> to
> >>>>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If
> the
> >>>>>>>> user
> >>>>>>>>>> is
> >>>>>>>>>>>> expecting that the hypervisor will provide a particular QoS,
> and
> >>>>>>>> that is
> >>>>>>>>>>>> not possible, it seems like we should inform them rather than
> >>>>>>>> silently
> >>>>>>>>>>>> ignoring their request.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Sure, that sounds reasonable.
> >>>>>>>>>>>
> >>>>>>>>>>> We'd have to come up with some way for the allocators to know
> >> about
> >>>>>>>> the
> >>>>>>>>>>> requested storage QoS and then query the candidate drivers.
> >>>>>>>>>>>
> >>>>>>>>>>> Any thoughts on how we might do that?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> To collect my thoughts from previous parts of the thread, I am
> >>>>>>>>>>>> uncomfortable with the idea that the management server can
> >>>>>>>> overcommit a
> >>>>>>>>>>>> resource. You had mentioned querying the device for available
> >>>> IOPS.
> >>>>>>>>>> While
> >>>>>>>>>>>> that would be nice, it seems like we could fall back to a max
> >> IOPS
> >>>>>>>> and
> >>>>>>>>>>>> overcommit factor manually calculated and entered by the
> >>>>>>>>>>>> administrator/operator. I think such threshold and allocation
> >>>> rails
> >>>>>>>>>> should
> >>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is
> a
> >>>>>>>> basic
> >>>>>>>>>>>> feature of any cloud orchestration platform.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Are you thinking this ability would make it into 4.2? Just
> >> curious
> >>>>>>>> what
> >>>>>>>>>>> release we're talking about here. For the SolidFire SAN, you
> >> might
> >>>>>>>> have,
> >>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and
> then
> >>>>>>>> later
> >>>>>>>>>> add
> >>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have
> a
> >>>> way
> >>>>>>>> to
> >>>>>>>>>>> know that the number of supported IOPS has increased.
> >>>>>>>>>>
> >>>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
> >>>>>>>> example,
> >>>>>>>>>> we may only support expanding capacity in 4.2, and not handle
> any
> >>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
> Extrapolating
> >>>>>>>> a
> >>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a similar
> >> set
> >>>>>>>> of
> >>>>>>>>>> rails defined per host.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed
> in
> >> a
> >>>>>>>>>>>> implementation specific manner. I think we need to arrive at a
> >>>>>>>> general
> >>>>>>>>>>>> model that can be exploited by the allocators and planners. We
> >>>>>>>> should
> >>>>>>>>>>>> restrict implementation specific key-value pairs (call them
> >>>> details,
> >>>>>>>>>>>> extended data, whatever) to information that is unique to the
> >>>>>>>> driver and
> >>>>>>>>>>>> would provide no useful information to the management server's
> >>>>>>>>>>>> orchestration functions. A resource QoS does not seem to fit
> >> those
> >>>>>>>>>>>> criteria.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I wonder if this would be a good discussion topic for Sunday's
> CS
> >>>>>>>> Collab
> >>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
> >>>>>>>>>>
> >>>>>>>>>> Yes, it would -- I will put something in the wiki topic. It will
> >>>>>>>> also be
> >>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
> include
> >>>>>>>> some of
> >>>>>>>>>> my opinions on the topic.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> So, here's my suggestion for 4.2:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Accept the values as they are currently required (four new
> >> fields
> >>>>>>>> for
> >>>>>>>>>>>> Wei's
> >>>>>>>>>>>>> feature or two new fields for mine).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1) No QoS
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2) Hypervisor QoS
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 3) Storage Qos
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The admin needs to specify storage tags that only map to
> >> storage
> >>>>>>>> that
> >>>>>>>>>>>>> supports Storage QoS.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless
> all
> >>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>> in use support the new fields, they may not be enforced.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Post 4.3:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Come up with a way to more generally enter these parameters
> >>>>>>>> (probably
> >>>>>>>>>>>> just
> >>>>>>>>>>>>> key/value pairs sent to the drivers).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Have the drivers expose their feature set so the allocators
> can
> >>>>>>>>>> consider
> >>>>>>>>>>>>> them more fully and throw an exception if there is not a
> >>>> sufficient
> >>>>>>>>>>>> match.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to
> put
> >>>> some
> >>>>>>>>>>>> burden
> >>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that
> makes
> >>>>>>>> sense.
> >>>>>>>>>> For
> >>>>>>>>>>>>>> example, if he wants to use storage QoS with supported Min
> and
> >>>> Max
> >>>>>>>>>>>> values,
> >>>>>>>>>>>>>> he'll have to put in a storage tag that references the
> >> SolidFire
> >>>>>>>>>> primary
> >>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
> >>>> then
> >>>>>>>> he's
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>> going to get the Min and Max feature. We could add help text
> >> to
> >>>>>>>> the
> >>>>>>>>>>>> pop-up
> >>>>>>>>>>>>>> dialog that's displayed when you click in the Min and Max
> text
> >>>>>>>> fields.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Same idea for Wei's feature.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
> >>>> comprehensive
> >>>>>>>>>>>>>> approach for post 4.2.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Maybe in the future we could have the drivers advertise
> their
> >>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>> and if the allocator feels a request is not being satisfied
> >> (say
> >>>>>>>> Min
> >>>>>>>>>> was
> >>>>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it
> can
> >>>>>>>> throw
> >>>>>>>>>> an
> >>>>>>>>>>>>>> exception.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Please see my comment in-line below.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I've put comments below in red.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>>>>>>> jburwell@basho.com
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
> features
> >>>> are
> >>>>>>>>>>>>>>>> mutually
> >>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration
> >> that
> >>>>>>>>>>>>>>>> contains both
> >>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >>>>>>>>>>>> non-deterministic
> >>>>>>>>>>>>>>>>>> operations.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
> >>>> storage
> >>>>>>>>>>>> oriented
> >>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
> storage
> >>>>>>>> tags,
> >>>>>>>>>> we
> >>>>>>>>>>>>>>>> have no
> >>>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
> >> wasn't
> >>>>>>>>>>>>>>>> necessarily
> >>>>>>>>>>>>>>>>>> intending that we implement this allocation model in
> 4.3,
> >>>> but
> >>>>>>>>>>>> instead,
> >>>>>>>>>>>>>>>>>> determine if this type model would be one we would want
> to
> >>>>>>>> support
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the
> >> data
> >>>>>>>> model
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> current implementation would not preclude evolution in
> >> that
> >>>>>>>>>>>>>>>> direction. My
> >>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
> user's
> >>>>>>>> expect
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>> "cloud"
> >>>>>>>>>>>>>>>>>> systems -- selecting the best available resource set to
> >>>>>>>> fulfill a
> >>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>> system requirements.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I believe we have meet your requirement here in that what
> >>>> we've
> >>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>> should not make refinement difficult in the future. If we
> >>>> don't
> >>>>>>>>>>>> modify
> >>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
> >> relatively
> >>>>>>>> simple
> >>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
> >>>> aggregated
> >>>>>>>>>> result
> >>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
> >>>>>>>> throttled
> >>>>>>>>>>>> I/O and
> >>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
> >> concern
> >>>>>>>> is
> >>>>>>>>>> not
> >>>>>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
> >>>>>>>> Javadoc
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> be added explaining the two sets of attributes and their
> >>>> mutual
> >>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
> >>>>>>>> exclusion.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
> >>>>>>>> provides
> >>>>>>>>>> both
> >>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned
> >> IOPS
> >>>>>>>> uses
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>> range. In order to select the best available resource to
> >>>>>>>> fulfill a
> >>>>>>>>>>>> QoS, we
> >>>>>>>>>>>>>>>> would need to have the QoS expression normalized in terms
> of
> >>>>>>>> units
> >>>>>>>>>>>> (IOPS or
> >>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want
> to
> >>>>>>>>>> achieve a
> >>>>>>>>>>>>>>>> model like I described, I think we would need to resolve
> >> this
> >>>>>>>> issue
> >>>>>>>>>>>> in 4.2
> >>>>>>>>>>>>>>>> to ensure a viable migration path.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think we're not likely to be able to normalize the input
> >> for
> >>>>>>>> 4.2.
> >>>>>>>>>>>> Plus
> >>>>>>>>>>>>>>> people probably want to input the data in terms they're
> >>>> familiar
> >>>>>>>> with
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> the products in question.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let
> >> the
> >>>>>>>> user
> >>>>>>>>>>>> send
> >>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected due
> to
> >> a
> >>>>>>>> given
> >>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it because
> >>>> what
> >>>>>>>>>>>> happens if
> >>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage after
> >>>>>>>> having
> >>>>>>>>>>>> created
> >>>>>>>>>>>>>>> a Disk Offering?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
> >>>>>>>> re-think
> >>>>>>>>>> it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
> >>>>>>>> supported
> >>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> As I think through the implications of these
> requirements
> >>>> and
> >>>>>>>>>>>> reflect
> >>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
> >>>>>>>> impacted
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
> >>>>>>>> provisioned
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> has no
> >>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
> devices
> >>>>>>>> that
> >>>>>>>>>> have
> >>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
> >>>>>>>> understand
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
> >>>>>>>> potentially
> >>>>>>>>>>>>>>>> causing
> >>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It
> seems
> >>>>>>>> to me
> >>>>>>>>>>>>>>>> that a
> >>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
> >>>>>>>> maximum
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>> it can support and some type of overcommitment factor.
> >> This
> >>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>> should be used by the storage allocators to determine
> the
> >>>>>>>> device
> >>>>>>>>>>>> best
> >>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems
> that
> >> a
> >>>>>>>>>> similar
> >>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
> >>>> layer
> >>>>>>>>>> which
> >>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path
> >> that
> >>>>>>>> was
> >>>>>>>>>>>>>>>> previously
> >>>>>>>>>>>>>>>>> established for other properties you see on dialogs in
> CS:
> >>>> Just
> >>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed
> >> to
> >>>>>>>>>> supports
> >>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
> >>>>>>>> storage
> >>>>>>>>>> tag
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> confines the deployment only to storage that supports the
> >>>>>>>> selected
> >>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
> >>>>>>>> CloudStack
> >>>>>>>>>>>> works
> >>>>>>>>>>>>>>>>> today.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I understand the tag functionality, and the need for the
> >>>>>>>>>> administrator
> >>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is that
> we
> >>>>>>>> over
> >>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
> >>>>>>>>>>>> illustration,
> >>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
> >>>> device
> >>>>>>>> is
> >>>>>>>>>>>> 100000.
> >>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
> >>>>>>>> Therefore,
> >>>>>>>>>> we
> >>>>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
> >>>>>>>> guaranteed
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
> >>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
> factor.
> >>>>>>>> As we
> >>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
> >>>>>>>> guarantee
> >>>>>>>>>>>> more IOPS
> >>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that
> >> make
> >>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm
> not
> >>>> even
> >>>>>>>>>> sure
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain.
> >> I'd
> >>>>>>>> have
> >>>>>>>>>> to
> >>>>>>>>>>>> get
> >>>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN and
> >>>> total
> >>>>>>>>>> them
> >>>>>>>>>>>> up,
> >>>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
> >>>> supported
> >>>>>>>>>> IOPS
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>> the system.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
> >>>>>>>> resources
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
> >>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>> IOPS, I
> >>>>>>>>>>>>>>>>>> think a side effect of the current implementation is
> >>>> SolidFire
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled
> I/O,
> >> it
> >>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
> >> side-by-side
> >>>>>>>> with
> >>>>>>>>>> QoS
> >>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
> potentially
> >>>>>>>> starve
> >>>>>>>>>>>> out
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined
> the
> >>>>>>>>>> collocated
> >>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>> from being fulfilled.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS)
> >> and
> >>>>>>>> not
> >>>>>>>>>>>>>>>> specify
> >>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at
> >> the
> >>>>>>>>>>>> defaults
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire
> SAN,
> >>>> you
> >>>>>>>> will
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> your Mins met.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
> >>>>>>>> exclusions
> >>>>>>>>>>>>>>>> list,
> >>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
> >> deploy
> >>>> my
> >>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> storage with these following tags).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as
> it
> >>>>>>>> would
> >>>>>>>>>>>>>>>> complicate component assembly. It would require a storage
> >>>>>>>> plugin to
> >>>>>>>>>>>>>>>> anticipate all of the possible component assemblies and
> >>>>>>>> determine
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
> >>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>> which can be matched to a set of requested requirements.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
> inclusion
> >>>> or
> >>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
> functionality
> >>>> in
> >>>>>>>> a
> >>>>>>>>>> way
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> allocator understands.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it
> >> only
> >>>>>>>>>> limits
> >>>>>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a
> system
> >>>>>>>> with
> >>>>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>> minimum IOPS does this.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does
> >> not
> >>>>>>>>>> exist.
> >>>>>>>>>>>>>>>> However, I am concerned with the more general case as we
> >>>>>>>> supported
> >>>>>>>>>>>> more
> >>>>>>>>>>>>>>>> devices with provisioned IOPS.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
> vendor-specific
> >>>>>>>> values
> >>>>>>>>>> to
> >>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
> >>>> provisioned
> >>>>>>>>>>>> IOPS, but
> >>>>>>>>>>>>>>> what if you break them out by read and write or do
> something
> >>>>>>>> else? We
> >>>>>>>>>>>> need
> >>>>>>>>>>>>>>> a more general mechanism.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
> throttled
> >>>>>>>> I/O
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
> >> volume.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> We also need to understand the implications of these QoS
> >>>>>>>>>> guarantees
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
> >>>> hardware
> >>>>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely
> be
> >>>>>>>> forced
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
> >>>>>>>> implementation
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>> 4.3.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some
> great
> >>>>>>>> ideas
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>> 4.3. :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning
> (with
> >>>>>>>> storage
> >>>>>>>>>>>>>>>> tags).
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
> >>>>>>>> addition to
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
> >>>> being
> >>>>>>>>>>>>>>>> preferred,
> >>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available,
> but
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>> is).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in values
> >> for
> >>>>>>>> Wei's
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
> >>>>>>>> yet), so
> >>>>>>>>>> he
> >>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
> allocators
> >>>> to
> >>>>>>>>>> prefer
> >>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
> >> today
> >>>>>>>> in
> >>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
> features
> >>>>>>>> (fields
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
> >> Perhaps
> >>>>>>>> post
> >>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
> >>>>>>>> OpenStack
> >>>>>>>>>>>>>>>> addresses
> >>>>>>>>>>>>>>>>>>> the issue).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
> >>>> until
> >>>>>>>>>> code
> >>>>>>>>>>>>>>>>>> freeze)
> >>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep
> >> Wei's
> >>>>>>>> fields
> >>>>>>>>>>>>>>>> and my
> >>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature
> has
> >>>>>>>> data
> >>>>>>>>>>>>>>>> entered
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
> general-purpose
> >>>>>>>> design
> >>>>>>>>>>>>>>>> that can
> >>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
> >>>>>>>> situations.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I just published my review @
> >>>>>>>>>> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming
> up
> >>>>>>>> with a
> >>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>> file.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to
> be
> >>>>>>>> checked
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
> >> file?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review.
> I
> >> am
> >>>>>>>>>>>> offline
> >>>>>>>>>>>>>>>> ATM,
> >>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
> Board?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
> reading
> >>>>>>>> this :)
> >>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all
> the
> >>>>>>>> areas
> >>>>>>>>>> we
> >>>>>>>>>>>>>>>> wanted
> >>>>>>>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> >>>>>>>>>> afternoon,
> >>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
> conflicts
> >>>> in
> >>>>>>>> Java
> >>>>>>>>>>>>>>>> code,
> >>>>>>>>>>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
> disk_io_throttling
> >>>>>>>> branch
> >>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>>>>>> object_store
> >>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we
> merge
> >>>> the
> >>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
> >>>> into
> >>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits
> into
> >>>> this
> >>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>>>>>>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
> >> number
> >>>>>>>> of
> >>>>>>>>>>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between
> >> the
> >>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects
> >> of
> >>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
> circular
> >>>>>>>>>> dependency
> >>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
> >> graph,
> >>>>>>>> we
> >>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my
> code
> >>>>>>>> into
> >>>>>>>>>> his
> >>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
> >>>>>>>> after his
> >>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
> Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
> >> here:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
> hypervisor
> >>>>>>>> code in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
> >> storage
> >>>>>>>>>> plug-in,
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in
> use
> >>>>>>>>>> (although
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
> >>>>>>>> actually
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
> placed
> >> in
> >>>>>>>> the
> >>>>>>>>>> url
> >>>>>>>>>>>>>>>> field
> >>>>>>>>>>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
> >>>> stored
> >>>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
> >>>>>>>> hypervisor
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes
> the
> >>>>>>>> place
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
> >> hypervisor
> >>>>>>>>>> checks
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
> necessary
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
> >>>>>>>> executes to
> >>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
> invoked
> >>>> to
> >>>>>>>>>>>> detach a
> >>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
> hypervisor
> >>>> data
> >>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
> >>>> IOPS
> >>>>>>>> in
> >>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
> table.
> >>>> If
> >>>>>>>> we
> >>>>>>>>>>>> have
> >>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
> >>>>>>>> avoiding
> >>>>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
> >> details
> >>>>>>>>>> field.
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
> >>>>>>>> regarding
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
> >> details
> >>>>>>>> are
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >>>>>>>>>> listStoragePool
> >>>>>>>>>>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would
> be
> >>>>>>>> useful
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
> >>>>>>>> field."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst
> >> IOPS
> >>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>>>>>>>>>>> Offering-by-Disk
> >>>>>>>>>>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
> >> have
> >>>>>>>> to be
> >>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>>>>>> disk_offering_details
> >>>>>>>>>>>> table.
> >>>>>>>>>>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
> >> Burst
> >>>>>>>> IOPS
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are
> in
> >>>> the
> >>>>>>>> DB.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Please see my comments in-line below.
Thanks,
-John
On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> I am OK with that approach, John.
>
> So, let me review to make sure I follow you correctly:
>
> We introduce two new parameters to the plug-in: Number of Total IOPS for
> the SAN and an overcommit ratio. (On a side note, if we are just
> multiplying the two numbers, why don't we have the user just input their
> product?)
It is a straw man suggestion to allow some fine tuning of the allocation algorithm. I defer to you to determine whether or not such a tunable would be valuable in this use case. My thinking was that it is tunable to support operators as they more deeply understand/observe the CS workload while allowing them to think about IOPS absolutely. For example, I start out at say 50% overcommit and realize that the load is actually heavier than expected, I can reduce it or vice versa.
>
> Are both of these new parameters to the create storage pool API command or
> are they passed into the create storage pool API command through its url
> parameter?
>
> If they are new parameters, we should make two new columns in the
> storage_pool table.
>
> If they are passed in via the url parameter, they should go in the
> storage_pool_details table.
I think these allocation parameters implementation agnostic, and should be columns on the storage_pool table and first class properties on the StoragePool class.
>
> For 4.2, if someone wants to change these values, they must update the DB
> manually.
Is it not possible to support updates through the updateStoragePool API call?
>
> Every time a volume is created or deleted for the SolidFire plug-in, the
> Current IOPS value (sum of all volumes' Min IOPS that are associated with
> the plug-in) is updated.
I don't think we need another column for this computed value. It will be simpler to simply ask the database to sum the values as needed.
>
> The allocator can use these fields to determine if it can fit in a new
> volume.
Exactly.
>
> Does it look like my understanding is OK?
Yes, looks good to me.
>
>
> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I apologize for not being clear -- this conversation has been admittedly
>> disjoint. I think we should allow the maximum IOPS and overcommitment
>> values to be updated though I would recommend restricting updates to to be
>> an increasing value for 4.2 (e.g. users can increase the number of total
>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to 200,000).
>> While not ideal, given the amount of time we have left for 4.2, it will
>> cover most cases, and we can address the implications of reducing resource
>> capacity in 4.3. This approach addresses both of your concerns. First, it
>> allows the administrator/operator to determine what portion of the device
>> they wish to dedicate. For example, if the device has a total capacity of
>> 200,000 IOPS, and they only want CS to use 25% of the device then they set
>> the maximum total IOPS to 50,000. Second, as they grow capacity, they can
>> update the DataStore to increase the number of IOPS they want to dedicate
>> to CS' use. I would imagine expansion of capacity happens infrequently
>> enough that increasing the maximum IOPS value would not be a significant
>> burden.
>>
>> Thanks,
>> -John
>>
>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> "the administrator/operator simply needs to tell us the total number of
>>> IOPS that can be committed to it and an overcommitment factor."
>>>
>>> Are you thinking when we create a plug-in as primary storage that we say
>> -
>>> up front - how many IOPS the SAN can handle?
>>>
>>> That is not a good move, in my opinion. Our SAN is designed to start
>> small
>>> and grow to PBs. As the need arises for more storage, the admin purchases
>>> additional storage nodes that join the cluster and the performance and
>>> capacity go up.
>>>
>>> We need to know how many IOPS total the SAN can handle and what it is
>>> committed to currently (the sum of the number of volumes' min IOPS).
>>>
>>> We also cannot assume the SAN is dedicated to CS.
>>>
>>>
>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Simon,
>>>>
>>>> Yes, it is CloudStack's job to protect, as best it can, from
>>>> oversubscribing resources. I would argue that resource management is
>> one,
>>>> if not the most, important functions of the system. It is no different
>>>> than the allocation/planning performed for hosts relative to cores and
>>>> memory. We can still oversubscribe resources, but we have rails + knobs
>>>> and dials to avoid it. Without these controls in place, we could easily
>>>> allow users to deploy workloads that overrun resources harming all
>> tenants.
>>>>
>>>> I also think that we are over thinking this issue for provisioned IOPS.
>>>> When the DataStore is configured, the administrator/operator simply
>> needs
>>>> to tell us the total number of IOPS that can be committed to it and an
>>>> overcommitment factor. As we allocate volumes to that DataStore, we
>> sum up
>>>> the committed IOPS of the existing Volumes attached to the DataStore,
>> apply
>>>> the overcommitment factor, and determine whether or not the requested
>>>> minimum IOPS for the new volume can be fulfilled. We can provide both
>>>> general and vendor specific documentation for determining these values
>> --
>>>> be they to consume the entire device or a portion of it.
>>>>
>>>> Querying the device is unnecessary and deceptive. CloudStack resource
>>>> management is not interested in the current state of the device which
>> could
>>>> be anywhere from extremely heavy to extremely light at any given time.
>> We
>>>> are interested in the worst case load that is anticipated for resource.
>> In
>>>> my view, it is up to administrators/operators to instrument their
>>>> environment to understand usage patterns and capacity. We should
>> provide
>>>> information that will help determine what should be
>> instrumented/monitored,
>>>> but that function should be performed outside of CloudStack.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>>>>
>>>>> I'd like to comment on this briefly.
>>>>>
>>>>>
>>>>>
>>>>> I think an assumption is being made that the SAN is being dedicated to
>> a
>>>> CS instance.
>>>>>
>>>>> My person opinion that this whole IOPS calculation is getting rather
>>>> complicated, and could probably be much simpler than this. Over
>>>> subscription is a fact of life on virtually all storage, and is really
>> no
>>>> different in concept than multiple virt instances on a single piece of
>>>> hardware. All decent SANs offer many management options for the storage
>>>> engineers to keep track of IOPS utilization, and plan for spindle
>>>> augmentation as required.
>>>>> Is it really the job of CS to become yet another management layer on
>> top
>>>> of this?
>>>>>
>>>>> ----- Original Message -----
>>>>>
>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>> To: dev@cloudstack.apache.org
>>>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
>>>> ustcweizhou@gmail.com>
>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>
>>>>> 1) We want number of IOPS currently supported by the SAN.
>>>>>
>>>>> 2) We want the number of IOPS that are committed (sum of min IOPS for
>>>> each
>>>>> volume).
>>>>>
>>>>> We could do the following to keep track of IOPS:
>>>>>
>>>>> The plug-in could have a timer thread that goes off every, say, 1
>> minute.
>>>>>
>>>>> It could query the SAN for the number of nodes that make up the SAN and
>>>>> multiple this by 50,000. This is essentially the number of supported
>> IOPS
>>>>> of the SAN.
>>>>>
>>>>> The next API call could be to get all of the volumes on the SAN.
>> Iterate
>>>>> through them all and add up their min IOPS values. This is the number
>> of
>>>>> IOPS the SAN is committed to.
>>>>>
>>>>> These two numbers can then be updated in the storage_pool table (a
>> column
>>>>> for each value).
>>>>>
>>>>> The allocators can get these values as needed (and they would be as
>>>>> accurate as the last time the thread asked the SAN for this info).
>>>>>
>>>>> These two fields, the min IOPS of the volume to create, and the
>>>> overcommit
>>>>> ratio of the plug-in would tell the allocator if it can select the
>> given
>>>>> storage pool.
>>>>>
>>>>> What do you think?
>>>>>
>>>>>
>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> "As I mentioned previously, I am very reluctant for any feature to
>> come
>>>>>> into master that can exhaust resources."
>>>>>>
>>>>>> Just wanted to mention that, worst case, the SAN would fail creation
>> of
>>>>>> the volume before allowing a new volume to break the system.
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> Hi John,
>>>>>>>
>>>>>>> Are you thinking we add a column on to the storage pool table,
>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
>>>>>>>
>>>>>>> That is easy enough.
>>>>>>>
>>>>>>> How do you want to determine what the SAN is capable of supporting
>> IOPS
>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
>>>> storage
>>>>>>> nodes to the cluster, the number of IOPS increases. Do we have a
>>>> thread we
>>>>>>> can use to query external devices like this SAN to update the
>> supported
>>>>>>> number of IOPS?
>>>>>>>
>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>>>>>>> overcommit ration to the plug-in when it's created? We would need to
>>>> store
>>>>>>> this in the storage_pool table, as well, I believe.
>>>>>>>
>>>>>>> We should also get Wei involved in this as his feature will need
>>>> similar
>>>>>>> functionality.
>>>>>>>
>>>>>>> Also, we should do this FAST as we have only two weeks left and many
>> of
>>>>>>> us will be out for several days for the CS Collab Conference.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> Querying the SAN only indicates the number of IOPS currently in use.
>>>>>>>> The allocator needs to check the number of IOPS committed which is
>>>> tracked
>>>>>>>> by CloudStack. For 4.2, we should be able to query the number of
>> IOPS
>>>>>>>> committed to a DataStore, and determine whether or not the number
>>>> requested
>>>>>>>> can be fulfilled by that device. It seems to be that a
>>>>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>>>>>>> DataStore's that don't support provisioned IOPS would return null.
>>>>>>>>
>>>>>>>> As I mentioned previously, I am very reluctant for any feature to
>> come
>>>>>>>> into master that can exhaust resources.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Yeah, I'm not sure I could come up with anything near an accurate
>>>>>>>>> assessment of how many IOPS are currently available on the SAN (or
>>>>>>>> even a
>>>>>>>>> total number that are available for volumes). Not sure if there's
>> yet
>>>>>>>> an
>>>>>>>>> API call for that.
>>>>>>>>>
>>>>>>>>> If I did know this number (total number of IOPS supported by the
>>>> SAN),
>>>>>>>> we'd
>>>>>>>>> still have to keep track of the total number of volumes we've
>> created
>>>>>>>> from
>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an
>> API
>>>>>>>> call
>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given volume
>> or
>>>>>>>> set of
>>>>>>>>> volumes (not supported from CS, but supported via the SolidFire
>> API),
>>>>>>>> our
>>>>>>>>> numbers in CS would be off.
>>>>>>>>>
>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a really good
>>>> idea
>>>>>>>> for
>>>>>>>>> a future release.
>>>>>>>>>
>>>>>>>>> For 4.2 I think we should stick to having the allocator detect if
>>>>>>>> storage
>>>>>>>>> QoS is desired and if the storage pool in question supports that
>>>>>>>> feature.
>>>>>>>>>
>>>>>>>>> If you really are over provisioned on your SAN in terms of IOPS or
>>>>>>>>> capacity, the SAN can let the admin know in several different ways
>>>>>>>> (e-mail,
>>>>>>>>> SNMP, GUI).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> Please see my comments in-line below.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
>> jburwell@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we
>> may
>>>>>>>> want
>>>>>>>>>> to
>>>>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
>>>>>>>> user
>>>>>>>>>> is
>>>>>>>>>>>> expecting that the hypervisor will provide a particular QoS, and
>>>>>>>> that is
>>>>>>>>>>>> not possible, it seems like we should inform them rather than
>>>>>>>> silently
>>>>>>>>>>>> ignoring their request.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Sure, that sounds reasonable.
>>>>>>>>>>>
>>>>>>>>>>> We'd have to come up with some way for the allocators to know
>> about
>>>>>>>> the
>>>>>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>>>>>
>>>>>>>>>>> Any thoughts on how we might do that?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> To collect my thoughts from previous parts of the thread, I am
>>>>>>>>>>>> uncomfortable with the idea that the management server can
>>>>>>>> overcommit a
>>>>>>>>>>>> resource. You had mentioned querying the device for available
>>>> IOPS.
>>>>>>>>>> While
>>>>>>>>>>>> that would be nice, it seems like we could fall back to a max
>> IOPS
>>>>>>>> and
>>>>>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>>>>>> administrator/operator. I think such threshold and allocation
>>>> rails
>>>>>>>>>> should
>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
>>>>>>>> basic
>>>>>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Are you thinking this ability would make it into 4.2? Just
>> curious
>>>>>>>> what
>>>>>>>>>>> release we're talking about here. For the SolidFire SAN, you
>> might
>>>>>>>> have,
>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
>>>>>>>> later
>>>>>>>>>> add
>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a
>>>> way
>>>>>>>> to
>>>>>>>>>>> know that the number of supported IOPS has increased.
>>>>>>>>>>
>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>>>>>>> example,
>>>>>>>>>> we may only support expanding capacity in 4.2, and not handle any
>>>>>>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
>>>>>>>> a
>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a similar
>> set
>>>>>>>> of
>>>>>>>>>> rails defined per host.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in
>> a
>>>>>>>>>>>> implementation specific manner. I think we need to arrive at a
>>>>>>>> general
>>>>>>>>>>>> model that can be exploited by the allocators and planners. We
>>>>>>>> should
>>>>>>>>>>>> restrict implementation specific key-value pairs (call them
>>>> details,
>>>>>>>>>>>> extended data, whatever) to information that is unique to the
>>>>>>>> driver and
>>>>>>>>>>>> would provide no useful information to the management server's
>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to fit
>> those
>>>>>>>>>>>> criteria.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
>>>>>>>> Collab
>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>>>>>
>>>>>>>>>> Yes, it would -- I will put something in the wiki topic. It will
>>>>>>>> also be
>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which include
>>>>>>>> some of
>>>>>>>>>> my opinions on the topic.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Accept the values as they are currently required (four new
>> fields
>>>>>>>> for
>>>>>>>>>>>> Wei's
>>>>>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>>>>>
>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) No QoS
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3) Storage Qos
>>>>>>>>>>>>>
>>>>>>>>>>>>> The admin needs to specify storage tags that only map to
>> storage
>>>>>>>> that
>>>>>>>>>>>>> supports Storage QoS.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Post 4.3:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Come up with a way to more generally enter these parameters
>>>>>>>> (probably
>>>>>>>>>>>> just
>>>>>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Have the drivers expose their feature set so the allocators can
>>>>>>>>>> consider
>>>>>>>>>>>>> them more fully and throw an exception if there is not a
>>>> sufficient
>>>>>>>>>>>> match.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put
>>>> some
>>>>>>>>>>>> burden
>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
>>>>>>>> sense.
>>>>>>>>>> For
>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported Min and
>>>> Max
>>>>>>>>>>>> values,
>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
>> SolidFire
>>>>>>>>>> primary
>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
>>>> then
>>>>>>>> he's
>>>>>>>>>>>> not
>>>>>>>>>>>>>> going to get the Min and Max feature. We could add help text
>> to
>>>>>>>> the
>>>>>>>>>>>> pop-up
>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and Max text
>>>>>>>> fields.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
>>>> comprehensive
>>>>>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Maybe in the future we could have the drivers advertise their
>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>> and if the allocator feels a request is not being satisfied
>> (say
>>>>>>>> Min
>>>>>>>>>> was
>>>>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
>>>>>>>> throw
>>>>>>>>>> an
>>>>>>>>>>>>>> exception.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>>>>>> jburwell@basho.com
>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two features
>>>> are
>>>>>>>>>>>>>>>> mutually
>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration
>> that
>>>>>>>>>>>>>>>> contains both
>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>>>>>> non-deterministic
>>>>>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
>>>> storage
>>>>>>>>>>>> oriented
>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand storage
>>>>>>>> tags,
>>>>>>>>>> we
>>>>>>>>>>>>>>>> have no
>>>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
>> wasn't
>>>>>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>>>>>> intending that we implement this allocation model in 4.3,
>>>> but
>>>>>>>>>>>> instead,
>>>>>>>>>>>>>>>>>> determine if this type model would be one we would want to
>>>>>>>> support
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the
>> data
>>>>>>>> model
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> current implementation would not preclude evolution in
>> that
>>>>>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what user's
>>>>>>>> expect
>>>>>>>>>> of
>>>>>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource set to
>>>>>>>> fulfill a
>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in that what
>>>> we've
>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>> should not make refinement difficult in the future. If we
>>>> don't
>>>>>>>>>>>> modify
>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
>> relatively
>>>>>>>> simple
>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
>>>> aggregated
>>>>>>>>>> result
>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>>>>>>>> throttled
>>>>>>>>>>>> I/O and
>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
>> concern
>>>>>>>> is
>>>>>>>>>> not
>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>>>>>>>> Javadoc
>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and their
>>>> mutual
>>>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>>>>>>> exclusion.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>>>>>>>> provides
>>>>>>>>>> both
>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned
>> IOPS
>>>>>>>> uses
>>>>>>>>>> a
>>>>>>>>>>>>>>>> range. In order to select the best available resource to
>>>>>>>> fulfill a
>>>>>>>>>>>> QoS, we
>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in terms of
>>>>>>>> units
>>>>>>>>>>>> (IOPS or
>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>>>>>>>>>> achieve a
>>>>>>>>>>>>>>>> model like I described, I think we would need to resolve
>> this
>>>>>>>> issue
>>>>>>>>>>>> in 4.2
>>>>>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the input
>> for
>>>>>>>> 4.2.
>>>>>>>>>>>> Plus
>>>>>>>>>>>>>>> people probably want to input the data in terms they're
>>>> familiar
>>>>>>>> with
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> the products in question.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let
>> the
>>>>>>>> user
>>>>>>>>>>>> send
>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected due to
>> a
>>>>>>>> given
>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it because
>>>> what
>>>>>>>>>>>> happens if
>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage after
>>>>>>>> having
>>>>>>>>>>>> created
>>>>>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>>>>>>>> re-think
>>>>>>>>>> it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>>>>>>>> supported
>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As I think through the implications of these requirements
>>>> and
>>>>>>>>>>>> reflect
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>>>>>>>> impacted
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>>>>>> provisioned
>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> has no
>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
>>>>>>>> that
>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>>>>>>> understand
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>>>>>>>> potentially
>>>>>>>>>>>>>>>> causing
>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>>>>>>>> to me
>>>>>>>>>>>>>>>> that a
>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>>>>>>> maximum
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment factor.
>> This
>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>> should be used by the storage allocators to determine the
>>>>>>>> device
>>>>>>>>>>>> best
>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems that
>> a
>>>>>>>>>> similar
>>>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
>>>> layer
>>>>>>>>>> which
>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path
>> that
>>>>>>>> was
>>>>>>>>>>>>>>>> previously
>>>>>>>>>>>>>>>>> established for other properties you see on dialogs in CS:
>>>> Just
>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed
>> to
>>>>>>>>>> supports
>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>>>>>>> storage
>>>>>>>>>> tag
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> confines the deployment only to storage that supports the
>>>>>>>> selected
>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>>>>>> CloudStack
>>>>>>>>>>>> works
>>>>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for the
>>>>>>>>>> administrator
>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is that we
>>>>>>>> over
>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>>>>>>>>>>> illustration,
>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
>>>> device
>>>>>>>> is
>>>>>>>>>>>> 100000.
>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>>>>>> Therefore,
>>>>>>>>>> we
>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>>>>>>>> guaranteed
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
>>>>>>>> As we
>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>>>>>>> guarantee
>>>>>>>>>>>> more IOPS
>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that
>> make
>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not
>>>> even
>>>>>>>>>> sure
>>>>>>>>>>>> I
>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain.
>> I'd
>>>>>>>> have
>>>>>>>>>> to
>>>>>>>>>>>> get
>>>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN and
>>>> total
>>>>>>>>>> them
>>>>>>>>>>>> up,
>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
>>>> supported
>>>>>>>>>> IOPS
>>>>>>>>>>>> of
>>>>>>>>>>>>>>> the system.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>>>>>>> resources
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>>>>>> think a side effect of the current implementation is
>>>> SolidFire
>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O,
>> it
>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
>> side-by-side
>>>>>>>> with
>>>>>>>>>> QoS
>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>>>>>>>> starve
>>>>>>>>>>>> out
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>>>>>>>>>> collocated
>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS)
>> and
>>>>>>>> not
>>>>>>>>>>>>>>>> specify
>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at
>> the
>>>>>>>>>>>> defaults
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN,
>>>> you
>>>>>>>> will
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>>>>>>> exclusions
>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
>> deploy
>>>> my
>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
>>>>>>>> would
>>>>>>>>>>>>>>>> complicate component assembly. It would require a storage
>>>>>>>> plugin to
>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies and
>>>>>>>> determine
>>>>>>>>>> the
>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion
>>>> or
>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its functionality
>>>> in
>>>>>>>> a
>>>>>>>>>> way
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> allocator understands.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it
>> only
>>>>>>>>>> limits
>>>>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>>>>>>>> with
>>>>>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does
>> not
>>>>>>>>>> exist.
>>>>>>>>>>>>>>>> However, I am concerned with the more general case as we
>>>>>>>> supported
>>>>>>>>>>>> more
>>>>>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>>>>>>>> values
>>>>>>>>>> to
>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
>>>> provisioned
>>>>>>>>>>>> IOPS, but
>>>>>>>>>>>>>>> what if you break them out by read and write or do something
>>>>>>>> else? We
>>>>>>>>>>>> need
>>>>>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>>>>>>>> I/O
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
>> volume.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We also need to understand the implications of these QoS
>>>>>>>>>> guarantees
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
>>>> hardware
>>>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>>>>>>>> forced
>>>>>>>>>> to
>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>>>>>> implementation
>>>>>>>>>> in
>>>>>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some great
>>>>>>>> ideas
>>>>>>>>>> for
>>>>>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>>>>>>>> storage
>>>>>>>>>>>>>>>> tags).
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>>>>>>>> addition to
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
>>>> being
>>>>>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in values
>> for
>>>>>>>> Wei's
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>>>>>>>> yet), so
>>>>>>>>>> he
>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators
>>>> to
>>>>>>>>>> prefer
>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
>> today
>>>>>>>> in
>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other features
>>>>>>>> (fields
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
>> Perhaps
>>>>>>>> post
>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>>>>>>>> OpenStack
>>>>>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
>>>> until
>>>>>>>>>> code
>>>>>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep
>> Wei's
>>>>>>>> fields
>>>>>>>>>>>>>>>> and my
>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
>>>>>>>> data
>>>>>>>>>>>>>>>> entered
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
>>>>>>>> design
>>>>>>>>>>>>>>>> that can
>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>>>>>>>> situations.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I just published my review @
>>>>>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
>>>>>>>> with a
>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>>>>>> checked
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
>> file?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I
>> am
>>>>>>>>>>>> offline
>>>>>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>>>>>> this :)
>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>>>>>> areas
>>>>>>>>>> we
>>>>>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>>>>>>> afternoon,
>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
>>>> in
>>>>>>>> Java
>>>>>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>>>>>> branch
>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>>>>> object_store
>>>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
>>>> the
>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
>>>> into
>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>>>> this
>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
>> number
>>>>>>>> of
>>>>>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between
>> the
>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects
>> of
>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>>>>>>>> dependency
>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
>> graph,
>>>>>>>> we
>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>>>>>> into
>>>>>>>>>> his
>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>>>>>> after his
>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
>> here:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>>>>>> code in
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
>> storage
>>>>>>>>>> plug-in,
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>>>>>>>> (although
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>>>>>> actually
>>>>>>>>>> in
>>>>>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed
>> in
>>>>>>>> the
>>>>>>>>>> url
>>>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>>>> stored
>>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>>>>>> hypervisor
>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>>>>>> place
>>>>>>>>>> of
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
>> hypervisor
>>>>>>>>>> checks
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>>>>>> executes to
>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
>>>> to
>>>>>>>>>>>> detach a
>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>>>> data
>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>>>> IOPS
>>>>>>>> in
>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
>>>> If
>>>>>>>> we
>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>>>>>> avoiding
>>>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
>> details
>>>>>>>>>> field.
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>>>>>> regarding
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>> details
>>>>>>>> are
>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>>>>>> listStoragePool
>>>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>>>>>> useful
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>>>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst
>> IOPS
>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
>> have
>>>>>>>> to be
>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>>>>> disk_offering_details
>>>>>>>>>>>> table.
>>>>>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
>> Burst
>>>>>>>> IOPS
>>>>>>>>>> in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>>>> the
>>>>>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Now that I think about it, we already have capacity_bytes and
available_bytes in the storage_pool table. We should just add two new
IOPS-related fields to the storage_pool table: capacity_iops and
available_iops.
Do you think we actually need an overcommit ratio? If the user was, for
example, going to input 1.5 for the overcommit ratio and 50,000 for total
IOPS (capacity_iops), he could just pass in 75,000 (50,000 * 1.5) for total
IOPS.
On Fri, Jun 14, 2013 at 2:31 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> I am OK with that approach, John.
>
> So, let me review to make sure I follow you correctly:
>
> We introduce two new parameters to the plug-in: Number of Total IOPS for
> the SAN and an overcommit ratio. (On a side note, if we are just
> multiplying the two numbers, why don't we have the user just input their
> product?)
>
> Are both of these new parameters to the create storage pool API command or
> are they passed into the create storage pool API command through its url
> parameter?
>
> If they are new parameters, we should make two new columns in the
> storage_pool table.
>
> If they are passed in via the url parameter, they should go in the
> storage_pool_details table.
>
> For 4.2, if someone wants to change these values, they must update the DB
> manually.
>
> Every time a volume is created or deleted for the SolidFire plug-in, the
> Current IOPS value (sum of all volumes' Min IOPS that are associated with
> the plug-in) is updated.
>
> The allocator can use these fields to determine if it can fit in a new
> volume.
>
> Does it look like my understanding is OK?
>
>
> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I apologize for not being clear -- this conversation has been admittedly
>> disjoint. I think we should allow the maximum IOPS and overcommitment
>> values to be updated though I would recommend restricting updates to to be
>> an increasing value for 4.2 (e.g. users can increase the number of total
>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to 200,000).
>> While not ideal, given the amount of time we have left for 4.2, it will
>> cover most cases, and we can address the implications of reducing resource
>> capacity in 4.3. This approach addresses both of your concerns. First, it
>> allows the administrator/operator to determine what portion of the device
>> they wish to dedicate. For example, if the device has a total capacity of
>> 200,000 IOPS, and they only want CS to use 25% of the device then they set
>> the maximum total IOPS to 50,000. Second, as they grow capacity, they can
>> update the DataStore to increase the number of IOPS they want to dedicate
>> to CS' use. I would imagine expansion of capacity happens infrequently
>> enough that increasing the maximum IOPS value would not be a significant
>> burden.
>>
>> Thanks,
>> -John
>>
>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > "the administrator/operator simply needs to tell us the total number of
>> > IOPS that can be committed to it and an overcommitment factor."
>> >
>> > Are you thinking when we create a plug-in as primary storage that we
>> say -
>> > up front - how many IOPS the SAN can handle?
>> >
>> > That is not a good move, in my opinion. Our SAN is designed to start
>> small
>> > and grow to PBs. As the need arises for more storage, the admin
>> purchases
>> > additional storage nodes that join the cluster and the performance and
>> > capacity go up.
>> >
>> > We need to know how many IOPS total the SAN can handle and what it is
>> > committed to currently (the sum of the number of volumes' min IOPS).
>> >
>> > We also cannot assume the SAN is dedicated to CS.
>> >
>> >
>> > On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Simon,
>> >>
>> >> Yes, it is CloudStack's job to protect, as best it can, from
>> >> oversubscribing resources. I would argue that resource management is
>> one,
>> >> if not the most, important functions of the system. It is no different
>> >> than the allocation/planning performed for hosts relative to cores and
>> >> memory. We can still oversubscribe resources, but we have rails +
>> knobs
>> >> and dials to avoid it. Without these controls in place, we could
>> easily
>> >> allow users to deploy workloads that overrun resources harming all
>> tenants.
>> >>
>> >> I also think that we are over thinking this issue for provisioned IOPS.
>> >> When the DataStore is configured, the administrator/operator simply
>> needs
>> >> to tell us the total number of IOPS that can be committed to it and an
>> >> overcommitment factor. As we allocate volumes to that DataStore, we
>> sum up
>> >> the committed IOPS of the existing Volumes attached to the DataStore,
>> apply
>> >> the overcommitment factor, and determine whether or not the requested
>> >> minimum IOPS for the new volume can be fulfilled. We can provide both
>> >> general and vendor specific documentation for determining these values
>> --
>> >> be they to consume the entire device or a portion of it.
>> >>
>> >> Querying the device is unnecessary and deceptive. CloudStack resource
>> >> management is not interested in the current state of the device which
>> could
>> >> be anywhere from extremely heavy to extremely light at any given time.
>> We
>> >> are interested in the worst case load that is anticipated for
>> resource. In
>> >> my view, it is up to administrators/operators to instrument their
>> >> environment to understand usage patterns and capacity. We should
>> provide
>> >> information that will help determine what should be
>> instrumented/monitored,
>> >> but that function should be performed outside of CloudStack.
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>> >>
>> >>> I'd like to comment on this briefly.
>> >>>
>> >>>
>> >>>
>> >>> I think an assumption is being made that the SAN is being dedicated
>> to a
>> >> CS instance.
>> >>>
>> >>> My person opinion that this whole IOPS calculation is getting rather
>> >> complicated, and could probably be much simpler than this. Over
>> >> subscription is a fact of life on virtually all storage, and is really
>> no
>> >> different in concept than multiple virt instances on a single piece of
>> >> hardware. All decent SANs offer many management options for the storage
>> >> engineers to keep track of IOPS utilization, and plan for spindle
>> >> augmentation as required.
>> >>> Is it really the job of CS to become yet another management layer on
>> top
>> >> of this?
>> >>>
>> >>> ----- Original Message -----
>> >>>
>> >>> From: "Mike Tutkowski" <mi...@solidfire.com>
>> >>> To: dev@cloudstack.apache.org
>> >>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
>> >> ustcweizhou@gmail.com>
>> >>> Sent: Friday, June 14, 2013 1:00:26 PM
>> >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >>>
>> >>> 1) We want number of IOPS currently supported by the SAN.
>> >>>
>> >>> 2) We want the number of IOPS that are committed (sum of min IOPS for
>> >> each
>> >>> volume).
>> >>>
>> >>> We could do the following to keep track of IOPS:
>> >>>
>> >>> The plug-in could have a timer thread that goes off every, say, 1
>> minute.
>> >>>
>> >>> It could query the SAN for the number of nodes that make up the SAN
>> and
>> >>> multiple this by 50,000. This is essentially the number of supported
>> IOPS
>> >>> of the SAN.
>> >>>
>> >>> The next API call could be to get all of the volumes on the SAN.
>> Iterate
>> >>> through them all and add up their min IOPS values. This is the number
>> of
>> >>> IOPS the SAN is committed to.
>> >>>
>> >>> These two numbers can then be updated in the storage_pool table (a
>> column
>> >>> for each value).
>> >>>
>> >>> The allocators can get these values as needed (and they would be as
>> >>> accurate as the last time the thread asked the SAN for this info).
>> >>>
>> >>> These two fields, the min IOPS of the volume to create, and the
>> >> overcommit
>> >>> ratio of the plug-in would tell the allocator if it can select the
>> given
>> >>> storage pool.
>> >>>
>> >>> What do you think?
>> >>>
>> >>>
>> >>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>> >>> mike.tutkowski@solidfire.com> wrote:
>> >>>
>> >>>> "As I mentioned previously, I am very reluctant for any feature to
>> come
>> >>>> into master that can exhaust resources."
>> >>>>
>> >>>> Just wanted to mention that, worst case, the SAN would fail creation
>> of
>> >>>> the volume before allowing a new volume to break the system.
>> >>>>
>> >>>>
>> >>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>
>> >>>>> Hi John,
>> >>>>>
>> >>>>> Are you thinking we add a column on to the storage pool table,
>> >>>>> IOPS_Count, where we add and subtract committed IOPS?
>> >>>>>
>> >>>>> That is easy enough.
>> >>>>>
>> >>>>> How do you want to determine what the SAN is capable of supporting
>> IOPS
>> >>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
>> >> storage
>> >>>>> nodes to the cluster, the number of IOPS increases. Do we have a
>> >> thread we
>> >>>>> can use to query external devices like this SAN to update the
>> supported
>> >>>>> number of IOPS?
>> >>>>>
>> >>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>> >>>>> overcommit ration to the plug-in when it's created? We would need to
>> >> store
>> >>>>> this in the storage_pool table, as well, I believe.
>> >>>>>
>> >>>>> We should also get Wei involved in this as his feature will need
>> >> similar
>> >>>>> functionality.
>> >>>>>
>> >>>>> Also, we should do this FAST as we have only two weeks left and
>> many of
>> >>>>> us will be out for several days for the CS Collab Conference.
>> >>>>>
>> >>>>> Thanks
>> >>>>>
>> >>>>>
>> >>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
>> >>> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> Querying the SAN only indicates the number of IOPS currently in
>> use.
>> >>>>>> The allocator needs to check the number of IOPS committed which is
>> >> tracked
>> >>>>>> by CloudStack. For 4.2, we should be able to query the number of
>> IOPS
>> >>>>>> committed to a DataStore, and determine whether or not the number
>> >> requested
>> >>>>>> can be fulfilled by that device. It seems to be that a
>> >>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>> >>>>>> DataStore's that don't support provisioned IOPS would return null.
>> >>>>>>
>> >>>>>> As I mentioned previously, I am very reluctant for any feature to
>> come
>> >>>>>> into master that can exhaust resources.
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>
>> >>>>>>> Yeah, I'm not sure I could come up with anything near an accurate
>> >>>>>>> assessment of how many IOPS are currently available on the SAN (or
>> >>>>>> even a
>> >>>>>>> total number that are available for volumes). Not sure if there's
>> yet
>> >>>>>> an
>> >>>>>>> API call for that.
>> >>>>>>>
>> >>>>>>> If I did know this number (total number of IOPS supported by the
>> >> SAN),
>> >>>>>> we'd
>> >>>>>>> still have to keep track of the total number of volumes we've
>> created
>> >>>>>> from
>> >>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an
>> API
>> >>>>>> call
>> >>>>>>> directly to the SAN to tweak the number of IOPS on a given volume
>> or
>> >>>>>> set of
>> >>>>>>> volumes (not supported from CS, but supported via the SolidFire
>> API),
>> >>>>>> our
>> >>>>>>> numbers in CS would be off.
>> >>>>>>>
>> >>>>>>> I'm thinking verifying sufficient number of IOPS is a really good
>> >> idea
>> >>>>>> for
>> >>>>>>> a future release.
>> >>>>>>>
>> >>>>>>> For 4.2 I think we should stick to having the allocator detect if
>> >>>>>> storage
>> >>>>>>> QoS is desired and if the storage pool in question supports that
>> >>>>>> feature.
>> >>>>>>>
>> >>>>>>> If you really are over provisioned on your SAN in terms of IOPS or
>> >>>>>>> capacity, the SAN can let the admin know in several different ways
>> >>>>>> (e-mail,
>> >>>>>>> SNMP, GUI).
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jburwell@basho.com
>> >
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> Please see my comments in-line below.
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Comments below in red.
>> >>>>>>>>>
>> >>>>>>>>> Thanks
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
>> jburwell@basho.com>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Mike,
>> >>>>>>>>>>
>> >>>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we
>> may
>> >>>>>> want
>> >>>>>>>> to
>> >>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If
>> the
>> >>>>>> user
>> >>>>>>>> is
>> >>>>>>>>>> expecting that the hypervisor will provide a particular QoS,
>> and
>> >>>>>> that is
>> >>>>>>>>>> not possible, it seems like we should inform them rather than
>> >>>>>> silently
>> >>>>>>>>>> ignoring their request.
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Sure, that sounds reasonable.
>> >>>>>>>>>
>> >>>>>>>>> We'd have to come up with some way for the allocators to know
>> about
>> >>>>>> the
>> >>>>>>>>> requested storage QoS and then query the candidate drivers.
>> >>>>>>>>>
>> >>>>>>>>> Any thoughts on how we might do that?
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> To collect my thoughts from previous parts of the thread, I am
>> >>>>>>>>>> uncomfortable with the idea that the management server can
>> >>>>>> overcommit a
>> >>>>>>>>>> resource. You had mentioned querying the device for available
>> >> IOPS.
>> >>>>>>>> While
>> >>>>>>>>>> that would be nice, it seems like we could fall back to a max
>> IOPS
>> >>>>>> and
>> >>>>>>>>>> overcommit factor manually calculated and entered by the
>> >>>>>>>>>> administrator/operator. I think such threshold and allocation
>> >> rails
>> >>>>>>>> should
>> >>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
>> >>>>>> basic
>> >>>>>>>>>> feature of any cloud orchestration platform.
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Are you thinking this ability would make it into 4.2? Just
>> curious
>> >>>>>> what
>> >>>>>>>>> release we're talking about here. For the SolidFire SAN, you
>> might
>> >>>>>> have,
>> >>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and
>> then
>> >>>>>> later
>> >>>>>>>> add
>> >>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a
>> >> way
>> >>>>>> to
>> >>>>>>>>> know that the number of supported IOPS has increased.
>> >>>>>>>>
>> >>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>> >>>>>> example,
>> >>>>>>>> we may only support expanding capacity in 4.2, and not handle any
>> >>>>>>>> re-balance scenarios -- node failure, addition, etc.
>> Extrapolating
>> >>>>>> a
>> >>>>>>>> bit, the throttled I/O enhancement seems like it needs a similar
>> set
>> >>>>>> of
>> >>>>>>>> rails defined per host.
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed
>> in a
>> >>>>>>>>>> implementation specific manner. I think we need to arrive at a
>> >>>>>> general
>> >>>>>>>>>> model that can be exploited by the allocators and planners. We
>> >>>>>> should
>> >>>>>>>>>> restrict implementation specific key-value pairs (call them
>> >> details,
>> >>>>>>>>>> extended data, whatever) to information that is unique to the
>> >>>>>> driver and
>> >>>>>>>>>> would provide no useful information to the management server's
>> >>>>>>>>>> orchestration functions. A resource QoS does not seem to fit
>> those
>> >>>>>>>>>> criteria.
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> I wonder if this would be a good discussion topic for Sunday's
>> CS
>> >>>>>> Collab
>> >>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>> >>>>>>>>
>> >>>>>>>> Yes, it would -- I will put something in the wiki topic. It will
>> >>>>>> also be
>> >>>>>>>> part of my talk on Monday -- How to Run from Zombie which include
>> >>>>>> some of
>> >>>>>>>> my opinions on the topic.
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> -John
>> >>>>>>>>>>
>> >>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> So, here's my suggestion for 4.2:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Accept the values as they are currently required (four new
>> fields
>> >>>>>> for
>> >>>>>>>>>> Wei's
>> >>>>>>>>>>> feature or two new fields for mine).
>> >>>>>>>>>>>
>> >>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>> >>>>>>>>>>>
>> >>>>>>>>>>> 1) No QoS
>> >>>>>>>>>>>
>> >>>>>>>>>>> 2) Hypervisor QoS
>> >>>>>>>>>>>
>> >>>>>>>>>>> 3) Storage Qos
>> >>>>>>>>>>>
>> >>>>>>>>>>> The admin needs to specify storage tags that only map to
>> storage
>> >>>>>> that
>> >>>>>>>>>>> supports Storage QoS.
>> >>>>>>>>>>>
>> >>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>> >>>>>>>>>> hypervisors
>> >>>>>>>>>>> in use support the new fields, they may not be enforced.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Post 4.3:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Come up with a way to more generally enter these parameters
>> >>>>>> (probably
>> >>>>>>>>>> just
>> >>>>>>>>>>> key/value pairs sent to the drivers).
>> >>>>>>>>>>>
>> >>>>>>>>>>> Have the drivers expose their feature set so the allocators
>> can
>> >>>>>>>> consider
>> >>>>>>>>>>> them more fully and throw an exception if there is not a
>> >> sufficient
>> >>>>>>>>>> match.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put
>> >> some
>> >>>>>>>>>> burden
>> >>>>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
>> >>>>>> sense.
>> >>>>>>>> For
>> >>>>>>>>>>>> example, if he wants to use storage QoS with supported Min
>> and
>> >> Max
>> >>>>>>>>>> values,
>> >>>>>>>>>>>> he'll have to put in a storage tag that references the
>> SolidFire
>> >>>>>>>> primary
>> >>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
>> >> then
>> >>>>>> he's
>> >>>>>>>>>> not
>> >>>>>>>>>>>> going to get the Min and Max feature. We could add help text
>> to
>> >>>>>> the
>> >>>>>>>>>> pop-up
>> >>>>>>>>>>>> dialog that's displayed when you click in the Min and Max
>> text
>> >>>>>> fields.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Same idea for Wei's feature.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
>> >> comprehensive
>> >>>>>>>>>>>> approach for post 4.2.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Maybe in the future we could have the drivers advertise their
>> >>>>>>>>>> capabilities
>> >>>>>>>>>>>> and if the allocator feels a request is not being satisfied
>> (say
>> >>>>>> Min
>> >>>>>>>> was
>> >>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it
>> can
>> >>>>>> throw
>> >>>>>>>> an
>> >>>>>>>>>>>> exception.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Comments below in red.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>> >>>>>> jburwell@basho.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Please see my comment in-line below.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I've put comments below in red.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>> >>>>>> jburwell@basho.com
>> >>>>>>>>>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
>> features
>> >> are
>> >>>>>>>>>>>>>> mutually
>> >>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration
>> that
>> >>>>>>>>>>>>>> contains both
>> >>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>> >>>>>>>>>> non-deterministic
>> >>>>>>>>>>>>>>>> operations.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Agreed
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
>> >> storage
>> >>>>>>>>>> oriented
>> >>>>>>>>>>>>>>>> solves the problem in short term. As I understand storage
>> >>>>>> tags,
>> >>>>>>>> we
>> >>>>>>>>>>>>>> have no
>> >>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
>> wasn't
>> >>>>>>>>>>>>>> necessarily
>> >>>>>>>>>>>>>>>> intending that we implement this allocation model in 4.3,
>> >> but
>> >>>>>>>>>> instead,
>> >>>>>>>>>>>>>>>> determine if this type model would be one we would want
>> to
>> >>>>>> support
>> >>>>>>>>>> in
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the
>> data
>> >>>>>> model
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>>> current implementation would not preclude evolution in
>> that
>> >>>>>>>>>>>>>> direction. My
>> >>>>>>>>>>>>>>>> view is that this type of allocation model is what user's
>> >>>>>> expect
>> >>>>>>>> of
>> >>>>>>>>>>>>>> "cloud"
>> >>>>>>>>>>>>>>>> systems -- selecting the best available resource set to
>> >>>>>> fulfill a
>> >>>>>>>>>>>>>> set of
>> >>>>>>>>>>>>>>>> system requirements.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I believe we have meet your requirement here in that what
>> >> we've
>> >>>>>>>>>>>>>> implemented
>> >>>>>>>>>>>>>>> should not make refinement difficult in the future. If we
>> >> don't
>> >>>>>>>>>> modify
>> >>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
>> relatively
>> >>>>>> simple
>> >>>>>>>>>>>>>> changes
>> >>>>>>>>>>>>>>> to enhance the current functioning of the system.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Looking through both patches, I have to say that the
>> >> aggregated
>> >>>>>>>> result
>> >>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>> >>>>>> throttled
>> >>>>>>>>>> I/O and
>> >>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
>> concern
>> >>>>>> is
>> >>>>>>>> not
>> >>>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>> >>>>>> Javadoc
>> >>>>>>>>>> should
>> >>>>>>>>>>>>>> be added explaining the two sets of attributes and their
>> >> mutual
>> >>>>>>>>>> exclusion.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>> >>>>>> exclusion.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>> >>>>>> provides
>> >>>>>>>> both
>> >>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned
>> IOPS
>> >>>>>> uses
>> >>>>>>>> a
>> >>>>>>>>>>>>>> range. In order to select the best available resource to
>> >>>>>> fulfill a
>> >>>>>>>>>> QoS, we
>> >>>>>>>>>>>>>> would need to have the QoS expression normalized in terms
>> of
>> >>>>>> units
>> >>>>>>>>>> (IOPS or
>> >>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>> >>>>>>>> achieve a
>> >>>>>>>>>>>>>> model like I described, I think we would need to resolve
>> this
>> >>>>>> issue
>> >>>>>>>>>> in 4.2
>> >>>>>>>>>>>>>> to ensure a viable migration path.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I think we're not likely to be able to normalize the input
>> for
>> >>>>>> 4.2.
>> >>>>>>>>>> Plus
>> >>>>>>>>>>>>> people probably want to input the data in terms they're
>> >> familiar
>> >>>>>> with
>> >>>>>>>>>> for
>> >>>>>>>>>>>>> the products in question.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let
>> the
>> >>>>>> user
>> >>>>>>>>>> send
>> >>>>>>>>>>>>> key/value pairs to each vendor that could be selected due
>> to a
>> >>>>>> given
>> >>>>>>>>>>>>> storage tag. I'm still not sure that would solve it because
>> >> what
>> >>>>>>>>>> happens if
>> >>>>>>>>>>>>> you change the storage tag of a given Primary Storage after
>> >>>>>> having
>> >>>>>>>>>> created
>> >>>>>>>>>>>>> a Disk Offering?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>> >>>>>> re-think
>> >>>>>>>> it.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>> >>>>>> supported
>> >>>>>>>>>>>>> feature sets so the allocators can make good choices.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> As I think through the implications of these requirements
>> >> and
>> >>>>>>>>>> reflect
>> >>>>>>>>>>>>>> on
>> >>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>> >>>>>> impacted
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>> >>>>>> provisioned
>> >>>>>>>> IOPS
>> >>>>>>>>>>>>>> has no
>> >>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
>> devices
>> >>>>>> that
>> >>>>>>>> have
>> >>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>> >>>>>> understand
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>> >>>>>> potentially
>> >>>>>>>>>>>>>> causing
>> >>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It
>> seems
>> >>>>>> to me
>> >>>>>>>>>>>>>> that a
>> >>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>> >>>>>> maximum
>> >>>>>>>>>> IOPS
>> >>>>>>>>>>>>>> which
>> >>>>>>>>>>>>>>>> it can support and some type of overcommitment factor.
>> This
>> >>>>>>>>>>>>>> information
>> >>>>>>>>>>>>>>>> should be used by the storage allocators to determine the
>> >>>>>> device
>> >>>>>>>>>> best
>> >>>>>>>>>>>>>> able
>> >>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems
>> that a
>> >>>>>>>> similar
>> >>>>>>>>>>>>>> set of
>> >>>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
>> >> layer
>> >>>>>>>> which
>> >>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path
>> that
>> >>>>>> was
>> >>>>>>>>>>>>>> previously
>> >>>>>>>>>>>>>>> established for other properties you see on dialogs in CS:
>> >> Just
>> >>>>>>>>>> because
>> >>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed
>> to
>> >>>>>>>> supports
>> >>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>> >>>>>> storage
>> >>>>>>>> tag
>> >>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>> confines the deployment only to storage that supports the
>> >>>>>> selected
>> >>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>> >>>>>> CloudStack
>> >>>>>>>>>> works
>> >>>>>>>>>>>>>>> today.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I understand the tag functionality, and the need for the
>> >>>>>>>> administrator
>> >>>>>>>>>>>>>> to very carefully construct offerings. My concern is that
>> we
>> >>>>>> over
>> >>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>> >>>>>>>>>> illustration,
>> >>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
>> >> device
>> >>>>>> is
>> >>>>>>>>>> 100000.
>> >>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>> >>>>>> Therefore,
>> >>>>>>>> we
>> >>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>> >>>>>> guaranteed
>> >>>>>>>>>> IOPS
>> >>>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>> >>>>>>>>>> configuration
>> >>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
>> factor.
>> >>>>>> As we
>> >>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>> >>>>>> guarantee
>> >>>>>>>>>> more IOPS
>> >>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that
>> make
>> >>>>>>>> sense?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not
>> >> even
>> >>>>>>>> sure
>> >>>>>>>>>> I
>> >>>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain.
>> I'd
>> >>>>>> have
>> >>>>>>>> to
>> >>>>>>>>>> get
>> >>>>>>>>>>>>> all of the min values for all of the volumes on the SAN and
>> >> total
>> >>>>>>>> them
>> >>>>>>>>>> up,
>> >>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
>> >> supported
>> >>>>>>>> IOPS
>> >>>>>>>>>> of
>> >>>>>>>>>>>>> the system.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>> >>>>>> resources
>> >>>>>>>>>> to
>> >>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>> >>>>>>>> provisioned
>> >>>>>>>>>>>>>> IOPS, I
>> >>>>>>>>>>>>>>>> think a side effect of the current implementation is
>> >> SolidFire
>> >>>>>>>>>> volumes
>> >>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled
>> I/O, it
>> >>>>>>>> seems
>> >>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
>> side-by-side
>> >>>>>> with
>> >>>>>>>> QoS
>> >>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
>> potentially
>> >>>>>> starve
>> >>>>>>>>>> out
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>> >>>>>>>> collocated
>> >>>>>>>>>>>>>> VMs
>> >>>>>>>>>>>>>>>> from being fulfilled.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS)
>> and
>> >>>>>> not
>> >>>>>>>>>>>>>> specify
>> >>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at
>> the
>> >>>>>>>>>> defaults
>> >>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN,
>> >> you
>> >>>>>> will
>> >>>>>>>>>>>>>> have
>> >>>>>>>>>>>>>>> your Mins met.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>> >>>>>> exclusions
>> >>>>>>>>>>>>>> list,
>> >>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
>> deploy
>> >> my
>> >>>>>>>>>> volume
>> >>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>> storage with these following tags).
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as
>> it
>> >>>>>> would
>> >>>>>>>>>>>>>> complicate component assembly. It would require a storage
>> >>>>>> plugin to
>> >>>>>>>>>>>>>> anticipate all of the possible component assemblies and
>> >>>>>> determine
>> >>>>>>>> the
>> >>>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>> >>>>>>>>>> capabilities
>> >>>>>>>>>>>>>> which can be matched to a set of requested requirements.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
>> inclusion
>> >> or
>> >>>>>>>>>>>>> exclusion lists. It just needs to advertise its
>> functionality
>> >> in
>> >>>>>> a
>> >>>>>>>> way
>> >>>>>>>>>> the
>> >>>>>>>>>>>>> allocator understands.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it
>> only
>> >>>>>>>> limits
>> >>>>>>>>>>>>>> IOPS,
>> >>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a
>> system
>> >>>>>> with
>> >>>>>>>>>>>>>> guaranteed
>> >>>>>>>>>>>>>>> minimum IOPS does this.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does
>> not
>> >>>>>>>> exist.
>> >>>>>>>>>>>>>> However, I am concerned with the more general case as we
>> >>>>>> supported
>> >>>>>>>>>> more
>> >>>>>>>>>>>>>> devices with provisioned IOPS.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
>> vendor-specific
>> >>>>>> values
>> >>>>>>>> to
>> >>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
>> >> provisioned
>> >>>>>>>>>> IOPS, but
>> >>>>>>>>>>>>> what if you break them out by read and write or do something
>> >>>>>> else? We
>> >>>>>>>>>> need
>> >>>>>>>>>>>>> a more general mechanism.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
>> throttled
>> >>>>>> I/O
>> >>>>>>>> and
>> >>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
>> volume.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Agreed
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> We also need to understand the implications of these QoS
>> >>>>>>>> guarantees
>> >>>>>>>>>> on
>> >>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
>> >> hardware
>> >>>>>>>>>>>>>> resources
>> >>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>> >>>>>> forced
>> >>>>>>>> to
>> >>>>>>>>>>>>>> make
>> >>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>> >>>>>> implementation
>> >>>>>>>> in
>> >>>>>>>>>>>>>> 4.3.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I agree, John. I also think you've come up with some great
>> >>>>>> ideas
>> >>>>>>>> for
>> >>>>>>>>>>>>>> 4.3. :)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning
>> (with
>> >>>>>> storage
>> >>>>>>>>>>>>>> tags).
>> >>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>> >>>>>> addition to
>> >>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
>> >> being
>> >>>>>>>>>>>>>> preferred,
>> >>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>> >>>>>>>> hypervisor
>> >>>>>>>>>>>>>> QoS
>> >>>>>>>>>>>>>>>> is).
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I think John's concern is that you can enter in values
>> for
>> >>>>>> Wei's
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>> my
>> >>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>> >>>>>> yet), so
>> >>>>>>>> he
>> >>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
>> allocators
>> >> to
>> >>>>>>>> prefer
>> >>>>>>>>>>>>>> these
>> >>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
>> today
>> >>>>>> in
>> >>>>>>>>>>>>>>>> CloudStack,
>> >>>>>>>>>>>>>>>>> we already have this kind of an issue with other
>> features
>> >>>>>> (fields
>> >>>>>>>>>> in
>> >>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
>> Perhaps
>> >>>>>> post
>> >>>>>>>>>> 4.2
>> >>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>> >>>>>> OpenStack
>> >>>>>>>>>>>>>> addresses
>> >>>>>>>>>>>>>>>>> the issue).
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
>> >> until
>> >>>>>>>> code
>> >>>>>>>>>>>>>>>> freeze)
>> >>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep
>> Wei's
>> >>>>>> fields
>> >>>>>>>>>>>>>> and my
>> >>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature
>> has
>> >>>>>> data
>> >>>>>>>>>>>>>> entered
>> >>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Then let's step back and look into a more
>> general-purpose
>> >>>>>> design
>> >>>>>>>>>>>>>> that can
>> >>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>> >>>>>> situations.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> What do you think?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>> >>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I just published my review @
>> >>>>>>>> https://reviews.apache.org/r/11479/.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I apologize for the delay,
>> >>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> No problem, John.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming
>> up
>> >>>>>> with a
>> >>>>>>>>>> new
>> >>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>> file.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to
>> be
>> >>>>>> checked
>> >>>>>>>>>> in
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
>> file?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> >>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review.
>> I am
>> >>>>>>>>>> offline
>> >>>>>>>>>>>>>> ATM,
>> >>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>> >>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> >>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
>> reading
>> >>>>>> this :)
>> >>>>>>>>>> ),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all
>> the
>> >>>>>> areas
>> >>>>>>>> we
>> >>>>>>>>>>>>>> wanted
>> >>>>>>>>>>>>>>>>>>>>> addressed.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>> >>>>>>>> afternoon,
>> >>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>> send
>> >>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>> another patch.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> >>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
>> conflicts
>> >> in
>> >>>>>> Java
>> >>>>>>>>>>>>>> code,
>> >>>>>>>>>>>>>>>>>>>> except
>> >>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>> >>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
>> disk_io_throttling
>> >>>>>> branch
>> >>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>> master
>> >>>>>>>>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com
>> >>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>> >>>>>> object_store
>> >>>>>>>>>>>>>> branch,
>> >>>>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>>>> -
>> >>>>>>>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
>> >> the
>> >>>>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>> branch
>> >>>>>>>>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
>> >> into
>> >>>>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>> >> this
>> >>>>>>>> merge
>> >>>>>>>>>>>>>>>>>> strategy.
>> >>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> >>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>> >>>>>>>>>>>>>>>> disk_io_throttling,
>> >>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
>> number
>> >>>>>> of
>> >>>>>>>>>>>>>>>> overlapping
>> >>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between
>> the
>> >>>>>>>>>> patches
>> >>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>>>>> follows:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects
>> of
>> >>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>> are
>> >>>>>>>>>>>>>>>>>>>>>>> additive
>> >>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>> >>>>>>>> dependency
>> >>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
>> graph,
>> >>>>>> we
>> >>>>>>>> can
>> >>>>>>>>>>>>>>>>>> determine
>> >>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my
>> code
>> >>>>>> into
>> >>>>>>>> his
>> >>>>>>>>>>>>>> branch
>> >>>>>>>>>>>>>>>>>>>>>>> before
>> >>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>> >>>>>> after his
>> >>>>>>>>>>>>>> merge
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
>> Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
>> here:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>> >>>>>> code in
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
>> storage
>> >>>>>>>> plug-in,
>> >>>>>>>>>>>>>> which
>> >>>>>>>>>>>>>>>>>>>>>>>>> currently
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in
>> use
>> >>>>>>>> (although
>> >>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>> does
>> >>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>> know
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>> >>>>>> actually
>> >>>>>>>> in
>> >>>>>>>>>>>>>> use))
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
>> placed in
>> >>>>>> the
>> >>>>>>>> url
>> >>>>>>>>>>>>>> field
>> >>>>>>>>>>>>>>>>>> (if
>> >>>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>> >> stored
>> >>>>>> in
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>> >>>>>> hypervisor
>> >>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>> question, we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes
>> the
>> >>>>>> place
>> >>>>>>>> of
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
>> hypervisor
>> >>>>>>>> checks
>> >>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> managed
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>> >>>>>>>> hypervisor
>> >>>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>>>>>>>>>>>> structure is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>> >>>>>> executes to
>> >>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> volume.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
>> invoked
>> >> to
>> >>>>>>>>>> detach a
>> >>>>>>>>>>>>>>>>>> volume,
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>> >> data
>> >>>>>>>>>>>>>> structure
>> >>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>> removed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>> >> IOPS
>> >>>>>> in
>> >>>>>>>> 4.2
>> >>>>>>>>>>>>>>>> unless
>> >>>>>>>>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings
>> table.
>> >> If
>> >>>>>> we
>> >>>>>>>>>> have
>> >>>>>>>>>>>>>> some
>> >>>>>>>>>>>>>>>>>>>>>>> idea,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
>> Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>> >>>>>> avoiding
>> >>>>>>>>>>>>>>>>>> implementation
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
>> details
>> >>>>>>>> field.
>> >>>>>>>>>> I
>> >>>>>>>>>>>>>>>> think
>> >>>>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>> >>>>>> regarding
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>> expected
>> >>>>>>>>>>>>>>>>>>>>>>>>> format
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>> details
>> >>>>>> are
>> >>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>> returned on
>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>> >>>>>>>> listStoragePool
>> >>>>>>>>>>>>>>>>>> response.
>> >>>>>>>>>>>>>>>>>>>>>>>> Why
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>> >>>>>> useful
>> >>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>> clients
>> >>>>>>>>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>> >>>>>> field."
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst
>> IOPS
>> >>>>>> here.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> >>>>>>>>>> Offering-by-Disk
>> >>>>>>>>>>>>>>>>>> Offering
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
>> have
>> >>>>>> to be
>> >>>>>>>>>>>>>> able to
>> >>>>>>>>>>>>>>>>>>>>>>>> associate
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>> >>>>>> disk_offering_details
>> >>>>>>>>>> table.
>> >>>>>>>>>>>>>>>> Maybe
>> >>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>> could
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
>> Burst
>> >>>>>> IOPS
>> >>>>>>>> in
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> GUI
>> >>>>>>>>>>>>>>>>>>>>>>> if
>> >>>>>>>>>>>>>>>>>>>>>>>>> it's
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>> >> the
>> >>>>>> DB.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play
>> >>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>> cloud<
>> >> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>> >>>
>> >>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the cloud<
>> >> http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the cloud<
>> >> http://solidfire.com/solution/overview/?video=play>
>> >>>> *â„¢*
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>>
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I am OK with that approach, John.
So, let me review to make sure I follow you correctly:
We introduce two new parameters to the plug-in: Number of Total IOPS for
the SAN and an overcommit ratio. (On a side note, if we are just
multiplying the two numbers, why don't we have the user just input their
product?)
Are both of these new parameters to the create storage pool API command or
are they passed into the create storage pool API command through its url
parameter?
If they are new parameters, we should make two new columns in the
storage_pool table.
If they are passed in via the url parameter, they should go in the
storage_pool_details table.
For 4.2, if someone wants to change these values, they must update the DB
manually.
Every time a volume is created or deleted for the SolidFire plug-in, the
Current IOPS value (sum of all volumes' Min IOPS that are associated with
the plug-in) is updated.
The allocator can use these fields to determine if it can fit in a new
volume.
Does it look like my understanding is OK?
On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I apologize for not being clear -- this conversation has been admittedly
> disjoint. I think we should allow the maximum IOPS and overcommitment
> values to be updated though I would recommend restricting updates to to be
> an increasing value for 4.2 (e.g. users can increase the number of total
> IOPS from 200,000 to 250,000, but not decrease from 250,000 to 200,000).
> While not ideal, given the amount of time we have left for 4.2, it will
> cover most cases, and we can address the implications of reducing resource
> capacity in 4.3. This approach addresses both of your concerns. First, it
> allows the administrator/operator to determine what portion of the device
> they wish to dedicate. For example, if the device has a total capacity of
> 200,000 IOPS, and they only want CS to use 25% of the device then they set
> the maximum total IOPS to 50,000. Second, as they grow capacity, they can
> update the DataStore to increase the number of IOPS they want to dedicate
> to CS' use. I would imagine expansion of capacity happens infrequently
> enough that increasing the maximum IOPS value would not be a significant
> burden.
>
> Thanks,
> -John
>
> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > "the administrator/operator simply needs to tell us the total number of
> > IOPS that can be committed to it and an overcommitment factor."
> >
> > Are you thinking when we create a plug-in as primary storage that we say
> -
> > up front - how many IOPS the SAN can handle?
> >
> > That is not a good move, in my opinion. Our SAN is designed to start
> small
> > and grow to PBs. As the need arises for more storage, the admin purchases
> > additional storage nodes that join the cluster and the performance and
> > capacity go up.
> >
> > We need to know how many IOPS total the SAN can handle and what it is
> > committed to currently (the sum of the number of volumes' min IOPS).
> >
> > We also cannot assume the SAN is dedicated to CS.
> >
> >
> > On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Simon,
> >>
> >> Yes, it is CloudStack's job to protect, as best it can, from
> >> oversubscribing resources. I would argue that resource management is
> one,
> >> if not the most, important functions of the system. It is no different
> >> than the allocation/planning performed for hosts relative to cores and
> >> memory. We can still oversubscribe resources, but we have rails + knobs
> >> and dials to avoid it. Without these controls in place, we could easily
> >> allow users to deploy workloads that overrun resources harming all
> tenants.
> >>
> >> I also think that we are over thinking this issue for provisioned IOPS.
> >> When the DataStore is configured, the administrator/operator simply
> needs
> >> to tell us the total number of IOPS that can be committed to it and an
> >> overcommitment factor. As we allocate volumes to that DataStore, we
> sum up
> >> the committed IOPS of the existing Volumes attached to the DataStore,
> apply
> >> the overcommitment factor, and determine whether or not the requested
> >> minimum IOPS for the new volume can be fulfilled. We can provide both
> >> general and vendor specific documentation for determining these values
> --
> >> be they to consume the entire device or a portion of it.
> >>
> >> Querying the device is unnecessary and deceptive. CloudStack resource
> >> management is not interested in the current state of the device which
> could
> >> be anywhere from extremely heavy to extremely light at any given time.
> We
> >> are interested in the worst case load that is anticipated for resource.
> In
> >> my view, it is up to administrators/operators to instrument their
> >> environment to understand usage patterns and capacity. We should
> provide
> >> information that will help determine what should be
> instrumented/monitored,
> >> but that function should be performed outside of CloudStack.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
> >>
> >>> I'd like to comment on this briefly.
> >>>
> >>>
> >>>
> >>> I think an assumption is being made that the SAN is being dedicated to
> a
> >> CS instance.
> >>>
> >>> My person opinion that this whole IOPS calculation is getting rather
> >> complicated, and could probably be much simpler than this. Over
> >> subscription is a fact of life on virtually all storage, and is really
> no
> >> different in concept than multiple virt instances on a single piece of
> >> hardware. All decent SANs offer many management options for the storage
> >> engineers to keep track of IOPS utilization, and plan for spindle
> >> augmentation as required.
> >>> Is it really the job of CS to become yet another management layer on
> top
> >> of this?
> >>>
> >>> ----- Original Message -----
> >>>
> >>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>> To: dev@cloudstack.apache.org
> >>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
> >> ustcweizhou@gmail.com>
> >>> Sent: Friday, June 14, 2013 1:00:26 PM
> >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>
> >>> 1) We want number of IOPS currently supported by the SAN.
> >>>
> >>> 2) We want the number of IOPS that are committed (sum of min IOPS for
> >> each
> >>> volume).
> >>>
> >>> We could do the following to keep track of IOPS:
> >>>
> >>> The plug-in could have a timer thread that goes off every, say, 1
> minute.
> >>>
> >>> It could query the SAN for the number of nodes that make up the SAN and
> >>> multiple this by 50,000. This is essentially the number of supported
> IOPS
> >>> of the SAN.
> >>>
> >>> The next API call could be to get all of the volumes on the SAN.
> Iterate
> >>> through them all and add up their min IOPS values. This is the number
> of
> >>> IOPS the SAN is committed to.
> >>>
> >>> These two numbers can then be updated in the storage_pool table (a
> column
> >>> for each value).
> >>>
> >>> The allocators can get these values as needed (and they would be as
> >>> accurate as the last time the thread asked the SAN for this info).
> >>>
> >>> These two fields, the min IOPS of the volume to create, and the
> >> overcommit
> >>> ratio of the plug-in would tell the allocator if it can select the
> given
> >>> storage pool.
> >>>
> >>> What do you think?
> >>>
> >>>
> >>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> "As I mentioned previously, I am very reluctant for any feature to
> come
> >>>> into master that can exhaust resources."
> >>>>
> >>>> Just wanted to mention that, worst case, the SAN would fail creation
> of
> >>>> the volume before allowing a new volume to break the system.
> >>>>
> >>>>
> >>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> Hi John,
> >>>>>
> >>>>> Are you thinking we add a column on to the storage pool table,
> >>>>> IOPS_Count, where we add and subtract committed IOPS?
> >>>>>
> >>>>> That is easy enough.
> >>>>>
> >>>>> How do you want to determine what the SAN is capable of supporting
> IOPS
> >>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
> >> storage
> >>>>> nodes to the cluster, the number of IOPS increases. Do we have a
> >> thread we
> >>>>> can use to query external devices like this SAN to update the
> supported
> >>>>> number of IOPS?
> >>>>>
> >>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
> >>>>> overcommit ration to the plug-in when it's created? We would need to
> >> store
> >>>>> this in the storage_pool table, as well, I believe.
> >>>>>
> >>>>> We should also get Wei involved in this as his feature will need
> >> similar
> >>>>> functionality.
> >>>>>
> >>>>> Also, we should do this FAST as we have only two weeks left and many
> of
> >>>>> us will be out for several days for the CS Collab Conference.
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
> >>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Querying the SAN only indicates the number of IOPS currently in use.
> >>>>>> The allocator needs to check the number of IOPS committed which is
> >> tracked
> >>>>>> by CloudStack. For 4.2, we should be able to query the number of
> IOPS
> >>>>>> committed to a DataStore, and determine whether or not the number
> >> requested
> >>>>>> can be fulfilled by that device. It seems to be that a
> >>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
> >>>>>> DataStore's that don't support provisioned IOPS would return null.
> >>>>>>
> >>>>>> As I mentioned previously, I am very reluctant for any feature to
> come
> >>>>>> into master that can exhaust resources.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Yeah, I'm not sure I could come up with anything near an accurate
> >>>>>>> assessment of how many IOPS are currently available on the SAN (or
> >>>>>> even a
> >>>>>>> total number that are available for volumes). Not sure if there's
> yet
> >>>>>> an
> >>>>>>> API call for that.
> >>>>>>>
> >>>>>>> If I did know this number (total number of IOPS supported by the
> >> SAN),
> >>>>>> we'd
> >>>>>>> still have to keep track of the total number of volumes we've
> created
> >>>>>> from
> >>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an
> API
> >>>>>> call
> >>>>>>> directly to the SAN to tweak the number of IOPS on a given volume
> or
> >>>>>> set of
> >>>>>>> volumes (not supported from CS, but supported via the SolidFire
> API),
> >>>>>> our
> >>>>>>> numbers in CS would be off.
> >>>>>>>
> >>>>>>> I'm thinking verifying sufficient number of IOPS is a really good
> >> idea
> >>>>>> for
> >>>>>>> a future release.
> >>>>>>>
> >>>>>>> For 4.2 I think we should stick to having the allocator detect if
> >>>>>> storage
> >>>>>>> QoS is desired and if the storage pool in question supports that
> >>>>>> feature.
> >>>>>>>
> >>>>>>> If you really are over provisioned on your SAN in terms of IOPS or
> >>>>>>> capacity, the SAN can let the admin know in several different ways
> >>>>>> (e-mail,
> >>>>>>> SNMP, GUI).
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> Please see my comments in-line below.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Comments below in red.
> >>>>>>>>>
> >>>>>>>>> Thanks
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
> jburwell@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we
> may
> >>>>>> want
> >>>>>>>> to
> >>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
> >>>>>> user
> >>>>>>>> is
> >>>>>>>>>> expecting that the hypervisor will provide a particular QoS, and
> >>>>>> that is
> >>>>>>>>>> not possible, it seems like we should inform them rather than
> >>>>>> silently
> >>>>>>>>>> ignoring their request.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Sure, that sounds reasonable.
> >>>>>>>>>
> >>>>>>>>> We'd have to come up with some way for the allocators to know
> about
> >>>>>> the
> >>>>>>>>> requested storage QoS and then query the candidate drivers.
> >>>>>>>>>
> >>>>>>>>> Any thoughts on how we might do that?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> To collect my thoughts from previous parts of the thread, I am
> >>>>>>>>>> uncomfortable with the idea that the management server can
> >>>>>> overcommit a
> >>>>>>>>>> resource. You had mentioned querying the device for available
> >> IOPS.
> >>>>>>>> While
> >>>>>>>>>> that would be nice, it seems like we could fall back to a max
> IOPS
> >>>>>> and
> >>>>>>>>>> overcommit factor manually calculated and entered by the
> >>>>>>>>>> administrator/operator. I think such threshold and allocation
> >> rails
> >>>>>>>> should
> >>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
> >>>>>> basic
> >>>>>>>>>> feature of any cloud orchestration platform.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Are you thinking this ability would make it into 4.2? Just
> curious
> >>>>>> what
> >>>>>>>>> release we're talking about here. For the SolidFire SAN, you
> might
> >>>>>> have,
> >>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
> >>>>>> later
> >>>>>>>> add
> >>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a
> >> way
> >>>>>> to
> >>>>>>>>> know that the number of supported IOPS has increased.
> >>>>>>>>
> >>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
> >>>>>> example,
> >>>>>>>> we may only support expanding capacity in 4.2, and not handle any
> >>>>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
> >>>>>> a
> >>>>>>>> bit, the throttled I/O enhancement seems like it needs a similar
> set
> >>>>>> of
> >>>>>>>> rails defined per host.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in
> a
> >>>>>>>>>> implementation specific manner. I think we need to arrive at a
> >>>>>> general
> >>>>>>>>>> model that can be exploited by the allocators and planners. We
> >>>>>> should
> >>>>>>>>>> restrict implementation specific key-value pairs (call them
> >> details,
> >>>>>>>>>> extended data, whatever) to information that is unique to the
> >>>>>> driver and
> >>>>>>>>>> would provide no useful information to the management server's
> >>>>>>>>>> orchestration functions. A resource QoS does not seem to fit
> those
> >>>>>>>>>> criteria.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
> >>>>>> Collab
> >>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
> >>>>>>>>
> >>>>>>>> Yes, it would -- I will put something in the wiki topic. It will
> >>>>>> also be
> >>>>>>>> part of my talk on Monday -- How to Run from Zombie which include
> >>>>>> some of
> >>>>>>>> my opinions on the topic.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> So, here's my suggestion for 4.2:
> >>>>>>>>>>>
> >>>>>>>>>>> Accept the values as they are currently required (four new
> fields
> >>>>>> for
> >>>>>>>>>> Wei's
> >>>>>>>>>>> feature or two new fields for mine).
> >>>>>>>>>>>
> >>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
> >>>>>>>>>>>
> >>>>>>>>>>> 1) No QoS
> >>>>>>>>>>>
> >>>>>>>>>>> 2) Hypervisor QoS
> >>>>>>>>>>>
> >>>>>>>>>>> 3) Storage Qos
> >>>>>>>>>>>
> >>>>>>>>>>> The admin needs to specify storage tags that only map to
> storage
> >>>>>> that
> >>>>>>>>>>> supports Storage QoS.
> >>>>>>>>>>>
> >>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
> >>>>>>>>>> hypervisors
> >>>>>>>>>>> in use support the new fields, they may not be enforced.
> >>>>>>>>>>>
> >>>>>>>>>>> Post 4.3:
> >>>>>>>>>>>
> >>>>>>>>>>> Come up with a way to more generally enter these parameters
> >>>>>> (probably
> >>>>>>>>>> just
> >>>>>>>>>>> key/value pairs sent to the drivers).
> >>>>>>>>>>>
> >>>>>>>>>>> Have the drivers expose their feature set so the allocators can
> >>>>>>>> consider
> >>>>>>>>>>> them more fully and throw an exception if there is not a
> >> sufficient
> >>>>>>>>>> match.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put
> >> some
> >>>>>>>>>> burden
> >>>>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
> >>>>>> sense.
> >>>>>>>> For
> >>>>>>>>>>>> example, if he wants to use storage QoS with supported Min and
> >> Max
> >>>>>>>>>> values,
> >>>>>>>>>>>> he'll have to put in a storage tag that references the
> SolidFire
> >>>>>>>> primary
> >>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
> >> then
> >>>>>> he's
> >>>>>>>>>> not
> >>>>>>>>>>>> going to get the Min and Max feature. We could add help text
> to
> >>>>>> the
> >>>>>>>>>> pop-up
> >>>>>>>>>>>> dialog that's displayed when you click in the Min and Max text
> >>>>>> fields.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Same idea for Wei's feature.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
> >> comprehensive
> >>>>>>>>>>>> approach for post 4.2.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Maybe in the future we could have the drivers advertise their
> >>>>>>>>>> capabilities
> >>>>>>>>>>>> and if the allocator feels a request is not being satisfied
> (say
> >>>>>> Min
> >>>>>>>> was
> >>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
> >>>>>> throw
> >>>>>>>> an
> >>>>>>>>>>>> exception.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please see my comment in-line below.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I've put comments below in red.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>>>>> jburwell@basho.com
> >>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> First and foremost, we must ensure that these two features
> >> are
> >>>>>>>>>>>>>> mutually
> >>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration
> that
> >>>>>>>>>>>>>> contains both
> >>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >>>>>>>>>> non-deterministic
> >>>>>>>>>>>>>>>> operations.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
> >> storage
> >>>>>>>>>> oriented
> >>>>>>>>>>>>>>>> solves the problem in short term. As I understand storage
> >>>>>> tags,
> >>>>>>>> we
> >>>>>>>>>>>>>> have no
> >>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I
> wasn't
> >>>>>>>>>>>>>> necessarily
> >>>>>>>>>>>>>>>> intending that we implement this allocation model in 4.3,
> >> but
> >>>>>>>>>> instead,
> >>>>>>>>>>>>>>>> determine if this type model would be one we would want to
> >>>>>> support
> >>>>>>>>>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the
> data
> >>>>>> model
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>> current implementation would not preclude evolution in
> that
> >>>>>>>>>>>>>> direction. My
> >>>>>>>>>>>>>>>> view is that this type of allocation model is what user's
> >>>>>> expect
> >>>>>>>> of
> >>>>>>>>>>>>>> "cloud"
> >>>>>>>>>>>>>>>> systems -- selecting the best available resource set to
> >>>>>> fulfill a
> >>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>> system requirements.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I believe we have meet your requirement here in that what
> >> we've
> >>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>> should not make refinement difficult in the future. If we
> >> don't
> >>>>>>>>>> modify
> >>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
> relatively
> >>>>>> simple
> >>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Looking through both patches, I have to say that the
> >> aggregated
> >>>>>>>> result
> >>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
> >>>>>> throttled
> >>>>>>>>>> I/O and
> >>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My
> concern
> >>>>>> is
> >>>>>>>> not
> >>>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
> >>>>>> Javadoc
> >>>>>>>>>> should
> >>>>>>>>>>>>>> be added explaining the two sets of attributes and their
> >> mutual
> >>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
> >>>>>> exclusion.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
> >>>>>> provides
> >>>>>>>> both
> >>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned
> IOPS
> >>>>>> uses
> >>>>>>>> a
> >>>>>>>>>>>>>> range. In order to select the best available resource to
> >>>>>> fulfill a
> >>>>>>>>>> QoS, we
> >>>>>>>>>>>>>> would need to have the QoS expression normalized in terms of
> >>>>>> units
> >>>>>>>>>> (IOPS or
> >>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
> >>>>>>>> achieve a
> >>>>>>>>>>>>>> model like I described, I think we would need to resolve
> this
> >>>>>> issue
> >>>>>>>>>> in 4.2
> >>>>>>>>>>>>>> to ensure a viable migration path.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think we're not likely to be able to normalize the input
> for
> >>>>>> 4.2.
> >>>>>>>>>> Plus
> >>>>>>>>>>>>> people probably want to input the data in terms they're
> >> familiar
> >>>>>> with
> >>>>>>>>>> for
> >>>>>>>>>>>>> the products in question.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let
> the
> >>>>>> user
> >>>>>>>>>> send
> >>>>>>>>>>>>> key/value pairs to each vendor that could be selected due to
> a
> >>>>>> given
> >>>>>>>>>>>>> storage tag. I'm still not sure that would solve it because
> >> what
> >>>>>>>>>> happens if
> >>>>>>>>>>>>> you change the storage tag of a given Primary Storage after
> >>>>>> having
> >>>>>>>>>> created
> >>>>>>>>>>>>> a Disk Offering?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
> >>>>>> re-think
> >>>>>>>> it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
> >>>>>> supported
> >>>>>>>>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> As I think through the implications of these requirements
> >> and
> >>>>>>>>>> reflect
> >>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
> >>>>>> impacted
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
> >>>>>> provisioned
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> has no
> >>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
> >>>>>> that
> >>>>>>>> have
> >>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
> >>>>>> understand
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
> >>>>>> potentially
> >>>>>>>>>>>>>> causing
> >>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
> >>>>>> to me
> >>>>>>>>>>>>>> that a
> >>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
> >>>>>> maximum
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>> it can support and some type of overcommitment factor.
> This
> >>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>> should be used by the storage allocators to determine the
> >>>>>> device
> >>>>>>>>>> best
> >>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems that
> a
> >>>>>>>> similar
> >>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
> >> layer
> >>>>>>>> which
> >>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path
> that
> >>>>>> was
> >>>>>>>>>>>>>> previously
> >>>>>>>>>>>>>>> established for other properties you see on dialogs in CS:
> >> Just
> >>>>>>>>>> because
> >>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed
> to
> >>>>>>>> supports
> >>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
> >>>>>> storage
> >>>>>>>> tag
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>> confines the deployment only to storage that supports the
> >>>>>> selected
> >>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
> >>>>>> CloudStack
> >>>>>>>>>> works
> >>>>>>>>>>>>>>> today.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I understand the tag functionality, and the need for the
> >>>>>>>> administrator
> >>>>>>>>>>>>>> to very carefully construct offerings. My concern is that we
> >>>>>> over
> >>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
> >>>>>>>>>> illustration,
> >>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
> >> device
> >>>>>> is
> >>>>>>>>>> 100000.
> >>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
> >>>>>> Therefore,
> >>>>>>>> we
> >>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
> >>>>>> guaranteed
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
> >>>>>>>>>> configuration
> >>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
> >>>>>> As we
> >>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
> >>>>>> guarantee
> >>>>>>>>>> more IOPS
> >>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that
> make
> >>>>>>>> sense?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not
> >> even
> >>>>>>>> sure
> >>>>>>>>>> I
> >>>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain.
> I'd
> >>>>>> have
> >>>>>>>> to
> >>>>>>>>>> get
> >>>>>>>>>>>>> all of the min values for all of the volumes on the SAN and
> >> total
> >>>>>>>> them
> >>>>>>>>>> up,
> >>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
> >> supported
> >>>>>>>> IOPS
> >>>>>>>>>> of
> >>>>>>>>>>>>> the system.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
> >>>>>> resources
> >>>>>>>>>> to
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
> >>>>>>>> provisioned
> >>>>>>>>>>>>>> IOPS, I
> >>>>>>>>>>>>>>>> think a side effect of the current implementation is
> >> SolidFire
> >>>>>>>>>> volumes
> >>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O,
> it
> >>>>>>>> seems
> >>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
> side-by-side
> >>>>>> with
> >>>>>>>> QoS
> >>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
> >>>>>> starve
> >>>>>>>>>> out
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
> >>>>>>>> collocated
> >>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>> from being fulfilled.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS)
> and
> >>>>>> not
> >>>>>>>>>>>>>> specify
> >>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at
> the
> >>>>>>>>>> defaults
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN,
> >> you
> >>>>>> will
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>> your Mins met.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
> >>>>>> exclusions
> >>>>>>>>>>>>>> list,
> >>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
> deploy
> >> my
> >>>>>>>>>> volume
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> storage with these following tags).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
> >>>>>> would
> >>>>>>>>>>>>>> complicate component assembly. It would require a storage
> >>>>>> plugin to
> >>>>>>>>>>>>>> anticipate all of the possible component assemblies and
> >>>>>> determine
> >>>>>>>> the
> >>>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
> >>>>>>>>>> capabilities
> >>>>>>>>>>>>>> which can be matched to a set of requested requirements.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion
> >> or
> >>>>>>>>>>>>> exclusion lists. It just needs to advertise its functionality
> >> in
> >>>>>> a
> >>>>>>>> way
> >>>>>>>>>> the
> >>>>>>>>>>>>> allocator understands.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it
> only
> >>>>>>>> limits
> >>>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
> >>>>>> with
> >>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>> minimum IOPS does this.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does
> not
> >>>>>>>> exist.
> >>>>>>>>>>>>>> However, I am concerned with the more general case as we
> >>>>>> supported
> >>>>>>>>>> more
> >>>>>>>>>>>>>> devices with provisioned IOPS.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
> >>>>>> values
> >>>>>>>> to
> >>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
> >> provisioned
> >>>>>>>>>> IOPS, but
> >>>>>>>>>>>>> what if you break them out by read and write or do something
> >>>>>> else? We
> >>>>>>>>>> need
> >>>>>>>>>>>>> a more general mechanism.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
> >>>>>> I/O
> >>>>>>>> and
> >>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per
> volume.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We also need to understand the implications of these QoS
> >>>>>>>> guarantees
> >>>>>>>>>> on
> >>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
> >> hardware
> >>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
> >>>>>> forced
> >>>>>>>> to
> >>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
> >>>>>> implementation
> >>>>>>>> in
> >>>>>>>>>>>>>> 4.3.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I agree, John. I also think you've come up with some great
> >>>>>> ideas
> >>>>>>>> for
> >>>>>>>>>>>>>> 4.3. :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
> >>>>>> storage
> >>>>>>>>>>>>>> tags).
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
> >>>>>> addition to
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
> >> being
> >>>>>>>>>>>>>> preferred,
> >>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>> is).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I think John's concern is that you can enter in values
> for
> >>>>>> Wei's
> >>>>>>>>>> and
> >>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
> >>>>>> yet), so
> >>>>>>>> he
> >>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators
> >> to
> >>>>>>>> prefer
> >>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
> today
> >>>>>> in
> >>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>> we already have this kind of an issue with other features
> >>>>>> (fields
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>> dialogs for features that not all vendors support).
> Perhaps
> >>>>>> post
> >>>>>>>>>> 4.2
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
> >>>>>> OpenStack
> >>>>>>>>>>>>>> addresses
> >>>>>>>>>>>>>>>>> the issue).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
> >> until
> >>>>>>>> code
> >>>>>>>>>>>>>>>> freeze)
> >>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep
> Wei's
> >>>>>> fields
> >>>>>>>>>>>>>> and my
> >>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
> >>>>>> data
> >>>>>>>>>>>>>> entered
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
> >>>>>> design
> >>>>>>>>>>>>>> that can
> >>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
> >>>>>> situations.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I just published my review @
> >>>>>>>> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
> >>>>>> with a
> >>>>>>>>>> new
> >>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>> file.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
> >>>>>> checked
> >>>>>>>>>> in
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
> file?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I
> am
> >>>>>>>>>> offline
> >>>>>>>>>>>>>> ATM,
> >>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
> >>>>>> this :)
> >>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
> >>>>>> areas
> >>>>>>>> we
> >>>>>>>>>>>>>> wanted
> >>>>>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> >>>>>>>> afternoon,
> >>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
> >> in
> >>>>>> Java
> >>>>>>>>>>>>>> code,
> >>>>>>>>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
> >>>>>> branch
> >>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> >> mike.tutkowski@solidfire.com
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>>>> object_store
> >>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
> >> the
> >>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
> >> into
> >>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
> >> this
> >>>>>>>> merge
> >>>>>>>>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>>>>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
> number
> >>>>>> of
> >>>>>>>>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between
> the
> >>>>>>>>>> patches
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects
> of
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> >>>>>>>> dependency
> >>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency
> graph,
> >>>>>> we
> >>>>>>>> can
> >>>>>>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
> >>>>>> into
> >>>>>>>> his
> >>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
> >>>>>> after his
> >>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
> here:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
> >>>>>> code in
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default
> storage
> >>>>>>>> plug-in,
> >>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> >>>>>>>> (although
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
> >>>>>> actually
> >>>>>>>> in
> >>>>>>>>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed
> in
> >>>>>> the
> >>>>>>>> url
> >>>>>>>>>>>>>> field
> >>>>>>>>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
> >> stored
> >>>>>> in
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
> >>>>>> hypervisor
> >>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
> >>>>>> place
> >>>>>>>> of
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
> hypervisor
> >>>>>>>> checks
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
> >>>>>> executes to
> >>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
> >> to
> >>>>>>>>>> detach a
> >>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
> >> data
> >>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
> >> IOPS
> >>>>>> in
> >>>>>>>> 4.2
> >>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
> >> If
> >>>>>> we
> >>>>>>>>>> have
> >>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
> >>>>>> avoiding
> >>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the
> details
> >>>>>>>> field.
> >>>>>>>>>> I
> >>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
> >>>>>> regarding
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
> details
> >>>>>> are
> >>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >>>>>>>> listStoragePool
> >>>>>>>>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
> >>>>>> useful
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
> >>>>>> field."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst
> IOPS
> >>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>>>>>>>>> Offering-by-Disk
> >>>>>>>>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you
> have
> >>>>>> to be
> >>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>>>> disk_offering_details
> >>>>>>>>>> table.
> >>>>>>>>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
> Burst
> >>>>>> IOPS
> >>>>>>>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
> >> the
> >>>>>> DB.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
I apologize for not being clear -- this conversation has been admittedly disjoint. I think we should allow the maximum IOPS and overcommitment values to be updated though I would recommend restricting updates to to be an increasing value for 4.2 (e.g. users can increase the number of total IOPS from 200,000 to 250,000, but not decrease from 250,000 to 200,000). While not ideal, given the amount of time we have left for 4.2, it will cover most cases, and we can address the implications of reducing resource capacity in 4.3. This approach addresses both of your concerns. First, it allows the administrator/operator to determine what portion of the device they wish to dedicate. For example, if the device has a total capacity of 200,000 IOPS, and they only want CS to use 25% of the device then they set the maximum total IOPS to 50,000. Second, as they grow capacity, they can update the DataStore to increase the number of IOPS they want to dedicate to CS' use. I would imagine expansion of capacity happens infrequently enough that increasing the maximum IOPS value would not be a significant burden.
Thanks,
-John
On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> "the administrator/operator simply needs to tell us the total number of
> IOPS that can be committed to it and an overcommitment factor."
>
> Are you thinking when we create a plug-in as primary storage that we say -
> up front - how many IOPS the SAN can handle?
>
> That is not a good move, in my opinion. Our SAN is designed to start small
> and grow to PBs. As the need arises for more storage, the admin purchases
> additional storage nodes that join the cluster and the performance and
> capacity go up.
>
> We need to know how many IOPS total the SAN can handle and what it is
> committed to currently (the sum of the number of volumes' min IOPS).
>
> We also cannot assume the SAN is dedicated to CS.
>
>
> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com> wrote:
>
>> Simon,
>>
>> Yes, it is CloudStack's job to protect, as best it can, from
>> oversubscribing resources. I would argue that resource management is one,
>> if not the most, important functions of the system. It is no different
>> than the allocation/planning performed for hosts relative to cores and
>> memory. We can still oversubscribe resources, but we have rails + knobs
>> and dials to avoid it. Without these controls in place, we could easily
>> allow users to deploy workloads that overrun resources harming all tenants.
>>
>> I also think that we are over thinking this issue for provisioned IOPS.
>> When the DataStore is configured, the administrator/operator simply needs
>> to tell us the total number of IOPS that can be committed to it and an
>> overcommitment factor. As we allocate volumes to that DataStore, we sum up
>> the committed IOPS of the existing Volumes attached to the DataStore, apply
>> the overcommitment factor, and determine whether or not the requested
>> minimum IOPS for the new volume can be fulfilled. We can provide both
>> general and vendor specific documentation for determining these values --
>> be they to consume the entire device or a portion of it.
>>
>> Querying the device is unnecessary and deceptive. CloudStack resource
>> management is not interested in the current state of the device which could
>> be anywhere from extremely heavy to extremely light at any given time. We
>> are interested in the worst case load that is anticipated for resource. In
>> my view, it is up to administrators/operators to instrument their
>> environment to understand usage patterns and capacity. We should provide
>> information that will help determine what should be instrumented/monitored,
>> but that function should be performed outside of CloudStack.
>>
>> Thanks,
>> -John
>>
>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>>
>>> I'd like to comment on this briefly.
>>>
>>>
>>>
>>> I think an assumption is being made that the SAN is being dedicated to a
>> CS instance.
>>>
>>> My person opinion that this whole IOPS calculation is getting rather
>> complicated, and could probably be much simpler than this. Over
>> subscription is a fact of life on virtually all storage, and is really no
>> different in concept than multiple virt instances on a single piece of
>> hardware. All decent SANs offer many management options for the storage
>> engineers to keep track of IOPS utilization, and plan for spindle
>> augmentation as required.
>>> Is it really the job of CS to become yet another management layer on top
>> of this?
>>>
>>> ----- Original Message -----
>>>
>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>> To: dev@cloudstack.apache.org
>>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
>> ustcweizhou@gmail.com>
>>> Sent: Friday, June 14, 2013 1:00:26 PM
>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>
>>> 1) We want number of IOPS currently supported by the SAN.
>>>
>>> 2) We want the number of IOPS that are committed (sum of min IOPS for
>> each
>>> volume).
>>>
>>> We could do the following to keep track of IOPS:
>>>
>>> The plug-in could have a timer thread that goes off every, say, 1 minute.
>>>
>>> It could query the SAN for the number of nodes that make up the SAN and
>>> multiple this by 50,000. This is essentially the number of supported IOPS
>>> of the SAN.
>>>
>>> The next API call could be to get all of the volumes on the SAN. Iterate
>>> through them all and add up their min IOPS values. This is the number of
>>> IOPS the SAN is committed to.
>>>
>>> These two numbers can then be updated in the storage_pool table (a column
>>> for each value).
>>>
>>> The allocators can get these values as needed (and they would be as
>>> accurate as the last time the thread asked the SAN for this info).
>>>
>>> These two fields, the min IOPS of the volume to create, and the
>> overcommit
>>> ratio of the plug-in would tell the allocator if it can select the given
>>> storage pool.
>>>
>>> What do you think?
>>>
>>>
>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> "As I mentioned previously, I am very reluctant for any feature to come
>>>> into master that can exhaust resources."
>>>>
>>>> Just wanted to mention that, worst case, the SAN would fail creation of
>>>> the volume before allowing a new volume to break the system.
>>>>
>>>>
>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> Are you thinking we add a column on to the storage pool table,
>>>>> IOPS_Count, where we add and subtract committed IOPS?
>>>>>
>>>>> That is easy enough.
>>>>>
>>>>> How do you want to determine what the SAN is capable of supporting IOPS
>>>>> wise? Remember we're dealing with a dynamic SAN here...as you add
>> storage
>>>>> nodes to the cluster, the number of IOPS increases. Do we have a
>> thread we
>>>>> can use to query external devices like this SAN to update the supported
>>>>> number of IOPS?
>>>>>
>>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>>>>> overcommit ration to the plug-in when it's created? We would need to
>> store
>>>>> this in the storage_pool table, as well, I believe.
>>>>>
>>>>> We should also get Wei involved in this as his feature will need
>> similar
>>>>> functionality.
>>>>>
>>>>> Also, we should do this FAST as we have only two weeks left and many of
>>>>> us will be out for several days for the CS Collab Conference.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Querying the SAN only indicates the number of IOPS currently in use.
>>>>>> The allocator needs to check the number of IOPS committed which is
>> tracked
>>>>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
>>>>>> committed to a DataStore, and determine whether or not the number
>> requested
>>>>>> can be fulfilled by that device. It seems to be that a
>>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>>>>> DataStore's that don't support provisioned IOPS would return null.
>>>>>>
>>>>>> As I mentioned previously, I am very reluctant for any feature to come
>>>>>> into master that can exhaust resources.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> Yeah, I'm not sure I could come up with anything near an accurate
>>>>>>> assessment of how many IOPS are currently available on the SAN (or
>>>>>> even a
>>>>>>> total number that are available for volumes). Not sure if there's yet
>>>>>> an
>>>>>>> API call for that.
>>>>>>>
>>>>>>> If I did know this number (total number of IOPS supported by the
>> SAN),
>>>>>> we'd
>>>>>>> still have to keep track of the total number of volumes we've created
>>>>>> from
>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an API
>>>>>> call
>>>>>>> directly to the SAN to tweak the number of IOPS on a given volume or
>>>>>> set of
>>>>>>> volumes (not supported from CS, but supported via the SolidFire API),
>>>>>> our
>>>>>>> numbers in CS would be off.
>>>>>>>
>>>>>>> I'm thinking verifying sufficient number of IOPS is a really good
>> idea
>>>>>> for
>>>>>>> a future release.
>>>>>>>
>>>>>>> For 4.2 I think we should stick to having the allocator detect if
>>>>>> storage
>>>>>>> QoS is desired and if the storage pool in question supports that
>>>>>> feature.
>>>>>>>
>>>>>>> If you really are over provisioned on your SAN in terms of IOPS or
>>>>>>> capacity, the SAN can let the admin know in several different ways
>>>>>> (e-mail,
>>>>>>> SNMP, GUI).
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> Please see my comments in-line below.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Comments below in red.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we may
>>>>>> want
>>>>>>>> to
>>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
>>>>>> user
>>>>>>>> is
>>>>>>>>>> expecting that the hypervisor will provide a particular QoS, and
>>>>>> that is
>>>>>>>>>> not possible, it seems like we should inform them rather than
>>>>>> silently
>>>>>>>>>> ignoring their request.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sure, that sounds reasonable.
>>>>>>>>>
>>>>>>>>> We'd have to come up with some way for the allocators to know about
>>>>>> the
>>>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>>>
>>>>>>>>> Any thoughts on how we might do that?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> To collect my thoughts from previous parts of the thread, I am
>>>>>>>>>> uncomfortable with the idea that the management server can
>>>>>> overcommit a
>>>>>>>>>> resource. You had mentioned querying the device for available
>> IOPS.
>>>>>>>> While
>>>>>>>>>> that would be nice, it seems like we could fall back to a max IOPS
>>>>>> and
>>>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>>>> administrator/operator. I think such threshold and allocation
>> rails
>>>>>>>> should
>>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
>>>>>> basic
>>>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Are you thinking this ability would make it into 4.2? Just curious
>>>>>> what
>>>>>>>>> release we're talking about here. For the SolidFire SAN, you might
>>>>>> have,
>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
>>>>>> later
>>>>>>>> add
>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a
>> way
>>>>>> to
>>>>>>>>> know that the number of supported IOPS has increased.
>>>>>>>>
>>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>>>>> example,
>>>>>>>> we may only support expanding capacity in 4.2, and not handle any
>>>>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
>>>>>> a
>>>>>>>> bit, the throttled I/O enhancement seems like it needs a similar set
>>>>>> of
>>>>>>>> rails defined per host.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>>>>>>>>> implementation specific manner. I think we need to arrive at a
>>>>>> general
>>>>>>>>>> model that can be exploited by the allocators and planners. We
>>>>>> should
>>>>>>>>>> restrict implementation specific key-value pairs (call them
>> details,
>>>>>>>>>> extended data, whatever) to information that is unique to the
>>>>>> driver and
>>>>>>>>>> would provide no useful information to the management server's
>>>>>>>>>> orchestration functions. A resource QoS does not seem to fit those
>>>>>>>>>> criteria.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
>>>>>> Collab
>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>>>
>>>>>>>> Yes, it would -- I will put something in the wiki topic. It will
>>>>>> also be
>>>>>>>> part of my talk on Monday -- How to Run from Zombie which include
>>>>>> some of
>>>>>>>> my opinions on the topic.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>>>
>>>>>>>>>>> Accept the values as they are currently required (four new fields
>>>>>> for
>>>>>>>>>> Wei's
>>>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>>>
>>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>>>>>>>
>>>>>>>>>>> 1) No QoS
>>>>>>>>>>>
>>>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>>>
>>>>>>>>>>> 3) Storage Qos
>>>>>>>>>>>
>>>>>>>>>>> The admin needs to specify storage tags that only map to storage
>>>>>> that
>>>>>>>>>>> supports Storage QoS.
>>>>>>>>>>>
>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>>>>>>>>> hypervisors
>>>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>>>
>>>>>>>>>>> Post 4.3:
>>>>>>>>>>>
>>>>>>>>>>> Come up with a way to more generally enter these parameters
>>>>>> (probably
>>>>>>>>>> just
>>>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>>>
>>>>>>>>>>> Have the drivers expose their feature set so the allocators can
>>>>>>>> consider
>>>>>>>>>>> them more fully and throw an exception if there is not a
>> sufficient
>>>>>>>>>> match.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put
>> some
>>>>>>>>>> burden
>>>>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
>>>>>> sense.
>>>>>>>> For
>>>>>>>>>>>> example, if he wants to use storage QoS with supported Min and
>> Max
>>>>>>>>>> values,
>>>>>>>>>>>> he'll have to put in a storage tag that references the SolidFire
>>>>>>>> primary
>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
>> then
>>>>>> he's
>>>>>>>>>> not
>>>>>>>>>>>> going to get the Min and Max feature. We could add help text to
>>>>>> the
>>>>>>>>>> pop-up
>>>>>>>>>>>> dialog that's displayed when you click in the Min and Max text
>>>>>> fields.
>>>>>>>>>>>>
>>>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>>>
>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
>> comprehensive
>>>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe in the future we could have the drivers advertise their
>>>>>>>>>> capabilities
>>>>>>>>>>>> and if the allocator feels a request is not being satisfied (say
>>>>>> Min
>>>>>>>> was
>>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
>>>>>> throw
>>>>>>>> an
>>>>>>>>>>>> exception.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>>>> jburwell@basho.com
>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two features
>> are
>>>>>>>>>>>>>> mutually
>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>>>>>>>>>>>>> contains both
>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>>>> non-deterministic
>>>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
>> storage
>>>>>>>>>> oriented
>>>>>>>>>>>>>>>> solves the problem in short term. As I understand storage
>>>>>> tags,
>>>>>>>> we
>>>>>>>>>>>>>> have no
>>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>>>> intending that we implement this allocation model in 4.3,
>> but
>>>>>>>>>> instead,
>>>>>>>>>>>>>>>> determine if this type model would be one we would want to
>>>>>> support
>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the data
>>>>>> model
>>>>>>>>>> and
>>>>>>>>>>>>>>>> current implementation would not preclude evolution in that
>>>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>>>> view is that this type of allocation model is what user's
>>>>>> expect
>>>>>>>> of
>>>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>>>> systems -- selecting the best available resource set to
>>>>>> fulfill a
>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I believe we have meet your requirement here in that what
>> we've
>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>> should not make refinement difficult in the future. If we
>> don't
>>>>>>>>>> modify
>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>>>>>> simple
>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Looking through both patches, I have to say that the
>> aggregated
>>>>>>>> result
>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>>>>>> throttled
>>>>>>>>>> I/O and
>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
>>>>>> is
>>>>>>>> not
>>>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>>>>>> Javadoc
>>>>>>>>>> should
>>>>>>>>>>>>>> be added explaining the two sets of attributes and their
>> mutual
>>>>>>>>>> exclusion.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>>>>> exclusion.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>>>>>> provides
>>>>>>>> both
>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
>>>>>> uses
>>>>>>>> a
>>>>>>>>>>>>>> range. In order to select the best available resource to
>>>>>> fulfill a
>>>>>>>>>> QoS, we
>>>>>>>>>>>>>> would need to have the QoS expression normalized in terms of
>>>>>> units
>>>>>>>>>> (IOPS or
>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>>>>>>>> achieve a
>>>>>>>>>>>>>> model like I described, I think we would need to resolve this
>>>>>> issue
>>>>>>>>>> in 4.2
>>>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think we're not likely to be able to normalize the input for
>>>>>> 4.2.
>>>>>>>>>> Plus
>>>>>>>>>>>>> people probably want to input the data in terms they're
>> familiar
>>>>>> with
>>>>>>>>>> for
>>>>>>>>>>>>> the products in question.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let the
>>>>>> user
>>>>>>>>>> send
>>>>>>>>>>>>> key/value pairs to each vendor that could be selected due to a
>>>>>> given
>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it because
>> what
>>>>>>>>>> happens if
>>>>>>>>>>>>> you change the storage tag of a given Primary Storage after
>>>>>> having
>>>>>>>>>> created
>>>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>>>>>> re-think
>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>>>>>> supported
>>>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As I think through the implications of these requirements
>> and
>>>>>>>>>> reflect
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>>>>>> impacted
>>>>>>>>>> the
>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>>>> provisioned
>>>>>>>> IOPS
>>>>>>>>>>>>>> has no
>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
>>>>>> that
>>>>>>>> have
>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>>>>> understand
>>>>>>>>>> the
>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>>>>>> potentially
>>>>>>>>>>>>>> causing
>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>>>>>> to me
>>>>>>>>>>>>>> that a
>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>>>>> maximum
>>>>>>>>>> IOPS
>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> it can support and some type of overcommitment factor. This
>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>> should be used by the storage allocators to determine the
>>>>>> device
>>>>>>>>>> best
>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems that a
>>>>>>>> similar
>>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
>> layer
>>>>>>>> which
>>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path that
>>>>>> was
>>>>>>>>>>>>>> previously
>>>>>>>>>>>>>>> established for other properties you see on dialogs in CS:
>> Just
>>>>>>>>>> because
>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>>>>>>>> supports
>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>>>>> storage
>>>>>>>> tag
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> confines the deployment only to storage that supports the
>>>>>> selected
>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>>>> CloudStack
>>>>>>>>>> works
>>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I understand the tag functionality, and the need for the
>>>>>>>> administrator
>>>>>>>>>>>>>> to very carefully construct offerings. My concern is that we
>>>>>> over
>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>>>>>>>>> illustration,
>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
>> device
>>>>>> is
>>>>>>>>>> 100000.
>>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>>>> Therefore,
>>>>>>>> we
>>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>>>>>> guaranteed
>>>>>>>>>> IOPS
>>>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>>>>>>>>>> configuration
>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
>>>>>> As we
>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>>>>> guarantee
>>>>>>>>>> more IOPS
>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that make
>>>>>>>> sense?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not
>> even
>>>>>>>> sure
>>>>>>>>>> I
>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>>>>>> have
>>>>>>>> to
>>>>>>>>>> get
>>>>>>>>>>>>> all of the min values for all of the volumes on the SAN and
>> total
>>>>>>>> them
>>>>>>>>>> up,
>>>>>>>>>>>>> I suppose, and subtract it from the total (user facing)
>> supported
>>>>>>>> IOPS
>>>>>>>>>> of
>>>>>>>>>>>>> the system.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>>>>> resources
>>>>>>>>>> to
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>>>>>>> provisioned
>>>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>>>> think a side effect of the current implementation is
>> SolidFire
>>>>>>>>>> volumes
>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>>>>>>>> seems
>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>>>>>> with
>>>>>>>> QoS
>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>>>>>> starve
>>>>>>>>>> out
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>>>>>>>> collocated
>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
>>>>>> not
>>>>>>>>>>>>>> specify
>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>>>>>>>>> defaults
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN,
>> you
>>>>>> will
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>>>>> exclusions
>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't deploy
>> my
>>>>>>>>>> volume
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
>>>>>> would
>>>>>>>>>>>>>> complicate component assembly. It would require a storage
>>>>>> plugin to
>>>>>>>>>>>>>> anticipate all of the possible component assemblies and
>>>>>> determine
>>>>>>>> the
>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>>>>>>>>>> capabilities
>>>>>>>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion
>> or
>>>>>>>>>>>>> exclusion lists. It just needs to advertise its functionality
>> in
>>>>>> a
>>>>>>>> way
>>>>>>>>>> the
>>>>>>>>>>>>> allocator understands.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>>>>>>>> limits
>>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>>>>>> with
>>>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>>>>>>>> exist.
>>>>>>>>>>>>>> However, I am concerned with the more general case as we
>>>>>> supported
>>>>>>>>>> more
>>>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>>>>>> values
>>>>>>>> to
>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
>> provisioned
>>>>>>>>>> IOPS, but
>>>>>>>>>>>>> what if you break them out by read and write or do something
>>>>>> else? We
>>>>>>>>>> need
>>>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>>>>>> I/O
>>>>>>>> and
>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We also need to understand the implications of these QoS
>>>>>>>> guarantees
>>>>>>>>>> on
>>>>>>>>>>>>>>>> operation of the system to ensure that the underlying
>> hardware
>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>>>>>> forced
>>>>>>>> to
>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>>>> implementation
>>>>>>>> in
>>>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I agree, John. I also think you've come up with some great
>>>>>> ideas
>>>>>>>> for
>>>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>>>>>> storage
>>>>>>>>>>>>>> tags).
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>>>>>> addition to
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
>> being
>>>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>>>>>>>> hypervisor
>>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in values for
>>>>>> Wei's
>>>>>>>>>> and
>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>>>>>> yet), so
>>>>>>>> he
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators
>> to
>>>>>>>> prefer
>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands today
>>>>>> in
>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>> we already have this kind of an issue with other features
>>>>>> (fields
>>>>>>>>>> in
>>>>>>>>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
>>>>>> post
>>>>>>>>>> 4.2
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>>>>>> OpenStack
>>>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
>> until
>>>>>>>> code
>>>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>>>>>> fields
>>>>>>>>>>>>>> and my
>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
>>>>>> data
>>>>>>>>>>>>>> entered
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
>>>>>> design
>>>>>>>>>>>>>> that can
>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>>>>>> situations.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I just published my review @
>>>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
>>>>>> with a
>>>>>>>>>> new
>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>>>> checked
>>>>>>>>>> in
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>>>>>>>>> offline
>>>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>>>> this :)
>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>>>> areas
>>>>>>>> we
>>>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>>>>> afternoon,
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
>> in
>>>>>> Java
>>>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>>>> branch
>>>>>>>>>> into
>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>>> object_store
>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
>> the
>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
>> into
>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>> this
>>>>>>>> merge
>>>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>>>>> of
>>>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>>>>>>>>> patches
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>>>>>> dependency
>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>>>>> we
>>>>>>>> can
>>>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>>>> into
>>>>>>>> his
>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>>>> after his
>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>>>> code in
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>>>>>> plug-in,
>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>>>>>> (although
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>>>> actually
>>>>>>>> in
>>>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>>>>> the
>>>>>>>> url
>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>> stored
>>>>>> in
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>>>> hypervisor
>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>>>> place
>>>>>>>> of
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>>>>>> checks
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>>>>>> hypervisor
>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>>>> executes to
>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
>> to
>>>>>>>>>> detach a
>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>> data
>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>> IOPS
>>>>>> in
>>>>>>>> 4.2
>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
>> If
>>>>>> we
>>>>>>>>>> have
>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>>>> avoiding
>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>>>>>> field.
>>>>>>>>>> I
>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>>>> regarding
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>>>>> are
>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>>>> listStoragePool
>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>>>> useful
>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>>>> to be
>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>>> disk_offering_details
>>>>>>>>>> table.
>>>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>>>>> IOPS
>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>> the
>>>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>>>>>>>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Also, as far as I remember, we just introduced overcommit ratios (for CPU
and memory) when creating/editing clusters in 4.2, so we did survive as a
product before that (albeit great) feature was introduced.
On Fri, Jun 14, 2013 at 2:06 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> "the administrator/operator simply needs to tell us the total number of
> IOPS that can be committed to it and an overcommitment factor."
>
> Are you thinking when we create a plug-in as primary storage that we say -
> up front - how many IOPS the SAN can handle?
>
> That is not a good move, in my opinion. Our SAN is designed to start small
> and grow to PBs. As the need arises for more storage, the admin purchases
> additional storage nodes that join the cluster and the performance and
> capacity go up.
>
> We need to know how many IOPS total the SAN can handle and what it is
> committed to currently (the sum of the number of volumes' min IOPS).
>
> We also cannot assume the SAN is dedicated to CS.
>
>
> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com> wrote:
>
>> Simon,
>>
>> Yes, it is CloudStack's job to protect, as best it can, from
>> oversubscribing resources. I would argue that resource management is one,
>> if not the most, important functions of the system. It is no different
>> than the allocation/planning performed for hosts relative to cores and
>> memory. We can still oversubscribe resources, but we have rails + knobs
>> and dials to avoid it. Without these controls in place, we could easily
>> allow users to deploy workloads that overrun resources harming all tenants.
>>
>> I also think that we are over thinking this issue for provisioned IOPS.
>> When the DataStore is configured, the administrator/operator simply needs
>> to tell us the total number of IOPS that can be committed to it and an
>> overcommitment factor. As we allocate volumes to that DataStore, we sum up
>> the committed IOPS of the existing Volumes attached to the DataStore, apply
>> the overcommitment factor, and determine whether or not the requested
>> minimum IOPS for the new volume can be fulfilled. We can provide both
>> general and vendor specific documentation for determining these values --
>> be they to consume the entire device or a portion of it.
>>
>> Querying the device is unnecessary and deceptive. CloudStack resource
>> management is not interested in the current state of the device which could
>> be anywhere from extremely heavy to extremely light at any given time. We
>> are interested in the worst case load that is anticipated for resource. In
>> my view, it is up to administrators/operators to instrument their
>> environment to understand usage patterns and capacity. We should provide
>> information that will help determine what should be instrumented/monitored,
>> but that function should be performed outside of CloudStack.
>>
>> Thanks,
>> -John
>>
>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>>
>> > I'd like to comment on this briefly.
>> >
>> >
>> >
>> > I think an assumption is being made that the SAN is being dedicated to
>> a CS instance.
>> >
>> > My person opinion that this whole IOPS calculation is getting rather
>> complicated, and could probably be much simpler than this. Over
>> subscription is a fact of life on virtually all storage, and is really no
>> different in concept than multiple virt instances on a single piece of
>> hardware. All decent SANs offer many management options for the storage
>> engineers to keep track of IOPS utilization, and plan for spindle
>> augmentation as required.
>> > Is it really the job of CS to become yet another management layer on
>> top of this?
>> >
>> > ----- Original Message -----
>> >
>> > From: "Mike Tutkowski" <mi...@solidfire.com>
>> > To: dev@cloudstack.apache.org
>> > Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
>> ustcweizhou@gmail.com>
>> > Sent: Friday, June 14, 2013 1:00:26 PM
>> > Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >
>> > 1) We want number of IOPS currently supported by the SAN.
>> >
>> > 2) We want the number of IOPS that are committed (sum of min IOPS for
>> each
>> > volume).
>> >
>> > We could do the following to keep track of IOPS:
>> >
>> > The plug-in could have a timer thread that goes off every, say, 1
>> minute.
>> >
>> > It could query the SAN for the number of nodes that make up the SAN and
>> > multiple this by 50,000. This is essentially the number of supported
>> IOPS
>> > of the SAN.
>> >
>> > The next API call could be to get all of the volumes on the SAN. Iterate
>> > through them all and add up their min IOPS values. This is the number of
>> > IOPS the SAN is committed to.
>> >
>> > These two numbers can then be updated in the storage_pool table (a
>> column
>> > for each value).
>> >
>> > The allocators can get these values as needed (and they would be as
>> > accurate as the last time the thread asked the SAN for this info).
>> >
>> > These two fields, the min IOPS of the volume to create, and the
>> overcommit
>> > ratio of the plug-in would tell the allocator if it can select the given
>> > storage pool.
>> >
>> > What do you think?
>> >
>> >
>> > On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com> wrote:
>> >
>> >> "As I mentioned previously, I am very reluctant for any feature to come
>> >> into master that can exhaust resources."
>> >>
>> >> Just wanted to mention that, worst case, the SAN would fail creation of
>> >> the volume before allowing a new volume to break the system.
>> >>
>> >>
>> >> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com> wrote:
>> >>
>> >>> Hi John,
>> >>>
>> >>> Are you thinking we add a column on to the storage pool table,
>> >>> IOPS_Count, where we add and subtract committed IOPS?
>> >>>
>> >>> That is easy enough.
>> >>>
>> >>> How do you want to determine what the SAN is capable of supporting
>> IOPS
>> >>> wise? Remember we're dealing with a dynamic SAN here...as you add
>> storage
>> >>> nodes to the cluster, the number of IOPS increases. Do we have a
>> thread we
>> >>> can use to query external devices like this SAN to update the
>> supported
>> >>> number of IOPS?
>> >>>
>> >>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>> >>> overcommit ration to the plug-in when it's created? We would need to
>> store
>> >>> this in the storage_pool table, as well, I believe.
>> >>>
>> >>> We should also get Wei involved in this as his feature will need
>> similar
>> >>> functionality.
>> >>>
>> >>> Also, we should do this FAST as we have only two weeks left and many
>> of
>> >>> us will be out for several days for the CS Collab Conference.
>> >>>
>> >>> Thanks
>> >>>
>> >>>
>> >>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
>> >wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> Querying the SAN only indicates the number of IOPS currently in use.
>> >>>> The allocator needs to check the number of IOPS committed which is
>> tracked
>> >>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
>> >>>> committed to a DataStore, and determine whether or not the number
>> requested
>> >>>> can be fulfilled by that device. It seems to be that a
>> >>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>> >>>> DataStore's that don't support provisioned IOPS would return null.
>> >>>>
>> >>>> As I mentioned previously, I am very reluctant for any feature to
>> come
>> >>>> into master that can exhaust resources.
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>
>> >>>>> Yeah, I'm not sure I could come up with anything near an accurate
>> >>>>> assessment of how many IOPS are currently available on the SAN (or
>> >>>> even a
>> >>>>> total number that are available for volumes). Not sure if there's
>> yet
>> >>>> an
>> >>>>> API call for that.
>> >>>>>
>> >>>>> If I did know this number (total number of IOPS supported by the
>> SAN),
>> >>>> we'd
>> >>>>> still have to keep track of the total number of volumes we've
>> created
>> >>>> from
>> >>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an
>> API
>> >>>> call
>> >>>>> directly to the SAN to tweak the number of IOPS on a given volume or
>> >>>> set of
>> >>>>> volumes (not supported from CS, but supported via the SolidFire
>> API),
>> >>>> our
>> >>>>> numbers in CS would be off.
>> >>>>>
>> >>>>> I'm thinking verifying sufficient number of IOPS is a really good
>> idea
>> >>>> for
>> >>>>> a future release.
>> >>>>>
>> >>>>> For 4.2 I think we should stick to having the allocator detect if
>> >>>> storage
>> >>>>> QoS is desired and if the storage pool in question supports that
>> >>>> feature.
>> >>>>>
>> >>>>> If you really are over provisioned on your SAN in terms of IOPS or
>> >>>>> capacity, the SAN can let the admin know in several different ways
>> >>>> (e-mail,
>> >>>>> SNMP, GUI).
>> >>>>>
>> >>>>>
>> >>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> Please see my comments in-line below.
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> Comments below in red.
>> >>>>>>>
>> >>>>>>> Thanks
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jburwell@basho.com
>> >
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> Overall, I agree with the steps to below for 4.2. However, we may
>> >>>> want
>> >>>>>> to
>> >>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
>> >>>> user
>> >>>>>> is
>> >>>>>>>> expecting that the hypervisor will provide a particular QoS, and
>> >>>> that is
>> >>>>>>>> not possible, it seems like we should inform them rather than
>> >>>> silently
>> >>>>>>>> ignoring their request.
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> Sure, that sounds reasonable.
>> >>>>>>>
>> >>>>>>> We'd have to come up with some way for the allocators to know
>> about
>> >>>> the
>> >>>>>>> requested storage QoS and then query the candidate drivers.
>> >>>>>>>
>> >>>>>>> Any thoughts on how we might do that?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>> To collect my thoughts from previous parts of the thread, I am
>> >>>>>>>> uncomfortable with the idea that the management server can
>> >>>> overcommit a
>> >>>>>>>> resource. You had mentioned querying the device for available
>> IOPS.
>> >>>>>> While
>> >>>>>>>> that would be nice, it seems like we could fall back to a max
>> IOPS
>> >>>> and
>> >>>>>>>> overcommit factor manually calculated and entered by the
>> >>>>>>>> administrator/operator. I think such threshold and allocation
>> rails
>> >>>>>> should
>> >>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
>> >>>> basic
>> >>>>>>>> feature of any cloud orchestration platform.
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> Are you thinking this ability would make it into 4.2? Just curious
>> >>>> what
>> >>>>>>> release we're talking about here. For the SolidFire SAN, you might
>> >>>> have,
>> >>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
>> >>>> later
>> >>>>>> add
>> >>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a
>> way
>> >>>> to
>> >>>>>>> know that the number of supported IOPS has increased.
>> >>>>>>
>> >>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>> >>>> example,
>> >>>>>> we may only support expanding capacity in 4.2, and not handle any
>> >>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
>> >>>> a
>> >>>>>> bit, the throttled I/O enhancement seems like it needs a similar
>> set
>> >>>> of
>> >>>>>> rails defined per host.
>> >>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>> >>>>>>>> implementation specific manner. I think we need to arrive at a
>> >>>> general
>> >>>>>>>> model that can be exploited by the allocators and planners. We
>> >>>> should
>> >>>>>>>> restrict implementation specific key-value pairs (call them
>> details,
>> >>>>>>>> extended data, whatever) to information that is unique to the
>> >>>> driver and
>> >>>>>>>> would provide no useful information to the management server's
>> >>>>>>>> orchestration functions. A resource QoS does not seem to fit
>> those
>> >>>>>>>> criteria.
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
>> >>>> Collab
>> >>>>>>> Conf hack day that Joe just sent out an e-mail about?
>> >>>>>>
>> >>>>>> Yes, it would -- I will put something in the wiki topic. It will
>> >>>> also be
>> >>>>>> part of my talk on Monday -- How to Run from Zombie which include
>> >>>> some of
>> >>>>>> my opinions on the topic.
>> >>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> So, here's my suggestion for 4.2:
>> >>>>>>>>>
>> >>>>>>>>> Accept the values as they are currently required (four new
>> fields
>> >>>> for
>> >>>>>>>> Wei's
>> >>>>>>>>> feature or two new fields for mine).
>> >>>>>>>>>
>> >>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>> >>>>>>>>>
>> >>>>>>>>> 1) No QoS
>> >>>>>>>>>
>> >>>>>>>>> 2) Hypervisor QoS
>> >>>>>>>>>
>> >>>>>>>>> 3) Storage Qos
>> >>>>>>>>>
>> >>>>>>>>> The admin needs to specify storage tags that only map to storage
>> >>>> that
>> >>>>>>>>> supports Storage QoS.
>> >>>>>>>>>
>> >>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>> >>>>>>>> hypervisors
>> >>>>>>>>> in use support the new fields, they may not be enforced.
>> >>>>>>>>>
>> >>>>>>>>> Post 4.3:
>> >>>>>>>>>
>> >>>>>>>>> Come up with a way to more generally enter these parameters
>> >>>> (probably
>> >>>>>>>> just
>> >>>>>>>>> key/value pairs sent to the drivers).
>> >>>>>>>>>
>> >>>>>>>>> Have the drivers expose their feature set so the allocators can
>> >>>>>> consider
>> >>>>>>>>> them more fully and throw an exception if there is not a
>> sufficient
>> >>>>>>>> match.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put
>> some
>> >>>>>>>> burden
>> >>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
>> >>>> sense.
>> >>>>>> For
>> >>>>>>>>>> example, if he wants to use storage QoS with supported Min and
>> Max
>> >>>>>>>> values,
>> >>>>>>>>>> he'll have to put in a storage tag that references the
>> SolidFire
>> >>>>>> primary
>> >>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
>> then
>> >>>> he's
>> >>>>>>>> not
>> >>>>>>>>>> going to get the Min and Max feature. We could add help text to
>> >>>> the
>> >>>>>>>> pop-up
>> >>>>>>>>>> dialog that's displayed when you click in the Min and Max text
>> >>>> fields.
>> >>>>>>>>>>
>> >>>>>>>>>> Same idea for Wei's feature.
>> >>>>>>>>>>
>> >>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
>> comprehensive
>> >>>>>>>>>> approach for post 4.2.
>> >>>>>>>>>>
>> >>>>>>>>>> Maybe in the future we could have the drivers advertise their
>> >>>>>>>> capabilities
>> >>>>>>>>>> and if the allocator feels a request is not being satisfied
>> (say
>> >>>> Min
>> >>>>>> was
>> >>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
>> >>>> throw
>> >>>>>> an
>> >>>>>>>>>> exception.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Comments below in red.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>> >>>> jburwell@basho.com>
>> >>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Please see my comment in-line below.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>> -John
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I've put comments below in red.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>> >>>> jburwell@basho.com
>> >>>>>>>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> First and foremost, we must ensure that these two features
>> are
>> >>>>>>>>>>>> mutually
>> >>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration
>> that
>> >>>>>>>>>>>> contains both
>> >>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>> >>>>>>>> non-deterministic
>> >>>>>>>>>>>>>> operations.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Agreed
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
>> storage
>> >>>>>>>> oriented
>> >>>>>>>>>>>>>> solves the problem in short term. As I understand storage
>> >>>> tags,
>> >>>>>> we
>> >>>>>>>>>>>> have no
>> >>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>> >>>>>>>>>>>> necessarily
>> >>>>>>>>>>>>>> intending that we implement this allocation model in 4.3,
>> but
>> >>>>>>>> instead,
>> >>>>>>>>>>>>>> determine if this type model would be one we would want to
>> >>>> support
>> >>>>>>>> in
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the data
>> >>>> model
>> >>>>>>>> and
>> >>>>>>>>>>>>>> current implementation would not preclude evolution in that
>> >>>>>>>>>>>> direction. My
>> >>>>>>>>>>>>>> view is that this type of allocation model is what user's
>> >>>> expect
>> >>>>>> of
>> >>>>>>>>>>>> "cloud"
>> >>>>>>>>>>>>>> systems -- selecting the best available resource set to
>> >>>> fulfill a
>> >>>>>>>>>>>> set of
>> >>>>>>>>>>>>>> system requirements.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I believe we have meet your requirement here in that what
>> we've
>> >>>>>>>>>>>> implemented
>> >>>>>>>>>>>>> should not make refinement difficult in the future. If we
>> don't
>> >>>>>>>> modify
>> >>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>> >>>> simple
>> >>>>>>>>>>>> changes
>> >>>>>>>>>>>>> to enhance the current functioning of the system.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Looking through both patches, I have to say that the
>> aggregated
>> >>>>>> result
>> >>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>> >>>> throttled
>> >>>>>>>> I/O and
>> >>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
>> >>>> is
>> >>>>>> not
>> >>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>> >>>> Javadoc
>> >>>>>>>> should
>> >>>>>>>>>>>> be added explaining the two sets of attributes and their
>> mutual
>> >>>>>>>> exclusion.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>> >>>> exclusion.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> The other part that is interesting is that throttled I/O
>> >>>> provides
>> >>>>>> both
>> >>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned
>> IOPS
>> >>>> uses
>> >>>>>> a
>> >>>>>>>>>>>> range. In order to select the best available resource to
>> >>>> fulfill a
>> >>>>>>>> QoS, we
>> >>>>>>>>>>>> would need to have the QoS expression normalized in terms of
>> >>>> units
>> >>>>>>>> (IOPS or
>> >>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>> >>>>>> achieve a
>> >>>>>>>>>>>> model like I described, I think we would need to resolve this
>> >>>> issue
>> >>>>>>>> in 4.2
>> >>>>>>>>>>>> to ensure a viable migration path.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> I think we're not likely to be able to normalize the input for
>> >>>> 4.2.
>> >>>>>>>> Plus
>> >>>>>>>>>>> people probably want to input the data in terms they're
>> familiar
>> >>>> with
>> >>>>>>>> for
>> >>>>>>>>>>> the products in question.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Ideally we would fix the way we do storage tagging and let the
>> >>>> user
>> >>>>>>>> send
>> >>>>>>>>>>> key/value pairs to each vendor that could be selected due to a
>> >>>> given
>> >>>>>>>>>>> storage tag. I'm still not sure that would solve it because
>> what
>> >>>>>>>> happens if
>> >>>>>>>>>>> you change the storage tag of a given Primary Storage after
>> >>>> having
>> >>>>>>>> created
>> >>>>>>>>>>> a Disk Offering?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>> >>>> re-think
>> >>>>>> it.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>> >>>> supported
>> >>>>>>>>>>> feature sets so the allocators can make good choices.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> As I think through the implications of these requirements
>> and
>> >>>>>>>> reflect
>> >>>>>>>>>>>> on
>> >>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>> >>>> impacted
>> >>>>>>>> the
>> >>>>>>>>>>>>>> allocators and planners. As it stands, the current
>> >>>> provisioned
>> >>>>>> IOPS
>> >>>>>>>>>>>> has no
>> >>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
>> >>>> that
>> >>>>>> have
>> >>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>> >>>> understand
>> >>>>>>>> the
>> >>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>> >>>> potentially
>> >>>>>>>>>>>> causing
>> >>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>> >>>> to me
>> >>>>>>>>>>>> that a
>> >>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>> >>>> maximum
>> >>>>>>>> IOPS
>> >>>>>>>>>>>> which
>> >>>>>>>>>>>>>> it can support and some type of overcommitment factor. This
>> >>>>>>>>>>>> information
>> >>>>>>>>>>>>>> should be used by the storage allocators to determine the
>> >>>> device
>> >>>>>>>> best
>> >>>>>>>>>>>> able
>> >>>>>>>>>>>>>> to support the resources needs of a volume. It seems that a
>> >>>>>> similar
>> >>>>>>>>>>>> set of
>> >>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
>> layer
>> >>>>>> which
>> >>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Yeah, for this first release, we just followed the path that
>> >>>> was
>> >>>>>>>>>>>> previously
>> >>>>>>>>>>>>> established for other properties you see on dialogs in CS:
>> Just
>> >>>>>>>> because
>> >>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>> >>>>>> supports
>> >>>>>>>>>>>> them.
>> >>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>> >>>> storage
>> >>>>>> tag
>> >>>>>>>>>>>> that
>> >>>>>>>>>>>>> confines the deployment only to storage that supports the
>> >>>> selected
>> >>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>> >>>> CloudStack
>> >>>>>>>> works
>> >>>>>>>>>>>>> today.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I understand the tag functionality, and the need for the
>> >>>>>> administrator
>> >>>>>>>>>>>> to very carefully construct offerings. My concern is that we
>> >>>> over
>> >>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>> >>>>>>>> illustration,
>> >>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
>> device
>> >>>> is
>> >>>>>>>> 100000.
>> >>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>> >>>> Therefore,
>> >>>>>> we
>> >>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>> >>>> guaranteed
>> >>>>>>>> IOPS
>> >>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>> >>>>>>>> configuration
>> >>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
>> >>>> As we
>> >>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>> >>>> guarantee
>> >>>>>>>> more IOPS
>> >>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that make
>> >>>>>> sense?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not
>> even
>> >>>>>> sure
>> >>>>>>>> I
>> >>>>>>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>> >>>> have
>> >>>>>> to
>> >>>>>>>> get
>> >>>>>>>>>>> all of the min values for all of the volumes on the SAN and
>> total
>> >>>>>> them
>> >>>>>>>> up,
>> >>>>>>>>>>> I suppose, and subtract it from the total (user facing)
>> supported
>> >>>>>> IOPS
>> >>>>>>>> of
>> >>>>>>>>>>> the system.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>> >>>> resources
>> >>>>>>>> to
>> >>>>>>>>>>>> be
>> >>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>> >>>>>> provisioned
>> >>>>>>>>>>>> IOPS, I
>> >>>>>>>>>>>>>> think a side effect of the current implementation is
>> SolidFire
>> >>>>>>>> volumes
>> >>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O,
>> it
>> >>>>>> seems
>> >>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>> >>>> with
>> >>>>>> QoS
>> >>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>> >>>> starve
>> >>>>>>>> out
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>> >>>>>> collocated
>> >>>>>>>>>>>> VMs
>> >>>>>>>>>>>>>> from being fulfilled.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS)
>> and
>> >>>> not
>> >>>>>>>>>>>> specify
>> >>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at
>> the
>> >>>>>>>> defaults
>> >>>>>>>>>>>> we
>> >>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN,
>> you
>> >>>> will
>> >>>>>>>>>>>> have
>> >>>>>>>>>>>>> your Mins met.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>> >>>> exclusions
>> >>>>>>>>>>>> list,
>> >>>>>>>>>>>>> which might be nice to have at some point (i.e. don't
>> deploy my
>> >>>>>>>> volume
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>>> storage with these following tags).
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
>> >>>> would
>> >>>>>>>>>>>> complicate component assembly. It would require a storage
>> >>>> plugin to
>> >>>>>>>>>>>> anticipate all of the possible component assemblies and
>> >>>> determine
>> >>>>>> the
>> >>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>> >>>>>>>> capabilities
>> >>>>>>>>>>>> which can be matched to a set of requested requirements.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion
>> or
>> >>>>>>>>>>> exclusion lists. It just needs to advertise its functionality
>> in
>> >>>> a
>> >>>>>> way
>> >>>>>>>> the
>> >>>>>>>>>>> allocator understands.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it
>> only
>> >>>>>> limits
>> >>>>>>>>>>>> IOPS,
>> >>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>> >>>> with
>> >>>>>>>>>>>> guaranteed
>> >>>>>>>>>>>>> minimum IOPS does this.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does
>> not
>> >>>>>> exist.
>> >>>>>>>>>>>> However, I am concerned with the more general case as we
>> >>>> supported
>> >>>>>>>> more
>> >>>>>>>>>>>> devices with provisioned IOPS.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>> >>>> values
>> >>>>>> to
>> >>>>>>>>>>> drivers. Min and Max and pretty industry standard for
>> provisioned
>> >>>>>>>> IOPS, but
>> >>>>>>>>>>> what if you break them out by read and write or do something
>> >>>> else? We
>> >>>>>>>> need
>> >>>>>>>>>>> a more general mechanism.
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>> >>>> I/O
>> >>>>>> and
>> >>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Agreed
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> We also need to understand the implications of these QoS
>> >>>>>> guarantees
>> >>>>>>>> on
>> >>>>>>>>>>>>>> operation of the system to ensure that the underlying
>> hardware
>> >>>>>>>>>>>> resources
>> >>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>> >>>> forced
>> >>>>>> to
>> >>>>>>>>>>>> make
>> >>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>> >>>> implementation
>> >>>>>> in
>> >>>>>>>>>>>> 4.3.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I agree, John. I also think you've come up with some great
>> >>>> ideas
>> >>>>>> for
>> >>>>>>>>>>>> 4.3. :)
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>> >>>> storage
>> >>>>>>>>>>>> tags).
>> >>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>> >>>> addition to
>> >>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
>> being
>> >>>>>>>>>>>> preferred,
>> >>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>> >>>>>> hypervisor
>> >>>>>>>>>>>> QoS
>> >>>>>>>>>>>>>> is).
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I think John's concern is that you can enter in values for
>> >>>> Wei's
>> >>>>>>>> and
>> >>>>>>>>>>>> my
>> >>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>> >>>> yet), so
>> >>>>>> he
>> >>>>>>>>>>>> was
>> >>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators
>> to
>> >>>>>> prefer
>> >>>>>>>>>>>> these
>> >>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands
>> today
>> >>>> in
>> >>>>>>>>>>>>>> CloudStack,
>> >>>>>>>>>>>>>>> we already have this kind of an issue with other features
>> >>>> (fields
>> >>>>>>>> in
>> >>>>>>>>>>>>>>> dialogs for features that not all vendors support).
>> Perhaps
>> >>>> post
>> >>>>>>>> 4.2
>> >>>>>>>>>>>> we
>> >>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>> >>>> OpenStack
>> >>>>>>>>>>>> addresses
>> >>>>>>>>>>>>>>> the issue).
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
>> until
>> >>>>>> code
>> >>>>>>>>>>>>>> freeze)
>> >>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>> >>>> fields
>> >>>>>>>>>>>> and my
>> >>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
>> >>>> data
>> >>>>>>>>>>>> entered
>> >>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
>> >>>> design
>> >>>>>>>>>>>> that can
>> >>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>> >>>> situations.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> What do you think?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>> >>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I just published my review @
>> >>>>>> https://reviews.apache.org/r/11479/.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I apologize for the delay,
>> >>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> No problem, John.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
>> >>>> with a
>> >>>>>>>> new
>> >>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>> file.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>> >>>> checked
>> >>>>>>>> in
>> >>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>> merge those into mine before generating a new patch
>> file?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> >>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I
>> am
>> >>>>>>>> offline
>> >>>>>>>>>>>> ATM,
>> >>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Sorry for the oversight,
>> >>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> >>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>> >>>> this :)
>> >>>>>>>> ),
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>> >>>> areas
>> >>>>>> we
>> >>>>>>>>>>>> wanted
>> >>>>>>>>>>>>>>>>>>> addressed.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>> >>>>>> afternoon,
>> >>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>> send
>> >>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>> another patch.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> >>>>>>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Hi Mike,
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
>> in
>> >>>> Java
>> >>>>>>>>>>>> code,
>> >>>>>>>>>>>>>>>>>> except
>> >>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>> >>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>> >>>> branch
>> >>>>>>>> into
>> >>>>>>>>>>>>>>>> master
>> >>>>>>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Hey John,
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>> >>>> object_store
>> >>>>>>>>>>>> branch,
>> >>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>> -
>> >>>>>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
>> the
>> >>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>> branch
>> >>>>>>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
>> into
>> >>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
>> this
>> >>>>>> merge
>> >>>>>>>>>>>>>>>> strategy.
>> >>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> >>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>> >>>>>>>>>>>>>> disk_io_throttling,
>> >>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a
>> number
>> >>>> of
>> >>>>>>>>>>>>>> overlapping
>> >>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between
>> the
>> >>>>>>>> patches
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>>> follows:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>> >>>>>>>> SolidFire
>> >>>>>>>>>>>> are
>> >>>>>>>>>>>>>>>>>>>>> additive
>> >>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>> >>>>>> dependency
>> >>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>> >>>> we
>> >>>>>> can
>> >>>>>>>>>>>>>>>> determine
>> >>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>> >>>> into
>> >>>>>> his
>> >>>>>>>>>>>> branch
>> >>>>>>>>>>>>>>>>>>>>> before
>> >>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>> >>>> after his
>> >>>>>>>>>>>> merge
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going
>> here:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>> >>>> code in
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>> >>>>>> plug-in,
>> >>>>>>>>>>>> which
>> >>>>>>>>>>>>>>>>>>>>>>> currently
>> >>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>> >>>>>> (although
>> >>>>>>>>>>>> it
>> >>>>>>>>>>>>>> does
>> >>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>> know
>> >>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>> >>>> actually
>> >>>>>> in
>> >>>>>>>>>>>> use))
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed
>> in
>> >>>> the
>> >>>>>> url
>> >>>>>>>>>>>> field
>> >>>>>>>>>>>>>>>> (if
>> >>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
>> stored
>> >>>> in
>> >>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>> >>>> hypervisor
>> >>>>>> in
>> >>>>>>>>>>>>>>>>>>>>> question, we
>> >>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>> >>>> place
>> >>>>>> of
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
>> hypervisor
>> >>>>>> checks
>> >>>>>>>>>>>> for
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>> managed
>> >>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>> >>>>>> hypervisor
>> >>>>>>>>>>>> data
>> >>>>>>>>>>>>>>>>>>>>>>> structure is
>> >>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>> >>>> executes to
>> >>>>>>>>>>>> attach
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>> volume.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
>> to
>> >>>>>>>> detach a
>> >>>>>>>>>>>>>>>> volume,
>> >>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
>> data
>> >>>>>>>>>>>> structure
>> >>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>> removed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
>> IOPS
>> >>>> in
>> >>>>>> 4.2
>> >>>>>>>>>>>>>> unless
>> >>>>>>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
>> If
>> >>>> we
>> >>>>>>>> have
>> >>>>>>>>>>>> some
>> >>>>>>>>>>>>>>>>>>>>> idea,
>> >>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski
>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>> >>>> avoiding
>> >>>>>>>>>>>>>>>> implementation
>> >>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>> >>>>>> field.
>> >>>>>>>> I
>> >>>>>>>>>>>>>> think
>> >>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>> >>>> regarding
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> expected
>> >>>>>>>>>>>>>>>>>>>>>>> format
>> >>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the
>> details
>> >>>> are
>> >>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>> returned on
>> >>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>> >>>>>> listStoragePool
>> >>>>>>>>>>>>>>>> response.
>> >>>>>>>>>>>>>>>>>>>>>> Why
>> >>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>> >>>> useful
>> >>>>>>>> for
>> >>>>>>>>>>>>>>>> clients
>> >>>>>>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>> >>>> field."
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>> >>>> here.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> >>>>>>>> Offering-by-Disk
>> >>>>>>>>>>>>>>>> Offering
>> >>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>> >>>> to be
>> >>>>>>>>>>>> able to
>> >>>>>>>>>>>>>>>>>>>>>> associate
>> >>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>> >>>> disk_offering_details
>> >>>>>>>> table.
>> >>>>>>>>>>>>>> Maybe
>> >>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>> could
>> >>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the
>> Burst
>> >>>> IOPS
>> >>>>>> in
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>> GUI
>> >>>>>>>>>>>>>>>>>>>>> if
>> >>>>>>>>>>>>>>>>>>>>>>> it's
>> >>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
>> the
>> >>>> DB.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>> >>>>>
>> >>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >> *â„¢*
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>> >
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"the administrator/operator simply needs to tell us the total number of
IOPS that can be committed to it and an overcommitment factor."
Are you thinking when we create a plug-in as primary storage that we say -
up front - how many IOPS the SAN can handle?
That is not a good move, in my opinion. Our SAN is designed to start small
and grow to PBs. As the need arises for more storage, the admin purchases
additional storage nodes that join the cluster and the performance and
capacity go up.
We need to know how many IOPS total the SAN can handle and what it is
committed to currently (the sum of the number of volumes' min IOPS).
We also cannot assume the SAN is dedicated to CS.
On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <jb...@basho.com> wrote:
> Simon,
>
> Yes, it is CloudStack's job to protect, as best it can, from
> oversubscribing resources. I would argue that resource management is one,
> if not the most, important functions of the system. It is no different
> than the allocation/planning performed for hosts relative to cores and
> memory. We can still oversubscribe resources, but we have rails + knobs
> and dials to avoid it. Without these controls in place, we could easily
> allow users to deploy workloads that overrun resources harming all tenants.
>
> I also think that we are over thinking this issue for provisioned IOPS.
> When the DataStore is configured, the administrator/operator simply needs
> to tell us the total number of IOPS that can be committed to it and an
> overcommitment factor. As we allocate volumes to that DataStore, we sum up
> the committed IOPS of the existing Volumes attached to the DataStore, apply
> the overcommitment factor, and determine whether or not the requested
> minimum IOPS for the new volume can be fulfilled. We can provide both
> general and vendor specific documentation for determining these values --
> be they to consume the entire device or a portion of it.
>
> Querying the device is unnecessary and deceptive. CloudStack resource
> management is not interested in the current state of the device which could
> be anywhere from extremely heavy to extremely light at any given time. We
> are interested in the worst case load that is anticipated for resource. In
> my view, it is up to administrators/operators to instrument their
> environment to understand usage patterns and capacity. We should provide
> information that will help determine what should be instrumented/monitored,
> but that function should be performed outside of CloudStack.
>
> Thanks,
> -John
>
> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>
> > I'd like to comment on this briefly.
> >
> >
> >
> > I think an assumption is being made that the SAN is being dedicated to a
> CS instance.
> >
> > My person opinion that this whole IOPS calculation is getting rather
> complicated, and could probably be much simpler than this. Over
> subscription is a fact of life on virtually all storage, and is really no
> different in concept than multiple virt instances on a single piece of
> hardware. All decent SANs offer many management options for the storage
> engineers to keep track of IOPS utilization, and plan for spindle
> augmentation as required.
> > Is it really the job of CS to become yet another management layer on top
> of this?
> >
> > ----- Original Message -----
> >
> > From: "Mike Tutkowski" <mi...@solidfire.com>
> > To: dev@cloudstack.apache.org
> > Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <
> ustcweizhou@gmail.com>
> > Sent: Friday, June 14, 2013 1:00:26 PM
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > 1) We want number of IOPS currently supported by the SAN.
> >
> > 2) We want the number of IOPS that are committed (sum of min IOPS for
> each
> > volume).
> >
> > We could do the following to keep track of IOPS:
> >
> > The plug-in could have a timer thread that goes off every, say, 1 minute.
> >
> > It could query the SAN for the number of nodes that make up the SAN and
> > multiple this by 50,000. This is essentially the number of supported IOPS
> > of the SAN.
> >
> > The next API call could be to get all of the volumes on the SAN. Iterate
> > through them all and add up their min IOPS values. This is the number of
> > IOPS the SAN is committed to.
> >
> > These two numbers can then be updated in the storage_pool table (a column
> > for each value).
> >
> > The allocators can get these values as needed (and they would be as
> > accurate as the last time the thread asked the SAN for this info).
> >
> > These two fields, the min IOPS of the volume to create, and the
> overcommit
> > ratio of the plug-in would tell the allocator if it can select the given
> > storage pool.
> >
> > What do you think?
> >
> >
> > On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> "As I mentioned previously, I am very reluctant for any feature to come
> >> into master that can exhaust resources."
> >>
> >> Just wanted to mention that, worst case, the SAN would fail creation of
> >> the volume before allowing a new volume to break the system.
> >>
> >>
> >> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> Hi John,
> >>>
> >>> Are you thinking we add a column on to the storage pool table,
> >>> IOPS_Count, where we add and subtract committed IOPS?
> >>>
> >>> That is easy enough.
> >>>
> >>> How do you want to determine what the SAN is capable of supporting IOPS
> >>> wise? Remember we're dealing with a dynamic SAN here...as you add
> storage
> >>> nodes to the cluster, the number of IOPS increases. Do we have a
> thread we
> >>> can use to query external devices like this SAN to update the supported
> >>> number of IOPS?
> >>>
> >>> Also, how do you want to enforce the IOPS limit? Do we pass in an
> >>> overcommit ration to the plug-in when it's created? We would need to
> store
> >>> this in the storage_pool table, as well, I believe.
> >>>
> >>> We should also get Wei involved in this as his feature will need
> similar
> >>> functionality.
> >>>
> >>> Also, we should do this FAST as we have only two weeks left and many of
> >>> us will be out for several days for the CS Collab Conference.
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jburwell@basho.com
> >wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Querying the SAN only indicates the number of IOPS currently in use.
> >>>> The allocator needs to check the number of IOPS committed which is
> tracked
> >>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
> >>>> committed to a DataStore, and determine whether or not the number
> requested
> >>>> can be fulfilled by that device. It seems to be that a
> >>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
> >>>> DataStore's that don't support provisioned IOPS would return null.
> >>>>
> >>>> As I mentioned previously, I am very reluctant for any feature to come
> >>>> into master that can exhaust resources.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> Yeah, I'm not sure I could come up with anything near an accurate
> >>>>> assessment of how many IOPS are currently available on the SAN (or
> >>>> even a
> >>>>> total number that are available for volumes). Not sure if there's yet
> >>>> an
> >>>>> API call for that.
> >>>>>
> >>>>> If I did know this number (total number of IOPS supported by the
> SAN),
> >>>> we'd
> >>>>> still have to keep track of the total number of volumes we've created
> >>>> from
> >>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an API
> >>>> call
> >>>>> directly to the SAN to tweak the number of IOPS on a given volume or
> >>>> set of
> >>>>> volumes (not supported from CS, but supported via the SolidFire API),
> >>>> our
> >>>>> numbers in CS would be off.
> >>>>>
> >>>>> I'm thinking verifying sufficient number of IOPS is a really good
> idea
> >>>> for
> >>>>> a future release.
> >>>>>
> >>>>> For 4.2 I think we should stick to having the allocator detect if
> >>>> storage
> >>>>> QoS is desired and if the storage pool in question supports that
> >>>> feature.
> >>>>>
> >>>>> If you really are over provisioned on your SAN in terms of IOPS or
> >>>>> capacity, the SAN can let the admin know in several different ways
> >>>> (e-mail,
> >>>>> SNMP, GUI).
> >>>>>
> >>>>>
> >>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Please see my comments in-line below.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Comments below in red.
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> Overall, I agree with the steps to below for 4.2. However, we may
> >>>> want
> >>>>>> to
> >>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
> >>>> user
> >>>>>> is
> >>>>>>>> expecting that the hypervisor will provide a particular QoS, and
> >>>> that is
> >>>>>>>> not possible, it seems like we should inform them rather than
> >>>> silently
> >>>>>>>> ignoring their request.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Sure, that sounds reasonable.
> >>>>>>>
> >>>>>>> We'd have to come up with some way for the allocators to know about
> >>>> the
> >>>>>>> requested storage QoS and then query the candidate drivers.
> >>>>>>>
> >>>>>>> Any thoughts on how we might do that?
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> To collect my thoughts from previous parts of the thread, I am
> >>>>>>>> uncomfortable with the idea that the management server can
> >>>> overcommit a
> >>>>>>>> resource. You had mentioned querying the device for available
> IOPS.
> >>>>>> While
> >>>>>>>> that would be nice, it seems like we could fall back to a max IOPS
> >>>> and
> >>>>>>>> overcommit factor manually calculated and entered by the
> >>>>>>>> administrator/operator. I think such threshold and allocation
> rails
> >>>>>> should
> >>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
> >>>> basic
> >>>>>>>> feature of any cloud orchestration platform.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Are you thinking this ability would make it into 4.2? Just curious
> >>>> what
> >>>>>>> release we're talking about here. For the SolidFire SAN, you might
> >>>> have,
> >>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
> >>>> later
> >>>>>> add
> >>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a
> way
> >>>> to
> >>>>>>> know that the number of supported IOPS has increased.
> >>>>>>
> >>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
> >>>> example,
> >>>>>> we may only support expanding capacity in 4.2, and not handle any
> >>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
> >>>> a
> >>>>>> bit, the throttled I/O enhancement seems like it needs a similar set
> >>>> of
> >>>>>> rails defined per host.
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in a
> >>>>>>>> implementation specific manner. I think we need to arrive at a
> >>>> general
> >>>>>>>> model that can be exploited by the allocators and planners. We
> >>>> should
> >>>>>>>> restrict implementation specific key-value pairs (call them
> details,
> >>>>>>>> extended data, whatever) to information that is unique to the
> >>>> driver and
> >>>>>>>> would provide no useful information to the management server's
> >>>>>>>> orchestration functions. A resource QoS does not seem to fit those
> >>>>>>>> criteria.
> >>>>>>>>
> >>>>>>>
> >>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
> >>>> Collab
> >>>>>>> Conf hack day that Joe just sent out an e-mail about?
> >>>>>>
> >>>>>> Yes, it would -- I will put something in the wiki topic. It will
> >>>> also be
> >>>>>> part of my talk on Monday -- How to Run from Zombie which include
> >>>> some of
> >>>>>> my opinions on the topic.
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> So, here's my suggestion for 4.2:
> >>>>>>>>>
> >>>>>>>>> Accept the values as they are currently required (four new fields
> >>>> for
> >>>>>>>> Wei's
> >>>>>>>>> feature or two new fields for mine).
> >>>>>>>>>
> >>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
> >>>>>>>>>
> >>>>>>>>> 1) No QoS
> >>>>>>>>>
> >>>>>>>>> 2) Hypervisor QoS
> >>>>>>>>>
> >>>>>>>>> 3) Storage Qos
> >>>>>>>>>
> >>>>>>>>> The admin needs to specify storage tags that only map to storage
> >>>> that
> >>>>>>>>> supports Storage QoS.
> >>>>>>>>>
> >>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
> >>>>>>>> hypervisors
> >>>>>>>>> in use support the new fields, they may not be enforced.
> >>>>>>>>>
> >>>>>>>>> Post 4.3:
> >>>>>>>>>
> >>>>>>>>> Come up with a way to more generally enter these parameters
> >>>> (probably
> >>>>>>>> just
> >>>>>>>>> key/value pairs sent to the drivers).
> >>>>>>>>>
> >>>>>>>>> Have the drivers expose their feature set so the allocators can
> >>>>>> consider
> >>>>>>>>> them more fully and throw an exception if there is not a
> sufficient
> >>>>>>>> match.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put
> some
> >>>>>>>> burden
> >>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
> >>>> sense.
> >>>>>> For
> >>>>>>>>>> example, if he wants to use storage QoS with supported Min and
> Max
> >>>>>>>> values,
> >>>>>>>>>> he'll have to put in a storage tag that references the SolidFire
> >>>>>> primary
> >>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't,
> then
> >>>> he's
> >>>>>>>> not
> >>>>>>>>>> going to get the Min and Max feature. We could add help text to
> >>>> the
> >>>>>>>> pop-up
> >>>>>>>>>> dialog that's displayed when you click in the Min and Max text
> >>>> fields.
> >>>>>>>>>>
> >>>>>>>>>> Same idea for Wei's feature.
> >>>>>>>>>>
> >>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
> comprehensive
> >>>>>>>>>> approach for post 4.2.
> >>>>>>>>>>
> >>>>>>>>>> Maybe in the future we could have the drivers advertise their
> >>>>>>>> capabilities
> >>>>>>>>>> and if the allocator feels a request is not being satisfied (say
> >>>> Min
> >>>>>> was
> >>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
> >>>> throw
> >>>>>> an
> >>>>>>>>>> exception.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>>> jburwell@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please see my comment in-line below.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I've put comments below in red.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>>> jburwell@basho.com
> >>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> First and foremost, we must ensure that these two features
> are
> >>>>>>>>>>>> mutually
> >>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
> >>>>>>>>>>>> contains both
> >>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >>>>>>>> non-deterministic
> >>>>>>>>>>>>>> operations.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or
> storage
> >>>>>>>> oriented
> >>>>>>>>>>>>>> solves the problem in short term. As I understand storage
> >>>> tags,
> >>>>>> we
> >>>>>>>>>>>> have no
> >>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
> >>>>>>>>>>>> necessarily
> >>>>>>>>>>>>>> intending that we implement this allocation model in 4.3,
> but
> >>>>>>>> instead,
> >>>>>>>>>>>>>> determine if this type model would be one we would want to
> >>>> support
> >>>>>>>> in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the data
> >>>> model
> >>>>>>>> and
> >>>>>>>>>>>>>> current implementation would not preclude evolution in that
> >>>>>>>>>>>> direction. My
> >>>>>>>>>>>>>> view is that this type of allocation model is what user's
> >>>> expect
> >>>>>> of
> >>>>>>>>>>>> "cloud"
> >>>>>>>>>>>>>> systems -- selecting the best available resource set to
> >>>> fulfill a
> >>>>>>>>>>>> set of
> >>>>>>>>>>>>>> system requirements.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I believe we have meet your requirement here in that what
> we've
> >>>>>>>>>>>> implemented
> >>>>>>>>>>>>> should not make refinement difficult in the future. If we
> don't
> >>>>>>>> modify
> >>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
> >>>> simple
> >>>>>>>>>>>> changes
> >>>>>>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Looking through both patches, I have to say that the
> aggregated
> >>>>>> result
> >>>>>>>>>>>> seems a bit confusing. There are six new attributes for
> >>>> throttled
> >>>>>>>> I/O and
> >>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
> >>>> is
> >>>>>> not
> >>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
> >>>> Javadoc
> >>>>>>>> should
> >>>>>>>>>>>> be added explaining the two sets of attributes and their
> mutual
> >>>>>>>> exclusion.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
> >>>> exclusion.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> The other part that is interesting is that throttled I/O
> >>>> provides
> >>>>>> both
> >>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
> >>>> uses
> >>>>>> a
> >>>>>>>>>>>> range. In order to select the best available resource to
> >>>> fulfill a
> >>>>>>>> QoS, we
> >>>>>>>>>>>> would need to have the QoS expression normalized in terms of
> >>>> units
> >>>>>>>> (IOPS or
> >>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
> >>>>>> achieve a
> >>>>>>>>>>>> model like I described, I think we would need to resolve this
> >>>> issue
> >>>>>>>> in 4.2
> >>>>>>>>>>>> to ensure a viable migration path.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I think we're not likely to be able to normalize the input for
> >>>> 4.2.
> >>>>>>>> Plus
> >>>>>>>>>>> people probably want to input the data in terms they're
> familiar
> >>>> with
> >>>>>>>> for
> >>>>>>>>>>> the products in question.
> >>>>>>>>>>>
> >>>>>>>>>>> Ideally we would fix the way we do storage tagging and let the
> >>>> user
> >>>>>>>> send
> >>>>>>>>>>> key/value pairs to each vendor that could be selected due to a
> >>>> given
> >>>>>>>>>>> storage tag. I'm still not sure that would solve it because
> what
> >>>>>>>> happens if
> >>>>>>>>>>> you change the storage tag of a given Primary Storage after
> >>>> having
> >>>>>>>> created
> >>>>>>>>>>> a Disk Offering?
> >>>>>>>>>>>
> >>>>>>>>>>> Basically storage tagging is kind of a mess and we should
> >>>> re-think
> >>>>>> it.
> >>>>>>>>>>>
> >>>>>>>>>>> Also, we need to have a way for the drivers to expose their
> >>>> supported
> >>>>>>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As I think through the implications of these requirements
> and
> >>>>>>>> reflect
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
> >>>> impacted
> >>>>>>>> the
> >>>>>>>>>>>>>> allocators and planners. As it stands, the current
> >>>> provisioned
> >>>>>> IOPS
> >>>>>>>>>>>> has no
> >>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
> >>>> that
> >>>>>> have
> >>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
> >>>> understand
> >>>>>>>> the
> >>>>>>>>>>>>>> current patch, we can overcommit storage resources --
> >>>> potentially
> >>>>>>>>>>>> causing
> >>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
> >>>> to me
> >>>>>>>>>>>> that a
> >>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
> >>>> maximum
> >>>>>>>> IOPS
> >>>>>>>>>>>> which
> >>>>>>>>>>>>>> it can support and some type of overcommitment factor. This
> >>>>>>>>>>>> information
> >>>>>>>>>>>>>> should be used by the storage allocators to determine the
> >>>> device
> >>>>>>>> best
> >>>>>>>>>>>> able
> >>>>>>>>>>>>>> to support the resources needs of a volume. It seems that a
> >>>>>> similar
> >>>>>>>>>>>> set of
> >>>>>>>>>>>>>> considerations would need to be added to the Hypervisor
> layer
> >>>>>> which
> >>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yeah, for this first release, we just followed the path that
> >>>> was
> >>>>>>>>>>>> previously
> >>>>>>>>>>>>> established for other properties you see on dialogs in CS:
> Just
> >>>>>>>> because
> >>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
> >>>>>> supports
> >>>>>>>>>>>> them.
> >>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
> >>>> storage
> >>>>>> tag
> >>>>>>>>>>>> that
> >>>>>>>>>>>>> confines the deployment only to storage that supports the
> >>>> selected
> >>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
> >>>> CloudStack
> >>>>>>>> works
> >>>>>>>>>>>>> today.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I understand the tag functionality, and the need for the
> >>>>>> administrator
> >>>>>>>>>>>> to very carefully construct offerings. My concern is that we
> >>>> over
> >>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
> >>>>>>>> illustration,
> >>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that
> device
> >>>> is
> >>>>>>>> 100000.
> >>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
> >>>> Therefore,
> >>>>>> we
> >>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
> >>>> guaranteed
> >>>>>>>> IOPS
> >>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
> >>>>>>>> configuration
> >>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
> >>>> As we
> >>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
> >>>> guarantee
> >>>>>>>> more IOPS
> >>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that make
> >>>>>> sense?
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not
> even
> >>>>>> sure
> >>>>>>>> I
> >>>>>>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
> >>>> have
> >>>>>> to
> >>>>>>>> get
> >>>>>>>>>>> all of the min values for all of the volumes on the SAN and
> total
> >>>>>> them
> >>>>>>>> up,
> >>>>>>>>>>> I suppose, and subtract it from the total (user facing)
> supported
> >>>>>> IOPS
> >>>>>>>> of
> >>>>>>>>>>> the system.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
> >>>> resources
> >>>>>>>> to
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
> >>>>>> provisioned
> >>>>>>>>>>>> IOPS, I
> >>>>>>>>>>>>>> think a side effect of the current implementation is
> SolidFire
> >>>>>>>> volumes
> >>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
> >>>>>> seems
> >>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
> >>>> with
> >>>>>> QoS
> >>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
> >>>> starve
> >>>>>>>> out
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
> >>>>>> collocated
> >>>>>>>>>>>> VMs
> >>>>>>>>>>>>>> from being fulfilled.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
> >>>> not
> >>>>>>>>>>>> specify
> >>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
> >>>>>>>> defaults
> >>>>>>>>>>>> we
> >>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN,
> you
> >>>> will
> >>>>>>>>>>>> have
> >>>>>>>>>>>>> your Mins met.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
> >>>> exclusions
> >>>>>>>>>>>> list,
> >>>>>>>>>>>>> which might be nice to have at some point (i.e. don't deploy
> my
> >>>>>>>> volume
> >>>>>>>>>>>> to
> >>>>>>>>>>>>> storage with these following tags).
> >>>>>>>>>>>>
> >>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
> >>>> would
> >>>>>>>>>>>> complicate component assembly. It would require a storage
> >>>> plugin to
> >>>>>>>>>>>> anticipate all of the possible component assemblies and
> >>>> determine
> >>>>>> the
> >>>>>>>>>>>> invalid relationships. I prefer that drivers express their
> >>>>>>>> capabilities
> >>>>>>>>>>>> which can be matched to a set of requested requirements.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion
> or
> >>>>>>>>>>> exclusion lists. It just needs to advertise its functionality
> in
> >>>> a
> >>>>>> way
> >>>>>>>> the
> >>>>>>>>>>> allocator understands.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
> >>>>>> limits
> >>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
> >>>> with
> >>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>> minimum IOPS does this.
> >>>>>>>>>>>>
> >>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does not
> >>>>>> exist.
> >>>>>>>>>>>> However, I am concerned with the more general case as we
> >>>> supported
> >>>>>>>> more
> >>>>>>>>>>>> devices with provisioned IOPS.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
> >>>> values
> >>>>>> to
> >>>>>>>>>>> drivers. Min and Max and pretty industry standard for
> provisioned
> >>>>>>>> IOPS, but
> >>>>>>>>>>> what if you break them out by read and write or do something
> >>>> else? We
> >>>>>>>> need
> >>>>>>>>>>> a more general mechanism.
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
> >>>> I/O
> >>>>>> and
> >>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> We also need to understand the implications of these QoS
> >>>>>> guarantees
> >>>>>>>> on
> >>>>>>>>>>>>>> operation of the system to ensure that the underlying
> hardware
> >>>>>>>>>>>> resources
> >>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
> >>>> forced
> >>>>>> to
> >>>>>>>>>>>> make
> >>>>>>>>>>>>>> compromises to achieve these goals, and refine the
> >>>> implementation
> >>>>>> in
> >>>>>>>>>>>> 4.3.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I agree, John. I also think you've come up with some great
> >>>> ideas
> >>>>>> for
> >>>>>>>>>>>> 4.3. :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
> >>>> storage
> >>>>>>>>>>>> tags).
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
> >>>> addition to
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS
> being
> >>>>>>>>>>>> preferred,
> >>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
> >>>>>> hypervisor
> >>>>>>>>>>>> QoS
> >>>>>>>>>>>>>> is).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think John's concern is that you can enter in values for
> >>>> Wei's
> >>>>>>>> and
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
> >>>> yet), so
> >>>>>> he
> >>>>>>>>>>>> was
> >>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators
> to
> >>>>>> prefer
> >>>>>>>>>>>> these
> >>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands today
> >>>> in
> >>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>> we already have this kind of an issue with other features
> >>>> (fields
> >>>>>>>> in
> >>>>>>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
> >>>> post
> >>>>>>>> 4.2
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
> >>>> OpenStack
> >>>>>>>>>>>> addresses
> >>>>>>>>>>>>>>> the issue).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks
> until
> >>>>>> code
> >>>>>>>>>>>>>> freeze)
> >>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
> >>>> fields
> >>>>>>>>>>>> and my
> >>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
> >>>> data
> >>>>>>>>>>>> entered
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
> >>>> design
> >>>>>>>>>>>> that can
> >>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
> >>>> situations.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I just published my review @
> >>>>>> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
> >>>> with a
> >>>>>>>> new
> >>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>> file.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
> >>>> checked
> >>>>>>>> in
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> >>>>>>>> offline
> >>>>>>>>>>>> ATM,
> >>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
> >>>> this :)
> >>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
> >>>> areas
> >>>>>> we
> >>>>>>>>>>>> wanted
> >>>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> >>>>>> afternoon,
> >>>>>>>>>>>> then
> >>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts
> in
> >>>> Java
> >>>>>>>>>>>> code,
> >>>>>>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
> >>>> branch
> >>>>>>>> into
> >>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>> object_store
> >>>>>>>>>>>> branch,
> >>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge
> the
> >>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes
> into
> >>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into
> this
> >>>>>> merge
> >>>>>>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
> >>>> of
> >>>>>>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> >>>>>>>> patches
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> >>>>>>>> SolidFire
> >>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> >>>>>> dependency
> >>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
> >>>> we
> >>>>>> can
> >>>>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
> >>>> into
> >>>>>> his
> >>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
> >>>> after his
> >>>>>>>>>>>> merge
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
> >>>> code in
> >>>>>>>> the
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> >>>>>> plug-in,
> >>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> >>>>>> (although
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
> >>>> actually
> >>>>>> in
> >>>>>>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
> >>>> the
> >>>>>> url
> >>>>>>>>>>>> field
> >>>>>>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is
> stored
> >>>> in
> >>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
> >>>> hypervisor
> >>>>>> in
> >>>>>>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
> >>>> place
> >>>>>> of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> >>>>>> checks
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> >>>>>> hypervisor
> >>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
> >>>> executes to
> >>>>>>>>>>>> attach
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked
> to
> >>>>>>>> detach a
> >>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor
> data
> >>>>>>>>>>>> structure
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst
> IOPS
> >>>> in
> >>>>>> 4.2
> >>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table.
> If
> >>>> we
> >>>>>>>> have
> >>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
> >>>> avoiding
> >>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> >>>>>> field.
> >>>>>>>> I
> >>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
> >>>> regarding
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
> >>>> are
> >>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >>>>>> listStoragePool
> >>>>>>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
> >>>> useful
> >>>>>>>> for
> >>>>>>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
> >>>> field."
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
> >>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>>>>>>> Offering-by-Disk
> >>>>>>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
> >>>> to be
> >>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>> disk_offering_details
> >>>>>>>> table.
> >>>>>>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
> >>>> IOPS
> >>>>>> in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in
> the
> >>>> DB.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
> >
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Simon,
I completely agree with you regarding SAN complexities and their associated performance guarantees. To that end, I am encouraging that we start out with a very simple approach -- let the operator tell us how many IOPS we can commit to the device rather than trying to somehow devise it from a management API. For 4.2, my primary concern is ensuring that we never oversubscribe the device which is a much easier problem than optimizing allocation to get the maximum IOPS from it. Oversubscription is a system stability issue against which we must always guard. In terms of optimization, I propose that we attack that problem in a later release once we have strong protections against oversubscription.
In terms of homogenizing storage (or hypervisors or network devices), it is a side effect of the integration and resource management provided by CloudStack. There is an inherent trade-off made between flexibility and optimization when adopting platforms such as CloudStack. My personal goal is to minimize that trade-off as much as possible which, in my experience, is best achieved by keeping things as simple as possible.
Thanks,
-John
On Jun 14, 2013, at 4:39 PM, Simon Weller <sw...@ena.com> wrote:
>
> John,
>
>
> I'm not arguing that CloudStack's job isn't to provide resource management. The challenge here is we're talking about managing a resource that is extremely complex in a very 'one size fits all' manner. For example, lets say you have a SAN that supports storage tiering, and can dynamically move data shards to different tiers of disks where the IOP vs capacity varies. So your data shard starts on an array of fast, lower capacity disks, and based on various criteria gets relocated to slower, higher capacity disks. In this scenario , your IOP max capacity has just changed for some subset (or all) of the data tied to this primary storage object based on some usage profile. Likewise, you may have some high use shards that never get demoted to larger disks, so you run out of your primary tier storage capacity.
> My point is, It's hard to account for these scenarios in an absolute world. I'm just concerned that we're getting ourselves tied up trying to paint all storage as being the same, when in fact every product and project whether commercial or open source has a different set of features and objectives.
>
>
> - Si
> ----- Original Message -----
>
> From: "John Burwell" <jb...@basho.com>
> To: dev@cloudstack.apache.org
> Sent: Friday, June 14, 2013 2:59:47 PM
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Simon,
>
> Yes, it is CloudStack's job to protect, as best it can, from oversubscribing resources. I would argue that resource management is one, if not the most, important functions of the system. It is no different than the allocation/planning performed for hosts relative to cores and memory. We can still oversubscribe resources, but we have rails + knobs and dials to avoid it. Without these controls in place, we could easily allow users to deploy workloads that overrun resources harming all tenants.
>
>
>
> I also think that we are over thinking this issue for provisioned IOPS. When the DataStore is configured, the administrator/operator simply needs to tell us the total number of IOPS that can be committed to it and an overcommitment factor. As we allocate volumes to that DataStore, we sum up the committed IOPS of the existing Volumes attached to the DataStore, apply the overcommitment factor, and determine whether or not the requested minimum IOPS for the new volume can be fulfilled. We can provide both general and vendor specific documentation for determining these values -- be they to consume the entire device or a portion of it.
>
> Querying the device is unnecessary and deceptive. CloudStack resource management is not interested in the current state of the device which could be anywhere from extremely heavy to extremely light at any given time. We are interested in the worst case load that is anticipated for resource. In my view, it is up to administrators/operators to instrument their environment to understand usage patterns and capacity. We should provide information that will help determine what should be instrumented/monitored, but that function should be performed outside of CloudStack.
>
>
>
> Thanks,
> -John
>
> On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
>
>> I'd like to comment on this briefly.
>>
>>
>>
>> I think an assumption is being made that the SAN is being dedicated to a CS instance.
>>
>> My person opinion that this whole IOPS calculation is getting rather complicated, and could probably be much simpler than this. Over subscription is a fact of life on virtually all storage, and is really no different in concept than multiple virt instances on a single piece of hardware. All decent SANs offer many management options for the storage engineers to keep track of IOPS utilization, and plan for spindle augmentation as required.
>> Is it really the job of CS to become yet another management layer on top of this?
>>
>> ----- Original Message -----
>>
>> From: "Mike Tutkowski" <mi...@solidfire.com>
>> To: dev@cloudstack.apache.org
>> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <us...@gmail.com>
>> Sent: Friday, June 14, 2013 1:00:26 PM
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>> 1) We want number of IOPS currently supported by the SAN.
>>
>> 2) We want the number of IOPS that are committed (sum of min IOPS for each
>> volume).
>>
>> We could do the following to keep track of IOPS:
>>
>> The plug-in could have a timer thread that goes off every, say, 1 minute.
>>
>> It could query the SAN for the number of nodes that make up the SAN and
>> multiple this by 50,000. This is essentially the number of supported IOPS
>> of the SAN.
>>
>> The next API call could be to get all of the volumes on the SAN. Iterate
>> through them all and add up their min IOPS values. This is the number of
>> IOPS the SAN is committed to.
>>
>> These two numbers can then be updated in the storage_pool table (a column
>> for each value).
>>
>> The allocators can get these values as needed (and they would be as
>> accurate as the last time the thread asked the SAN for this info).
>>
>> These two fields, the min IOPS of the volume to create, and the overcommit
>> ratio of the plug-in would tell the allocator if it can select the given
>> storage pool.
>>
>> What do you think?
>>
>>
>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> "As I mentioned previously, I am very reluctant for any feature to come
>>> into master that can exhaust resources."
>>>
>>> Just wanted to mention that, worst case, the SAN would fail creation of
>>> the volume before allowing a new volume to break the system.
>>>
>>>
>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> Hi John,
>>>>
>>>> Are you thinking we add a column on to the storage pool table,
>>>> IOPS_Count, where we add and subtract committed IOPS?
>>>>
>>>> That is easy enough.
>>>>
>>>> How do you want to determine what the SAN is capable of supporting IOPS
>>>> wise? Remember we're dealing with a dynamic SAN here...as you add storage
>>>> nodes to the cluster, the number of IOPS increases. Do we have a thread we
>>>> can use to query external devices like this SAN to update the supported
>>>> number of IOPS?
>>>>
>>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>>>> overcommit ration to the plug-in when it's created? We would need to store
>>>> this in the storage_pool table, as well, I believe.
>>>>
>>>> We should also get Wei involved in this as his feature will need similar
>>>> functionality.
>>>>
>>>> Also, we should do this FAST as we have only two weeks left and many of
>>>> us will be out for several days for the CS Collab Conference.
>>>>
>>>> Thanks
>>>>
>>>>
>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jb...@basho.com>wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> Querying the SAN only indicates the number of IOPS currently in use.
>>>>> The allocator needs to check the number of IOPS committed which is tracked
>>>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
>>>>> committed to a DataStore, and determine whether or not the number requested
>>>>> can be fulfilled by that device. It seems to be that a
>>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>>>> DataStore's that don't support provisioned IOPS would return null.
>>>>>
>>>>> As I mentioned previously, I am very reluctant for any feature to come
>>>>> into master that can exhaust resources.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> Yeah, I'm not sure I could come up with anything near an accurate
>>>>>> assessment of how many IOPS are currently available on the SAN (or
>>>>> even a
>>>>>> total number that are available for volumes). Not sure if there's yet
>>>>> an
>>>>>> API call for that.
>>>>>>
>>>>>> If I did know this number (total number of IOPS supported by the SAN),
>>>>> we'd
>>>>>> still have to keep track of the total number of volumes we've created
>>>>> from
>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an API
>>>>> call
>>>>>> directly to the SAN to tweak the number of IOPS on a given volume or
>>>>> set of
>>>>>> volumes (not supported from CS, but supported via the SolidFire API),
>>>>> our
>>>>>> numbers in CS would be off.
>>>>>>
>>>>>> I'm thinking verifying sufficient number of IOPS is a really good idea
>>>>> for
>>>>>> a future release.
>>>>>>
>>>>>> For 4.2 I think we should stick to having the allocator detect if
>>>>> storage
>>>>>> QoS is desired and if the storage pool in question supports that
>>>>> feature.
>>>>>>
>>>>>> If you really are over provisioned on your SAN in terms of IOPS or
>>>>>> capacity, the SAN can let the admin know in several different ways
>>>>> (e-mail,
>>>>>> SNMP, GUI).
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> Please see my comments in-line below.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Comments below in red.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we may
>>>>> want
>>>>>>> to
>>>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
>>>>> user
>>>>>>> is
>>>>>>>>> expecting that the hypervisor will provide a particular QoS, and
>>>>> that is
>>>>>>>>> not possible, it seems like we should inform them rather than
>>>>> silently
>>>>>>>>> ignoring their request.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Sure, that sounds reasonable.
>>>>>>>>
>>>>>>>> We'd have to come up with some way for the allocators to know about
>>>>> the
>>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>>
>>>>>>>> Any thoughts on how we might do that?
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> To collect my thoughts from previous parts of the thread, I am
>>>>>>>>> uncomfortable with the idea that the management server can
>>>>> overcommit a
>>>>>>>>> resource. You had mentioned querying the device for available IOPS.
>>>>>>> While
>>>>>>>>> that would be nice, it seems like we could fall back to a max IOPS
>>>>> and
>>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>>> administrator/operator. I think such threshold and allocation rails
>>>>>>> should
>>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
>>>>> basic
>>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Are you thinking this ability would make it into 4.2? Just curious
>>>>> what
>>>>>>>> release we're talking about here. For the SolidFire SAN, you might
>>>>> have,
>>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
>>>>> later
>>>>>>> add
>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a way
>>>>> to
>>>>>>>> know that the number of supported IOPS has increased.
>>>>>>>
>>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>>>> example,
>>>>>>> we may only support expanding capacity in 4.2, and not handle any
>>>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
>>>>> a
>>>>>>> bit, the throttled I/O enhancement seems like it needs a similar set
>>>>> of
>>>>>>> rails defined per host.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>>>>>>>> implementation specific manner. I think we need to arrive at a
>>>>> general
>>>>>>>>> model that can be exploited by the allocators and planners. We
>>>>> should
>>>>>>>>> restrict implementation specific key-value pairs (call them details,
>>>>>>>>> extended data, whatever) to information that is unique to the
>>>>> driver and
>>>>>>>>> would provide no useful information to the management server's
>>>>>>>>> orchestration functions. A resource QoS does not seem to fit those
>>>>>>>>> criteria.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
>>>>> Collab
>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>>
>>>>>>> Yes, it would -- I will put something in the wiki topic. It will
>>>>> also be
>>>>>>> part of my talk on Monday -- How to Run from Zombie which include
>>>>> some of
>>>>>>> my opinions on the topic.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>>
>>>>>>>>>> Accept the values as they are currently required (four new fields
>>>>> for
>>>>>>>>> Wei's
>>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>>
>>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>>>>>>
>>>>>>>>>> 1) No QoS
>>>>>>>>>>
>>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>>
>>>>>>>>>> 3) Storage Qos
>>>>>>>>>>
>>>>>>>>>> The admin needs to specify storage tags that only map to storage
>>>>> that
>>>>>>>>>> supports Storage QoS.
>>>>>>>>>>
>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>>>>>>>> hypervisors
>>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>>
>>>>>>>>>> Post 4.3:
>>>>>>>>>>
>>>>>>>>>> Come up with a way to more generally enter these parameters
>>>>> (probably
>>>>>>>>> just
>>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>>
>>>>>>>>>> Have the drivers expose their feature set so the allocators can
>>>>>>> consider
>>>>>>>>>> them more fully and throw an exception if there is not a sufficient
>>>>>>>>> match.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>>>>>>>>> burden
>>>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
>>>>> sense.
>>>>>>> For
>>>>>>>>>>> example, if he wants to use storage QoS with supported Min and Max
>>>>>>>>> values,
>>>>>>>>>>> he'll have to put in a storage tag that references the SolidFire
>>>>>>> primary
>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then
>>>>> he's
>>>>>>>>> not
>>>>>>>>>>> going to get the Min and Max feature. We could add help text to
>>>>> the
>>>>>>>>> pop-up
>>>>>>>>>>> dialog that's displayed when you click in the Min and Max text
>>>>> fields.
>>>>>>>>>>>
>>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>>
>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>>
>>>>>>>>>>> Maybe in the future we could have the drivers advertise their
>>>>>>>>> capabilities
>>>>>>>>>>> and if the allocator feels a request is not being satisfied (say
>>>>> Min
>>>>>>> was
>>>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
>>>>> throw
>>>>>>> an
>>>>>>>>>>> exception.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>>> jburwell@basho.com
>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> First and foremost, we must ensure that these two features are
>>>>>>>>>>>>> mutually
>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>>>>>>>>>>>> contains both
>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>>> non-deterministic
>>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>>>>>>>>> oriented
>>>>>>>>>>>>>>> solves the problem in short term. As I understand storage
>>>>> tags,
>>>>>>> we
>>>>>>>>>>>>> have no
>>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>>> intending that we implement this allocation model in 4.3, but
>>>>>>>>> instead,
>>>>>>>>>>>>>>> determine if this type model would be one we would want to
>>>>> support
>>>>>>>>> in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the data
>>>>> model
>>>>>>>>> and
>>>>>>>>>>>>>>> current implementation would not preclude evolution in that
>>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>>> view is that this type of allocation model is what user's
>>>>> expect
>>>>>>> of
>>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>>> systems -- selecting the best available resource set to
>>>>> fulfill a
>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I believe we have meet your requirement here in that what we've
>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>> should not make refinement difficult in the future. If we don't
>>>>>>>>> modify
>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>>>>> simple
>>>>>>>>>>>>> changes
>>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Looking through both patches, I have to say that the aggregated
>>>>>>> result
>>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>>>>> throttled
>>>>>>>>> I/O and
>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
>>>>> is
>>>>>>> not
>>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>>>>> Javadoc
>>>>>>>>> should
>>>>>>>>>>>>> be added explaining the two sets of attributes and their mutual
>>>>>>>>> exclusion.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>>>> exclusion.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>>>>> provides
>>>>>>> both
>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
>>>>> uses
>>>>>>> a
>>>>>>>>>>>>> range. In order to select the best available resource to
>>>>> fulfill a
>>>>>>>>> QoS, we
>>>>>>>>>>>>> would need to have the QoS expression normalized in terms of
>>>>> units
>>>>>>>>> (IOPS or
>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>>>>>>> achieve a
>>>>>>>>>>>>> model like I described, I think we would need to resolve this
>>>>> issue
>>>>>>>>> in 4.2
>>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I think we're not likely to be able to normalize the input for
>>>>> 4.2.
>>>>>>>>> Plus
>>>>>>>>>>>> people probably want to input the data in terms they're familiar
>>>>> with
>>>>>>>>> for
>>>>>>>>>>>> the products in question.
>>>>>>>>>>>>
>>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let the
>>>>> user
>>>>>>>>> send
>>>>>>>>>>>> key/value pairs to each vendor that could be selected due to a
>>>>> given
>>>>>>>>>>>> storage tag. I'm still not sure that would solve it because what
>>>>>>>>> happens if
>>>>>>>>>>>> you change the storage tag of a given Primary Storage after
>>>>> having
>>>>>>>>> created
>>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>>
>>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>>>>> re-think
>>>>>>> it.
>>>>>>>>>>>>
>>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>>>>> supported
>>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As I think through the implications of these requirements and
>>>>>>>>> reflect
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>>>>> impacted
>>>>>>>>> the
>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>>> provisioned
>>>>>>> IOPS
>>>>>>>>>>>>> has no
>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
>>>>> that
>>>>>>> have
>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>>>> understand
>>>>>>>>> the
>>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>>>>> potentially
>>>>>>>>>>>>> causing
>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>>>>> to me
>>>>>>>>>>>>> that a
>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>>>> maximum
>>>>>>>>> IOPS
>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> it can support and some type of overcommitment factor. This
>>>>>>>>>>>>> information
>>>>>>>>>>>>>>> should be used by the storage allocators to determine the
>>>>> device
>>>>>>>>> best
>>>>>>>>>>>>> able
>>>>>>>>>>>>>>> to support the resources needs of a volume. It seems that a
>>>>>>> similar
>>>>>>>>>>>>> set of
>>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor layer
>>>>>>> which
>>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, for this first release, we just followed the path that
>>>>> was
>>>>>>>>>>>>> previously
>>>>>>>>>>>>>> established for other properties you see on dialogs in CS: Just
>>>>>>>>> because
>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>>>>>>> supports
>>>>>>>>>>>>> them.
>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>>>> storage
>>>>>>> tag
>>>>>>>>>>>>> that
>>>>>>>>>>>>>> confines the deployment only to storage that supports the
>>>>> selected
>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>>> CloudStack
>>>>>>>>> works
>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I understand the tag functionality, and the need for the
>>>>>>> administrator
>>>>>>>>>>>>> to very carefully construct offerings. My concern is that we
>>>>> over
>>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>>>>>>>> illustration,
>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that device
>>>>> is
>>>>>>>>> 100000.
>>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>>> Therefore,
>>>>>>> we
>>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>>>>> guaranteed
>>>>>>>>> IOPS
>>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>>>>>>>>> configuration
>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
>>>>> As we
>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>>>> guarantee
>>>>>>>>> more IOPS
>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that make
>>>>>>> sense?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not even
>>>>>>> sure
>>>>>>>>> I
>>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>>>>> have
>>>>>>> to
>>>>>>>>> get
>>>>>>>>>>>> all of the min values for all of the volumes on the SAN and total
>>>>>>> them
>>>>>>>>> up,
>>>>>>>>>>>> I suppose, and subtract it from the total (user facing) supported
>>>>>>> IOPS
>>>>>>>>> of
>>>>>>>>>>>> the system.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>>>> resources
>>>>>>>>> to
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>>>>>> provisioned
>>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>>> think a side effect of the current implementation is SolidFire
>>>>>>>>> volumes
>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>>>>>>> seems
>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>>>>> with
>>>>>>> QoS
>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>>>>> starve
>>>>>>>>> out
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>>>>>>> collocated
>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
>>>>> not
>>>>>>>>>>>>> specify
>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>>>>>>>> defaults
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you
>>>>> will
>>>>>>>>>>>>> have
>>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>>>> exclusions
>>>>>>>>>>>>> list,
>>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>>>>>>>>> volume
>>>>>>>>>>>>> to
>>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
>>>>> would
>>>>>>>>>>>>> complicate component assembly. It would require a storage
>>>>> plugin to
>>>>>>>>>>>>> anticipate all of the possible component assemblies and
>>>>> determine
>>>>>>> the
>>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>>>>>>>>> capabilities
>>>>>>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion or
>>>>>>>>>>>> exclusion lists. It just needs to advertise its functionality in
>>>>> a
>>>>>>> way
>>>>>>>>> the
>>>>>>>>>>>> allocator understands.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>>>>>>> limits
>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>>>>> with
>>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>>
>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>>>>>>> exist.
>>>>>>>>>>>>> However, I am concerned with the more general case as we
>>>>> supported
>>>>>>>>> more
>>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>>>>> values
>>>>>>> to
>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for provisioned
>>>>>>>>> IOPS, but
>>>>>>>>>>>> what if you break them out by read and write or do something
>>>>> else? We
>>>>>>>>> need
>>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>>>>> I/O
>>>>>>> and
>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We also need to understand the implications of these QoS
>>>>>>> guarantees
>>>>>>>>> on
>>>>>>>>>>>>>>> operation of the system to ensure that the underlying hardware
>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>>>>> forced
>>>>>>> to
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>>> implementation
>>>>>>> in
>>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I agree, John. I also think you've come up with some great
>>>>> ideas
>>>>>>> for
>>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>>>>> storage
>>>>>>>>>>>>> tags).
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>>>>> addition to
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS being
>>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>>>>>>> hypervisor
>>>>>>>>>>>>> QoS
>>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think John's concern is that you can enter in values for
>>>>> Wei's
>>>>>>>>> and
>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>>>>> yet), so
>>>>>>> he
>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
>>>>>>> prefer
>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands today
>>>>> in
>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>> we already have this kind of an issue with other features
>>>>> (fields
>>>>>>>>> in
>>>>>>>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
>>>>> post
>>>>>>>>> 4.2
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>>>>> OpenStack
>>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
>>>>>>> code
>>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>>>>> fields
>>>>>>>>>>>>> and my
>>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
>>>>> data
>>>>>>>>>>>>> entered
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
>>>>> design
>>>>>>>>>>>>> that can
>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>>>>> situations.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I just published my review @
>>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
>>>>> with a
>>>>>>>>> new
>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>>> checked
>>>>>>>>> in
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>>>>>>>> offline
>>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>>> this :)
>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>>> areas
>>>>>>> we
>>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>>>> afternoon,
>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>>>>> Java
>>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>>> branch
>>>>>>>>> into
>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.com
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>>> object_store
>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>>>>>> merge
>>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>>>> of
>>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>>>>>>>> patches
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>>>>>>> SolidFire
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>>>>> dependency
>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>>>> we
>>>>>>> can
>>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>>> into
>>>>>>> his
>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>>> after his
>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>>> code in
>>>>>>>>> the
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>>>>> plug-in,
>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>>>>> (although
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>>> actually
>>>>>>> in
>>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>>>> the
>>>>>>> url
>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>>>>> in
>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>>> hypervisor
>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>>> place
>>>>>>> of
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>>>>> checks
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>>>>> hypervisor
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>>> executes to
>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>>>>>>>> detach a
>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>>>>> in
>>>>>>> 4.2
>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>>>>> we
>>>>>>>>> have
>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>>> avoiding
>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>>>>> field.
>>>>>>>>> I
>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>>> regarding
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>>>> are
>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>>> listStoragePool
>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>>> useful
>>>>>>>>> for
>>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>>> to be
>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>>> disk_offering_details
>>>>>>>>> table.
>>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>>>> IOPS
>>>>>>> in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Simon Weller <sw...@ena.com>.
John,
I'm not arguing that CloudStack's job isn't to provide resource management. The challenge here is we're talking about managing a resource that is extremely complex in a very 'one size fits all' manner. For example, lets say you have a SAN that supports storage tiering, and can dynamically move data shards to different tiers of disks where the IOP vs capacity varies. So your data shard starts on an array of fast, lower capacity disks, and based on various criteria gets relocated to slower, higher capacity disks. In this scenario , your IOP max capacity has just changed for some subset (or all) of the data tied to this primary storage object based on some usage profile. Likewise, you may have some high use shards that never get demoted to larger disks, so you run out of your primary tier storage capacity.
My point is, It's hard to account for these scenarios in an absolute world. I'm just concerned that we're getting ourselves tied up trying to paint all storage as being the same, when in fact every product and project whether commercial or open source has a different set of features and objectives.
- Si
----- Original Message -----
From: "John Burwell" <jb...@basho.com>
To: dev@cloudstack.apache.org
Sent: Friday, June 14, 2013 2:59:47 PM
Subject: Re: [MERGE] disk_io_throttling to MASTER
Simon,
Yes, it is CloudStack's job to protect, as best it can, from oversubscribing resources. I would argue that resource management is one, if not the most, important functions of the system. It is no different than the allocation/planning performed for hosts relative to cores and memory. We can still oversubscribe resources, but we have rails + knobs and dials to avoid it. Without these controls in place, we could easily allow users to deploy workloads that overrun resources harming all tenants.
I also think that we are over thinking this issue for provisioned IOPS. When the DataStore is configured, the administrator/operator simply needs to tell us the total number of IOPS that can be committed to it and an overcommitment factor. As we allocate volumes to that DataStore, we sum up the committed IOPS of the existing Volumes attached to the DataStore, apply the overcommitment factor, and determine whether or not the requested minimum IOPS for the new volume can be fulfilled. We can provide both general and vendor specific documentation for determining these values -- be they to consume the entire device or a portion of it.
Querying the device is unnecessary and deceptive. CloudStack resource management is not interested in the current state of the device which could be anywhere from extremely heavy to extremely light at any given time. We are interested in the worst case load that is anticipated for resource. In my view, it is up to administrators/operators to instrument their environment to understand usage patterns and capacity. We should provide information that will help determine what should be instrumented/monitored, but that function should be performed outside of CloudStack.
Thanks,
-John
On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
> I'd like to comment on this briefly.
>
>
>
> I think an assumption is being made that the SAN is being dedicated to a CS instance.
>
> My person opinion that this whole IOPS calculation is getting rather complicated, and could probably be much simpler than this. Over subscription is a fact of life on virtually all storage, and is really no different in concept than multiple virt instances on a single piece of hardware. All decent SANs offer many management options for the storage engineers to keep track of IOPS utilization, and plan for spindle augmentation as required.
> Is it really the job of CS to become yet another management layer on top of this?
>
> ----- Original Message -----
>
> From: "Mike Tutkowski" <mi...@solidfire.com>
> To: dev@cloudstack.apache.org
> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <us...@gmail.com>
> Sent: Friday, June 14, 2013 1:00:26 PM
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> 1) We want number of IOPS currently supported by the SAN.
>
> 2) We want the number of IOPS that are committed (sum of min IOPS for each
> volume).
>
> We could do the following to keep track of IOPS:
>
> The plug-in could have a timer thread that goes off every, say, 1 minute.
>
> It could query the SAN for the number of nodes that make up the SAN and
> multiple this by 50,000. This is essentially the number of supported IOPS
> of the SAN.
>
> The next API call could be to get all of the volumes on the SAN. Iterate
> through them all and add up their min IOPS values. This is the number of
> IOPS the SAN is committed to.
>
> These two numbers can then be updated in the storage_pool table (a column
> for each value).
>
> The allocators can get these values as needed (and they would be as
> accurate as the last time the thread asked the SAN for this info).
>
> These two fields, the min IOPS of the volume to create, and the overcommit
> ratio of the plug-in would tell the allocator if it can select the given
> storage pool.
>
> What do you think?
>
>
> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> "As I mentioned previously, I am very reluctant for any feature to come
>> into master that can exhaust resources."
>>
>> Just wanted to mention that, worst case, the SAN would fail creation of
>> the volume before allowing a new volume to break the system.
>>
>>
>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Hi John,
>>>
>>> Are you thinking we add a column on to the storage pool table,
>>> IOPS_Count, where we add and subtract committed IOPS?
>>>
>>> That is easy enough.
>>>
>>> How do you want to determine what the SAN is capable of supporting IOPS
>>> wise? Remember we're dealing with a dynamic SAN here...as you add storage
>>> nodes to the cluster, the number of IOPS increases. Do we have a thread we
>>> can use to query external devices like this SAN to update the supported
>>> number of IOPS?
>>>
>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>>> overcommit ration to the plug-in when it's created? We would need to store
>>> this in the storage_pool table, as well, I believe.
>>>
>>> We should also get Wei involved in this as his feature will need similar
>>> functionality.
>>>
>>> Also, we should do this FAST as we have only two weeks left and many of
>>> us will be out for several days for the CS Collab Conference.
>>>
>>> Thanks
>>>
>>>
>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jb...@basho.com>wrote:
>>>
>>>> Mike,
>>>>
>>>> Querying the SAN only indicates the number of IOPS currently in use.
>>>> The allocator needs to check the number of IOPS committed which is tracked
>>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
>>>> committed to a DataStore, and determine whether or not the number requested
>>>> can be fulfilled by that device. It seems to be that a
>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>>> DataStore's that don't support provisioned IOPS would return null.
>>>>
>>>> As I mentioned previously, I am very reluctant for any feature to come
>>>> into master that can exhaust resources.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Yeah, I'm not sure I could come up with anything near an accurate
>>>>> assessment of how many IOPS are currently available on the SAN (or
>>>> even a
>>>>> total number that are available for volumes). Not sure if there's yet
>>>> an
>>>>> API call for that.
>>>>>
>>>>> If I did know this number (total number of IOPS supported by the SAN),
>>>> we'd
>>>>> still have to keep track of the total number of volumes we've created
>>>> from
>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an API
>>>> call
>>>>> directly to the SAN to tweak the number of IOPS on a given volume or
>>>> set of
>>>>> volumes (not supported from CS, but supported via the SolidFire API),
>>>> our
>>>>> numbers in CS would be off.
>>>>>
>>>>> I'm thinking verifying sufficient number of IOPS is a really good idea
>>>> for
>>>>> a future release.
>>>>>
>>>>> For 4.2 I think we should stick to having the allocator detect if
>>>> storage
>>>>> QoS is desired and if the storage pool in question supports that
>>>> feature.
>>>>>
>>>>> If you really are over provisioned on your SAN in terms of IOPS or
>>>>> capacity, the SAN can let the admin know in several different ways
>>>> (e-mail,
>>>>> SNMP, GUI).
>>>>>
>>>>>
>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Please see my comments in-line below.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Comments below in red.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we may
>>>> want
>>>>>> to
>>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
>>>> user
>>>>>> is
>>>>>>>> expecting that the hypervisor will provide a particular QoS, and
>>>> that is
>>>>>>>> not possible, it seems like we should inform them rather than
>>>> silently
>>>>>>>> ignoring their request.
>>>>>>>>
>>>>>>>
>>>>>>> Sure, that sounds reasonable.
>>>>>>>
>>>>>>> We'd have to come up with some way for the allocators to know about
>>>> the
>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>
>>>>>>> Any thoughts on how we might do that?
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> To collect my thoughts from previous parts of the thread, I am
>>>>>>>> uncomfortable with the idea that the management server can
>>>> overcommit a
>>>>>>>> resource. You had mentioned querying the device for available IOPS.
>>>>>> While
>>>>>>>> that would be nice, it seems like we could fall back to a max IOPS
>>>> and
>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>> administrator/operator. I think such threshold and allocation rails
>>>>>> should
>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
>>>> basic
>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>
>>>>>>>
>>>>>>> Are you thinking this ability would make it into 4.2? Just curious
>>>> what
>>>>>>> release we're talking about here. For the SolidFire SAN, you might
>>>> have,
>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
>>>> later
>>>>>> add
>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a way
>>>> to
>>>>>>> know that the number of supported IOPS has increased.
>>>>>>
>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>>> example,
>>>>>> we may only support expanding capacity in 4.2, and not handle any
>>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
>>>> a
>>>>>> bit, the throttled I/O enhancement seems like it needs a similar set
>>>> of
>>>>>> rails defined per host.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>>>>>>> implementation specific manner. I think we need to arrive at a
>>>> general
>>>>>>>> model that can be exploited by the allocators and planners. We
>>>> should
>>>>>>>> restrict implementation specific key-value pairs (call them details,
>>>>>>>> extended data, whatever) to information that is unique to the
>>>> driver and
>>>>>>>> would provide no useful information to the management server's
>>>>>>>> orchestration functions. A resource QoS does not seem to fit those
>>>>>>>> criteria.
>>>>>>>>
>>>>>>>
>>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
>>>> Collab
>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>
>>>>>> Yes, it would -- I will put something in the wiki topic. It will
>>>> also be
>>>>>> part of my talk on Monday -- How to Run from Zombie which include
>>>> some of
>>>>>> my opinions on the topic.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>
>>>>>>>>> Accept the values as they are currently required (four new fields
>>>> for
>>>>>>>> Wei's
>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>
>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>>>>>
>>>>>>>>> 1) No QoS
>>>>>>>>>
>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>
>>>>>>>>> 3) Storage Qos
>>>>>>>>>
>>>>>>>>> The admin needs to specify storage tags that only map to storage
>>>> that
>>>>>>>>> supports Storage QoS.
>>>>>>>>>
>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>>>>>>> hypervisors
>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>
>>>>>>>>> Post 4.3:
>>>>>>>>>
>>>>>>>>> Come up with a way to more generally enter these parameters
>>>> (probably
>>>>>>>> just
>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>
>>>>>>>>> Have the drivers expose their feature set so the allocators can
>>>>>> consider
>>>>>>>>> them more fully and throw an exception if there is not a sufficient
>>>>>>>> match.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>>>>>>>> burden
>>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
>>>> sense.
>>>>>> For
>>>>>>>>>> example, if he wants to use storage QoS with supported Min and Max
>>>>>>>> values,
>>>>>>>>>> he'll have to put in a storage tag that references the SolidFire
>>>>>> primary
>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then
>>>> he's
>>>>>>>> not
>>>>>>>>>> going to get the Min and Max feature. We could add help text to
>>>> the
>>>>>>>> pop-up
>>>>>>>>>> dialog that's displayed when you click in the Min and Max text
>>>> fields.
>>>>>>>>>>
>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>
>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>
>>>>>>>>>> Maybe in the future we could have the drivers advertise their
>>>>>>>> capabilities
>>>>>>>>>> and if the allocator feels a request is not being satisfied (say
>>>> Min
>>>>>> was
>>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
>>>> throw
>>>>>> an
>>>>>>>>>> exception.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>> jburwell@basho.com>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>> jburwell@basho.com
>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> First and foremost, we must ensure that these two features are
>>>>>>>>>>>> mutually
>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>>>>>>>>>>> contains both
>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>> non-deterministic
>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>>>>>>>> oriented
>>>>>>>>>>>>>> solves the problem in short term. As I understand storage
>>>> tags,
>>>>>> we
>>>>>>>>>>>> have no
>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>> intending that we implement this allocation model in 4.3, but
>>>>>>>> instead,
>>>>>>>>>>>>>> determine if this type model would be one we would want to
>>>> support
>>>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the data
>>>> model
>>>>>>>> and
>>>>>>>>>>>>>> current implementation would not preclude evolution in that
>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>> view is that this type of allocation model is what user's
>>>> expect
>>>>>> of
>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>> systems -- selecting the best available resource set to
>>>> fulfill a
>>>>>>>>>>>> set of
>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I believe we have meet your requirement here in that what we've
>>>>>>>>>>>> implemented
>>>>>>>>>>>>> should not make refinement difficult in the future. If we don't
>>>>>>>> modify
>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>>>> simple
>>>>>>>>>>>> changes
>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Looking through both patches, I have to say that the aggregated
>>>>>> result
>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>>>> throttled
>>>>>>>> I/O and
>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
>>>> is
>>>>>> not
>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>>>> Javadoc
>>>>>>>> should
>>>>>>>>>>>> be added explaining the two sets of attributes and their mutual
>>>>>>>> exclusion.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>>> exclusion.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>>>> provides
>>>>>> both
>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
>>>> uses
>>>>>> a
>>>>>>>>>>>> range. In order to select the best available resource to
>>>> fulfill a
>>>>>>>> QoS, we
>>>>>>>>>>>> would need to have the QoS expression normalized in terms of
>>>> units
>>>>>>>> (IOPS or
>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>>>>>> achieve a
>>>>>>>>>>>> model like I described, I think we would need to resolve this
>>>> issue
>>>>>>>> in 4.2
>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I think we're not likely to be able to normalize the input for
>>>> 4.2.
>>>>>>>> Plus
>>>>>>>>>>> people probably want to input the data in terms they're familiar
>>>> with
>>>>>>>> for
>>>>>>>>>>> the products in question.
>>>>>>>>>>>
>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let the
>>>> user
>>>>>>>> send
>>>>>>>>>>> key/value pairs to each vendor that could be selected due to a
>>>> given
>>>>>>>>>>> storage tag. I'm still not sure that would solve it because what
>>>>>>>> happens if
>>>>>>>>>>> you change the storage tag of a given Primary Storage after
>>>> having
>>>>>>>> created
>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>
>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>>>> re-think
>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>>>> supported
>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I think through the implications of these requirements and
>>>>>>>> reflect
>>>>>>>>>>>> on
>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>>>> impacted
>>>>>>>> the
>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>> provisioned
>>>>>> IOPS
>>>>>>>>>>>> has no
>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
>>>> that
>>>>>> have
>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>>> understand
>>>>>>>> the
>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>>>> potentially
>>>>>>>>>>>> causing
>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>>>> to me
>>>>>>>>>>>> that a
>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>>> maximum
>>>>>>>> IOPS
>>>>>>>>>>>> which
>>>>>>>>>>>>>> it can support and some type of overcommitment factor. This
>>>>>>>>>>>> information
>>>>>>>>>>>>>> should be used by the storage allocators to determine the
>>>> device
>>>>>>>> best
>>>>>>>>>>>> able
>>>>>>>>>>>>>> to support the resources needs of a volume. It seems that a
>>>>>> similar
>>>>>>>>>>>> set of
>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor layer
>>>>>> which
>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yeah, for this first release, we just followed the path that
>>>> was
>>>>>>>>>>>> previously
>>>>>>>>>>>>> established for other properties you see on dialogs in CS: Just
>>>>>>>> because
>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>>>>>> supports
>>>>>>>>>>>> them.
>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>>> storage
>>>>>> tag
>>>>>>>>>>>> that
>>>>>>>>>>>>> confines the deployment only to storage that supports the
>>>> selected
>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>> CloudStack
>>>>>>>> works
>>>>>>>>>>>>> today.
>>>>>>>>>>>>
>>>>>>>>>>>> I understand the tag functionality, and the need for the
>>>>>> administrator
>>>>>>>>>>>> to very carefully construct offerings. My concern is that we
>>>> over
>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>>>>>>> illustration,
>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that device
>>>> is
>>>>>>>> 100000.
>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>> Therefore,
>>>>>> we
>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>>>> guaranteed
>>>>>>>> IOPS
>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>>>>>>>> configuration
>>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
>>>> As we
>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>>> guarantee
>>>>>>>> more IOPS
>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that make
>>>>>> sense?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not even
>>>>>> sure
>>>>>>>> I
>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>>>> have
>>>>>> to
>>>>>>>> get
>>>>>>>>>>> all of the min values for all of the volumes on the SAN and total
>>>>>> them
>>>>>>>> up,
>>>>>>>>>>> I suppose, and subtract it from the total (user facing) supported
>>>>>> IOPS
>>>>>>>> of
>>>>>>>>>>> the system.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>>> resources
>>>>>>>> to
>>>>>>>>>>>> be
>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>>>>> provisioned
>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>> think a side effect of the current implementation is SolidFire
>>>>>>>> volumes
>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>>>>>> seems
>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>>>> with
>>>>>> QoS
>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>>>> starve
>>>>>>>> out
>>>>>>>>>>>> the
>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>>>>>> collocated
>>>>>>>>>>>> VMs
>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
>>>> not
>>>>>>>>>>>> specify
>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>>>>>>> defaults
>>>>>>>>>>>> we
>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you
>>>> will
>>>>>>>>>>>> have
>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>>> exclusions
>>>>>>>>>>>> list,
>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>>>>>>>> volume
>>>>>>>>>>>> to
>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>
>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
>>>> would
>>>>>>>>>>>> complicate component assembly. It would require a storage
>>>> plugin to
>>>>>>>>>>>> anticipate all of the possible component assemblies and
>>>> determine
>>>>>> the
>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>>>>>>>> capabilities
>>>>>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion or
>>>>>>>>>>> exclusion lists. It just needs to advertise its functionality in
>>>> a
>>>>>> way
>>>>>>>> the
>>>>>>>>>>> allocator understands.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>>>>>> limits
>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>>>> with
>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>
>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>>>>>> exist.
>>>>>>>>>>>> However, I am concerned with the more general case as we
>>>> supported
>>>>>>>> more
>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>>>> values
>>>>>> to
>>>>>>>>>>> drivers. Min and Max and pretty industry standard for provisioned
>>>>>>>> IOPS, but
>>>>>>>>>>> what if you break them out by read and write or do something
>>>> else? We
>>>>>>>> need
>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>>>> I/O
>>>>>> and
>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> We also need to understand the implications of these QoS
>>>>>> guarantees
>>>>>>>> on
>>>>>>>>>>>>>> operation of the system to ensure that the underlying hardware
>>>>>>>>>>>> resources
>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>>>> forced
>>>>>> to
>>>>>>>>>>>> make
>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>> implementation
>>>>>> in
>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree, John. I also think you've come up with some great
>>>> ideas
>>>>>> for
>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>>>> storage
>>>>>>>>>>>> tags).
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>>>> addition to
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS being
>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>>>>>> hypervisor
>>>>>>>>>>>> QoS
>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think John's concern is that you can enter in values for
>>>> Wei's
>>>>>>>> and
>>>>>>>>>>>> my
>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>>>> yet), so
>>>>>> he
>>>>>>>>>>>> was
>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
>>>>>> prefer
>>>>>>>>>>>> these
>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands today
>>>> in
>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>> we already have this kind of an issue with other features
>>>> (fields
>>>>>>>> in
>>>>>>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
>>>> post
>>>>>>>> 4.2
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>>>> OpenStack
>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
>>>>>> code
>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>>>> fields
>>>>>>>>>>>> and my
>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
>>>> data
>>>>>>>>>>>> entered
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
>>>> design
>>>>>>>>>>>> that can
>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>>>> situations.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I just published my review @
>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
>>>> with a
>>>>>>>> new
>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>> checked
>>>>>>>> in
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>>>>>>> offline
>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>> this :)
>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>> areas
>>>>>> we
>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>>> afternoon,
>>>>>>>>>>>> then
>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>>>> Java
>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>> branch
>>>>>>>> into
>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.com
>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>> object_store
>>>>>>>>>>>> branch,
>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>>>>> merge
>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>>> of
>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>>>>>>> patches
>>>>>>>>>>>> to
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>>>>>> SolidFire
>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>>>> dependency
>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>>> we
>>>>>> can
>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>> into
>>>>>> his
>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>> after his
>>>>>>>>>>>> merge
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>> code in
>>>>>>>> the
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>>>> plug-in,
>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>>>> (although
>>>>>>>>>>>> it
>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>> actually
>>>>>> in
>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>>> the
>>>>>> url
>>>>>>>>>>>> field
>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>>>> in
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>> hypervisor
>>>>>> in
>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>> place
>>>>>> of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>>>> checks
>>>>>>>>>>>> for
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>>>> hypervisor
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>> executes to
>>>>>>>>>>>> attach
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>>>>>>> detach a
>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>>>>>>>>>>> structure
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>>>> in
>>>>>> 4.2
>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>>>> we
>>>>>>>> have
>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>> avoiding
>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>>>> field.
>>>>>>>> I
>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>> regarding
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>>> are
>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>> listStoragePool
>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>> useful
>>>>>>>> for
>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>> to be
>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>> disk_offering_details
>>>>>>>> table.
>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>>> IOPS
>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Simon,
Yes, it is CloudStack's job to protect, as best it can, from oversubscribing resources. I would argue that resource management is one, if not the most, important functions of the system. It is no different than the allocation/planning performed for hosts relative to cores and memory. We can still oversubscribe resources, but we have rails + knobs and dials to avoid it. Without these controls in place, we could easily allow users to deploy workloads that overrun resources harming all tenants.
I also think that we are over thinking this issue for provisioned IOPS. When the DataStore is configured, the administrator/operator simply needs to tell us the total number of IOPS that can be committed to it and an overcommitment factor. As we allocate volumes to that DataStore, we sum up the committed IOPS of the existing Volumes attached to the DataStore, apply the overcommitment factor, and determine whether or not the requested minimum IOPS for the new volume can be fulfilled. We can provide both general and vendor specific documentation for determining these values -- be they to consume the entire device or a portion of it.
Querying the device is unnecessary and deceptive. CloudStack resource management is not interested in the current state of the device which could be anywhere from extremely heavy to extremely light at any given time. We are interested in the worst case load that is anticipated for resource. In my view, it is up to administrators/operators to instrument their environment to understand usage patterns and capacity. We should provide information that will help determine what should be instrumented/monitored, but that function should be performed outside of CloudStack.
Thanks,
-John
On Jun 14, 2013, at 2:20 PM, Simon Weller <sw...@ena.com> wrote:
> I'd like to comment on this briefly.
>
>
>
> I think an assumption is being made that the SAN is being dedicated to a CS instance.
>
> My person opinion that this whole IOPS calculation is getting rather complicated, and could probably be much simpler than this. Over subscription is a fact of life on virtually all storage, and is really no different in concept than multiple virt instances on a single piece of hardware. All decent SANs offer many management options for the storage engineers to keep track of IOPS utilization, and plan for spindle augmentation as required.
> Is it really the job of CS to become yet another management layer on top of this?
>
> ----- Original Message -----
>
> From: "Mike Tutkowski" <mi...@solidfire.com>
> To: dev@cloudstack.apache.org
> Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <us...@gmail.com>
> Sent: Friday, June 14, 2013 1:00:26 PM
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> 1) We want number of IOPS currently supported by the SAN.
>
> 2) We want the number of IOPS that are committed (sum of min IOPS for each
> volume).
>
> We could do the following to keep track of IOPS:
>
> The plug-in could have a timer thread that goes off every, say, 1 minute.
>
> It could query the SAN for the number of nodes that make up the SAN and
> multiple this by 50,000. This is essentially the number of supported IOPS
> of the SAN.
>
> The next API call could be to get all of the volumes on the SAN. Iterate
> through them all and add up their min IOPS values. This is the number of
> IOPS the SAN is committed to.
>
> These two numbers can then be updated in the storage_pool table (a column
> for each value).
>
> The allocators can get these values as needed (and they would be as
> accurate as the last time the thread asked the SAN for this info).
>
> These two fields, the min IOPS of the volume to create, and the overcommit
> ratio of the plug-in would tell the allocator if it can select the given
> storage pool.
>
> What do you think?
>
>
> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> "As I mentioned previously, I am very reluctant for any feature to come
>> into master that can exhaust resources."
>>
>> Just wanted to mention that, worst case, the SAN would fail creation of
>> the volume before allowing a new volume to break the system.
>>
>>
>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Hi John,
>>>
>>> Are you thinking we add a column on to the storage pool table,
>>> IOPS_Count, where we add and subtract committed IOPS?
>>>
>>> That is easy enough.
>>>
>>> How do you want to determine what the SAN is capable of supporting IOPS
>>> wise? Remember we're dealing with a dynamic SAN here...as you add storage
>>> nodes to the cluster, the number of IOPS increases. Do we have a thread we
>>> can use to query external devices like this SAN to update the supported
>>> number of IOPS?
>>>
>>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>>> overcommit ration to the plug-in when it's created? We would need to store
>>> this in the storage_pool table, as well, I believe.
>>>
>>> We should also get Wei involved in this as his feature will need similar
>>> functionality.
>>>
>>> Also, we should do this FAST as we have only two weeks left and many of
>>> us will be out for several days for the CS Collab Conference.
>>>
>>> Thanks
>>>
>>>
>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jb...@basho.com>wrote:
>>>
>>>> Mike,
>>>>
>>>> Querying the SAN only indicates the number of IOPS currently in use.
>>>> The allocator needs to check the number of IOPS committed which is tracked
>>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
>>>> committed to a DataStore, and determine whether or not the number requested
>>>> can be fulfilled by that device. It seems to be that a
>>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>>> DataStore's that don't support provisioned IOPS would return null.
>>>>
>>>> As I mentioned previously, I am very reluctant for any feature to come
>>>> into master that can exhaust resources.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Yeah, I'm not sure I could come up with anything near an accurate
>>>>> assessment of how many IOPS are currently available on the SAN (or
>>>> even a
>>>>> total number that are available for volumes). Not sure if there's yet
>>>> an
>>>>> API call for that.
>>>>>
>>>>> If I did know this number (total number of IOPS supported by the SAN),
>>>> we'd
>>>>> still have to keep track of the total number of volumes we've created
>>>> from
>>>>> CS on the SAN in terms of their IOPS. Also, if an admin issues an API
>>>> call
>>>>> directly to the SAN to tweak the number of IOPS on a given volume or
>>>> set of
>>>>> volumes (not supported from CS, but supported via the SolidFire API),
>>>> our
>>>>> numbers in CS would be off.
>>>>>
>>>>> I'm thinking verifying sufficient number of IOPS is a really good idea
>>>> for
>>>>> a future release.
>>>>>
>>>>> For 4.2 I think we should stick to having the allocator detect if
>>>> storage
>>>>> QoS is desired and if the storage pool in question supports that
>>>> feature.
>>>>>
>>>>> If you really are over provisioned on your SAN in terms of IOPS or
>>>>> capacity, the SAN can let the admin know in several different ways
>>>> (e-mail,
>>>>> SNMP, GUI).
>>>>>
>>>>>
>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Please see my comments in-line below.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Comments below in red.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> Overall, I agree with the steps to below for 4.2. However, we may
>>>> want
>>>>>> to
>>>>>>>> throw an exception if we can not fulfill a requested QoS. If the
>>>> user
>>>>>> is
>>>>>>>> expecting that the hypervisor will provide a particular QoS, and
>>>> that is
>>>>>>>> not possible, it seems like we should inform them rather than
>>>> silently
>>>>>>>> ignoring their request.
>>>>>>>>
>>>>>>>
>>>>>>> Sure, that sounds reasonable.
>>>>>>>
>>>>>>> We'd have to come up with some way for the allocators to know about
>>>> the
>>>>>>> requested storage QoS and then query the candidate drivers.
>>>>>>>
>>>>>>> Any thoughts on how we might do that?
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> To collect my thoughts from previous parts of the thread, I am
>>>>>>>> uncomfortable with the idea that the management server can
>>>> overcommit a
>>>>>>>> resource. You had mentioned querying the device for available IOPS.
>>>>>> While
>>>>>>>> that would be nice, it seems like we could fall back to a max IOPS
>>>> and
>>>>>>>> overcommit factor manually calculated and entered by the
>>>>>>>> administrator/operator. I think such threshold and allocation rails
>>>>>> should
>>>>>>>> be added for both provisioned IOPS and throttled I/O -- it is a
>>>> basic
>>>>>>>> feature of any cloud orchestration platform.
>>>>>>>>
>>>>>>>
>>>>>>> Are you thinking this ability would make it into 4.2? Just curious
>>>> what
>>>>>>> release we're talking about here. For the SolidFire SAN, you might
>>>> have,
>>>>>>> say, four separate storage nodes to start (200,000 IOPS) and then
>>>> later
>>>>>> add
>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to have a way
>>>> to
>>>>>>> know that the number of supported IOPS has increased.
>>>>>>
>>>>>> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>>> example,
>>>>>> we may only support expanding capacity in 4.2, and not handle any
>>>>>> re-balance scenarios -- node failure, addition, etc. Extrapolating
>>>> a
>>>>>> bit, the throttled I/O enhancement seems like it needs a similar set
>>>> of
>>>>>> rails defined per host.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>>>>>>> implementation specific manner. I think we need to arrive at a
>>>> general
>>>>>>>> model that can be exploited by the allocators and planners. We
>>>> should
>>>>>>>> restrict implementation specific key-value pairs (call them details,
>>>>>>>> extended data, whatever) to information that is unique to the
>>>> driver and
>>>>>>>> would provide no useful information to the management server's
>>>>>>>> orchestration functions. A resource QoS does not seem to fit those
>>>>>>>> criteria.
>>>>>>>>
>>>>>>>
>>>>>>> I wonder if this would be a good discussion topic for Sunday's CS
>>>> Collab
>>>>>>> Conf hack day that Joe just sent out an e-mail about?
>>>>>>
>>>>>> Yes, it would -- I will put something in the wiki topic. It will
>>>> also be
>>>>>> part of my talk on Monday -- How to Run from Zombie which include
>>>> some of
>>>>>> my opinions on the topic.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> So, here's my suggestion for 4.2:
>>>>>>>>>
>>>>>>>>> Accept the values as they are currently required (four new fields
>>>> for
>>>>>>>> Wei's
>>>>>>>>> feature or two new fields for mine).
>>>>>>>>>
>>>>>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>>>>>
>>>>>>>>> 1) No QoS
>>>>>>>>>
>>>>>>>>> 2) Hypervisor QoS
>>>>>>>>>
>>>>>>>>> 3) Storage Qos
>>>>>>>>>
>>>>>>>>> The admin needs to specify storage tags that only map to storage
>>>> that
>>>>>>>>> supports Storage QoS.
>>>>>>>>>
>>>>>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>>>>>>> hypervisors
>>>>>>>>> in use support the new fields, they may not be enforced.
>>>>>>>>>
>>>>>>>>> Post 4.3:
>>>>>>>>>
>>>>>>>>> Come up with a way to more generally enter these parameters
>>>> (probably
>>>>>>>> just
>>>>>>>>> key/value pairs sent to the drivers).
>>>>>>>>>
>>>>>>>>> Have the drivers expose their feature set so the allocators can
>>>>>> consider
>>>>>>>>> them more fully and throw an exception if there is not a sufficient
>>>>>>>> match.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>>>>>>>> burden
>>>>>>>>>> on the admin to configure a Disk Offering in a way that makes
>>>> sense.
>>>>>> For
>>>>>>>>>> example, if he wants to use storage QoS with supported Min and Max
>>>>>>>> values,
>>>>>>>>>> he'll have to put in a storage tag that references the SolidFire
>>>>>> primary
>>>>>>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then
>>>> he's
>>>>>>>> not
>>>>>>>>>> going to get the Min and Max feature. We could add help text to
>>>> the
>>>>>>>> pop-up
>>>>>>>>>> dialog that's displayed when you click in the Min and Max text
>>>> fields.
>>>>>>>>>>
>>>>>>>>>> Same idea for Wei's feature.
>>>>>>>>>>
>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>>>>>>>>> approach for post 4.2.
>>>>>>>>>>
>>>>>>>>>> Maybe in the future we could have the drivers advertise their
>>>>>>>> capabilities
>>>>>>>>>> and if the allocator feels a request is not being satisfied (say
>>>> Min
>>>>>> was
>>>>>>>>>> entered, but it not's supported by any storage plug-in) it can
>>>> throw
>>>>>> an
>>>>>>>>>> exception.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Comments below in red.
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>>> jburwell@basho.com>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Please see my comment in-line below.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've put comments below in red.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>>> jburwell@basho.com
>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> First and foremost, we must ensure that these two features are
>>>>>>>>>>>> mutually
>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>>>>>>>>>>> contains both
>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>>>>>> non-deterministic
>>>>>>>>>>>>>> operations.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>>>>>>>> oriented
>>>>>>>>>>>>>> solves the problem in short term. As I understand storage
>>>> tags,
>>>>>> we
>>>>>>>>>>>> have no
>>>>>>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>>>>>>>>>>> necessarily
>>>>>>>>>>>>>> intending that we implement this allocation model in 4.3, but
>>>>>>>> instead,
>>>>>>>>>>>>>> determine if this type model would be one we would want to
>>>> support
>>>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that the data
>>>> model
>>>>>>>> and
>>>>>>>>>>>>>> current implementation would not preclude evolution in that
>>>>>>>>>>>> direction. My
>>>>>>>>>>>>>> view is that this type of allocation model is what user's
>>>> expect
>>>>>> of
>>>>>>>>>>>> "cloud"
>>>>>>>>>>>>>> systems -- selecting the best available resource set to
>>>> fulfill a
>>>>>>>>>>>> set of
>>>>>>>>>>>>>> system requirements.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I believe we have meet your requirement here in that what we've
>>>>>>>>>>>> implemented
>>>>>>>>>>>>> should not make refinement difficult in the future. If we don't
>>>>>>>> modify
>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>>>> simple
>>>>>>>>>>>> changes
>>>>>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Looking through both patches, I have to say that the aggregated
>>>>>> result
>>>>>>>>>>>> seems a bit confusing. There are six new attributes for
>>>> throttled
>>>>>>>> I/O and
>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
>>>> is
>>>>>> not
>>>>>>>>>>>> technical, but rather, about maintainability. At minimum,
>>>> Javadoc
>>>>>>>> should
>>>>>>>>>>>> be added explaining the two sets of attributes and their mutual
>>>>>>>> exclusion.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>>> exclusion.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The other part that is interesting is that throttled I/O
>>>> provides
>>>>>> both
>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
>>>> uses
>>>>>> a
>>>>>>>>>>>> range. In order to select the best available resource to
>>>> fulfill a
>>>>>>>> QoS, we
>>>>>>>>>>>> would need to have the QoS expression normalized in terms of
>>>> units
>>>>>>>> (IOPS or
>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>>>>>> achieve a
>>>>>>>>>>>> model like I described, I think we would need to resolve this
>>>> issue
>>>>>>>> in 4.2
>>>>>>>>>>>> to ensure a viable migration path.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I think we're not likely to be able to normalize the input for
>>>> 4.2.
>>>>>>>> Plus
>>>>>>>>>>> people probably want to input the data in terms they're familiar
>>>> with
>>>>>>>> for
>>>>>>>>>>> the products in question.
>>>>>>>>>>>
>>>>>>>>>>> Ideally we would fix the way we do storage tagging and let the
>>>> user
>>>>>>>> send
>>>>>>>>>>> key/value pairs to each vendor that could be selected due to a
>>>> given
>>>>>>>>>>> storage tag. I'm still not sure that would solve it because what
>>>>>>>> happens if
>>>>>>>>>>> you change the storage tag of a given Primary Storage after
>>>> having
>>>>>>>> created
>>>>>>>>>>> a Disk Offering?
>>>>>>>>>>>
>>>>>>>>>>> Basically storage tagging is kind of a mess and we should
>>>> re-think
>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> Also, we need to have a way for the drivers to expose their
>>>> supported
>>>>>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I think through the implications of these requirements and
>>>>>>>> reflect
>>>>>>>>>>>> on
>>>>>>>>>>>>>> the reviews, I don't understand why they haven't already
>>>> impacted
>>>>>>>> the
>>>>>>>>>>>>>> allocators and planners. As it stands, the current
>>>> provisioned
>>>>>> IOPS
>>>>>>>>>>>> has no
>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to devices
>>>> that
>>>>>> have
>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>>> understand
>>>>>>>> the
>>>>>>>>>>>>>> current patch, we can overcommit storage resources --
>>>> potentially
>>>>>>>>>>>> causing
>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>>>> to me
>>>>>>>>>>>> that a
>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>>> maximum
>>>>>>>> IOPS
>>>>>>>>>>>> which
>>>>>>>>>>>>>> it can support and some type of overcommitment factor. This
>>>>>>>>>>>> information
>>>>>>>>>>>>>> should be used by the storage allocators to determine the
>>>> device
>>>>>>>> best
>>>>>>>>>>>> able
>>>>>>>>>>>>>> to support the resources needs of a volume. It seems that a
>>>>>> similar
>>>>>>>>>>>> set of
>>>>>>>>>>>>>> considerations would need to be added to the Hypervisor layer
>>>>>> which
>>>>>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yeah, for this first release, we just followed the path that
>>>> was
>>>>>>>>>>>> previously
>>>>>>>>>>>>> established for other properties you see on dialogs in CS: Just
>>>>>>>> because
>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>>>>>> supports
>>>>>>>>>>>> them.
>>>>>>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>>> storage
>>>>>> tag
>>>>>>>>>>>> that
>>>>>>>>>>>>> confines the deployment only to storage that supports the
>>>> selected
>>>>>>>>>>>>> features. This is not ideal, but it's kind of the way
>>>> CloudStack
>>>>>>>> works
>>>>>>>>>>>>> today.
>>>>>>>>>>>>
>>>>>>>>>>>> I understand the tag functionality, and the need for the
>>>>>> administrator
>>>>>>>>>>>> to very carefully construct offerings. My concern is that we
>>>> over
>>>>>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>>>>>>> illustration,
>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for that device
>>>> is
>>>>>>>> 100000.
>>>>>>>>>>>> We also know that we can safely oversubscribe by 50%.
>>>> Therefore,
>>>>>> we
>>>>>>>>>>>> need to ensure that we don't allocate more than 150,000
>>>> guaranteed
>>>>>>>> IOPS
>>>>>>>>>>>> from that device. Intuitively, it seems like the DataStore
>>>>>>>> configuration
>>>>>>>>>>>> should have a max assignable IOPS and overcommitment factor.
>>>> As we
>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>>> guarantee
>>>>>>>> more IOPS
>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does that make
>>>>>> sense?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I think that's a good idea for a future enhancement. I'm not even
>>>>>> sure
>>>>>>>> I
>>>>>>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>>>> have
>>>>>> to
>>>>>>>> get
>>>>>>>>>>> all of the min values for all of the volumes on the SAN and total
>>>>>> them
>>>>>>>> up,
>>>>>>>>>>> I suppose, and subtract it from the total (user facing) supported
>>>>>> IOPS
>>>>>>>> of
>>>>>>>>>>> the system.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>>> resources
>>>>>>>> to
>>>>>>>>>>>> be
>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>>>>> provisioned
>>>>>>>>>>>> IOPS, I
>>>>>>>>>>>>>> think a side effect of the current implementation is SolidFire
>>>>>>>> volumes
>>>>>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>>>>>> seems
>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>>>> with
>>>>>> QoS
>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>>>> starve
>>>>>>>> out
>>>>>>>>>>>> the
>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>>>>>> collocated
>>>>>>>>>>>> VMs
>>>>>>>>>>>>>> from being fulfilled.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
>>>> not
>>>>>>>>>>>> specify
>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>>>>>>> defaults
>>>>>>>>>>>> we
>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you
>>>> will
>>>>>>>>>>>> have
>>>>>>>>>>>>> your Mins met.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>>> exclusions
>>>>>>>>>>>> list,
>>>>>>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>>>>>>>> volume
>>>>>>>>>>>> to
>>>>>>>>>>>>> storage with these following tags).
>>>>>>>>>>>>
>>>>>>>>>>>> I don't like the idea of a storage tags exclusion list as it
>>>> would
>>>>>>>>>>>> complicate component assembly. It would require a storage
>>>> plugin to
>>>>>>>>>>>> anticipate all of the possible component assemblies and
>>>> determine
>>>>>> the
>>>>>>>>>>>> invalid relationships. I prefer that drivers express their
>>>>>>>> capabilities
>>>>>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure why a storage plug-in would care about inclusion or
>>>>>>>>>>> exclusion lists. It just needs to advertise its functionality in
>>>> a
>>>>>> way
>>>>>>>> the
>>>>>>>>>>> allocator understands.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>>>>>> limits
>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>>>> with
>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>> minimum IOPS does this.
>>>>>>>>>>>>
>>>>>>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>>>>>> exist.
>>>>>>>>>>>> However, I am concerned with the more general case as we
>>>> supported
>>>>>>>> more
>>>>>>>>>>>> devices with provisioned IOPS.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>>>> values
>>>>>> to
>>>>>>>>>>> drivers. Min and Max and pretty industry standard for provisioned
>>>>>>>> IOPS, but
>>>>>>>>>>> what if you break them out by read and write or do something
>>>> else? We
>>>>>>>> need
>>>>>>>>>>> a more general mechanism.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>>>> I/O
>>>>>> and
>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Agreed
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> We also need to understand the implications of these QoS
>>>>>> guarantees
>>>>>>>> on
>>>>>>>>>>>>>> operation of the system to ensure that the underlying hardware
>>>>>>>>>>>> resources
>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>>>> forced
>>>>>> to
>>>>>>>>>>>> make
>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
>>>> implementation
>>>>>> in
>>>>>>>>>>>> 4.3.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree, John. I also think you've come up with some great
>>>> ideas
>>>>>> for
>>>>>>>>>>>> 4.3. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>>>> storage
>>>>>>>>>>>> tags).
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> believe John was just throwing out an idea that - in
>>>> addition to
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>> tags - we could look into these allocators (storage QoS being
>>>>>>>>>>>> preferred,
>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>>>>>> hypervisor
>>>>>>>>>>>> QoS
>>>>>>>>>>>>>> is).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think John's concern is that you can enter in values for
>>>> Wei's
>>>>>>>> and
>>>>>>>>>>>> my
>>>>>>>>>>>>>>> feature that are not honored by other vendors (at least
>>>> yet), so
>>>>>> he
>>>>>>>>>>>> was
>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
>>>>>> prefer
>>>>>>>>>>>> these
>>>>>>>>>>>>>>> vendors when these fields are filled in. As it stands today
>>>> in
>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>> we already have this kind of an issue with other features
>>>> (fields
>>>>>>>> in
>>>>>>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
>>>> post
>>>>>>>> 4.2
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> could look into generic name/value pairs (this is how
>>>> OpenStack
>>>>>>>>>>>> addresses
>>>>>>>>>>>>>>> the issue).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
>>>>>> code
>>>>>>>>>>>>>> freeze)
>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>>>> fields
>>>>>>>>>>>> and my
>>>>>>>>>>>>>>> fields as is, make sure only one or the other feature has
>>>> data
>>>>>>>>>>>> entered
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then let's step back and look into a more general-purpose
>>>> design
>>>>>>>>>>>> that can
>>>>>>>>>>>>>>> be applied throughout CloudStack where we have these
>>>> situations.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I just published my review @
>>>>>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I still want to re-review it by myself before coming up
>>>> with a
>>>>>>>> new
>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>>> checked
>>>>>>>> in
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>>>>>>> offline
>>>>>>>>>>>> ATM,
>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>>> this :)
>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>>> areas
>>>>>> we
>>>>>>>>>>>> wanted
>>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>>>>> afternoon,
>>>>>>>>>>>> then
>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>>>> Java
>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>>> branch
>>>>>>>> into
>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.com
>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>>> object_store
>>>>>>>>>>>> branch,
>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>>>>> merge
>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>>> of
>>>>>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>>>>>>> patches
>>>>>>>>>>>> to
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>>>>>> SolidFire
>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>>>>> dependency
>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>>> we
>>>>>> can
>>>>>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>>> into
>>>>>> his
>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>>> after his
>>>>>>>>>>>> merge
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>>> code in
>>>>>>>> the
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>>>>> plug-in,
>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>>>>> (although
>>>>>>>>>>>> it
>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>>> actually
>>>>>> in
>>>>>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>>> the
>>>>>> url
>>>>>>>>>>>> field
>>>>>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>>>> in
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>>> hypervisor
>>>>>> in
>>>>>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>>> place
>>>>>> of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>>>>> checks
>>>>>>>>>>>> for
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>>>>> hypervisor
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>>> executes to
>>>>>>>>>>>> attach
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>>>>>>> detach a
>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>>>>>>>>>>> structure
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>>>> in
>>>>>> 4.2
>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>>>> we
>>>>>>>> have
>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>>> avoiding
>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>>>>> field.
>>>>>>>> I
>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>>> regarding
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>>> are
>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>>>>> listStoragePool
>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>>> useful
>>>>>>>> for
>>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>>> field."
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>>>>>> Offering-by-Disk
>>>>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>>> to be
>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>>> disk_offering_details
>>>>>>>> table.
>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>>> IOPS
>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>>>> DB.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Simon Weller <sw...@ena.com>.
I'd like to comment on this briefly.
I think an assumption is being made that the SAN is being dedicated to a CS instance.
My person opinion that this whole IOPS calculation is getting rather complicated, and could probably be much simpler than this. Over subscription is a fact of life on virtually all storage, and is really no different in concept than multiple virt instances on a single piece of hardware. All decent SANs offer many management options for the storage engineers to keep track of IOPS utilization, and plan for spindle augmentation as required.
Is it really the job of CS to become yet another management layer on top of this?
----- Original Message -----
From: "Mike Tutkowski" <mi...@solidfire.com>
To: dev@cloudstack.apache.org
Cc: "John Burwell" <jb...@basho.com>, "Wei Zhou" <us...@gmail.com>
Sent: Friday, June 14, 2013 1:00:26 PM
Subject: Re: [MERGE] disk_io_throttling to MASTER
1) We want number of IOPS currently supported by the SAN.
2) We want the number of IOPS that are committed (sum of min IOPS for each
volume).
We could do the following to keep track of IOPS:
The plug-in could have a timer thread that goes off every, say, 1 minute.
It could query the SAN for the number of nodes that make up the SAN and
multiple this by 50,000. This is essentially the number of supported IOPS
of the SAN.
The next API call could be to get all of the volumes on the SAN. Iterate
through them all and add up their min IOPS values. This is the number of
IOPS the SAN is committed to.
These two numbers can then be updated in the storage_pool table (a column
for each value).
The allocators can get these values as needed (and they would be as
accurate as the last time the thread asked the SAN for this info).
These two fields, the min IOPS of the volume to create, and the overcommit
ratio of the plug-in would tell the allocator if it can select the given
storage pool.
What do you think?
On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> "As I mentioned previously, I am very reluctant for any feature to come
> into master that can exhaust resources."
>
> Just wanted to mention that, worst case, the SAN would fail creation of
> the volume before allowing a new volume to break the system.
>
>
> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Hi John,
>>
>> Are you thinking we add a column on to the storage pool table,
>> IOPS_Count, where we add and subtract committed IOPS?
>>
>> That is easy enough.
>>
>> How do you want to determine what the SAN is capable of supporting IOPS
>> wise? Remember we're dealing with a dynamic SAN here...as you add storage
>> nodes to the cluster, the number of IOPS increases. Do we have a thread we
>> can use to query external devices like this SAN to update the supported
>> number of IOPS?
>>
>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>> overcommit ration to the plug-in when it's created? We would need to store
>> this in the storage_pool table, as well, I believe.
>>
>> We should also get Wei involved in this as his feature will need similar
>> functionality.
>>
>> Also, we should do this FAST as we have only two weeks left and many of
>> us will be out for several days for the CS Collab Conference.
>>
>> Thanks
>>
>>
>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jb...@basho.com>wrote:
>>
>>> Mike,
>>>
>>> Querying the SAN only indicates the number of IOPS currently in use.
>>> The allocator needs to check the number of IOPS committed which is tracked
>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
>>> committed to a DataStore, and determine whether or not the number requested
>>> can be fulfilled by that device. It seems to be that a
>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>> DataStore's that don't support provisioned IOPS would return null.
>>>
>>> As I mentioned previously, I am very reluctant for any feature to come
>>> into master that can exhaust resources.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> > Yeah, I'm not sure I could come up with anything near an accurate
>>> > assessment of how many IOPS are currently available on the SAN (or
>>> even a
>>> > total number that are available for volumes). Not sure if there's yet
>>> an
>>> > API call for that.
>>> >
>>> > If I did know this number (total number of IOPS supported by the SAN),
>>> we'd
>>> > still have to keep track of the total number of volumes we've created
>>> from
>>> > CS on the SAN in terms of their IOPS. Also, if an admin issues an API
>>> call
>>> > directly to the SAN to tweak the number of IOPS on a given volume or
>>> set of
>>> > volumes (not supported from CS, but supported via the SolidFire API),
>>> our
>>> > numbers in CS would be off.
>>> >
>>> > I'm thinking verifying sufficient number of IOPS is a really good idea
>>> for
>>> > a future release.
>>> >
>>> > For 4.2 I think we should stick to having the allocator detect if
>>> storage
>>> > QoS is desired and if the storage pool in question supports that
>>> feature.
>>> >
>>> > If you really are over provisioned on your SAN in terms of IOPS or
>>> > capacity, the SAN can let the admin know in several different ways
>>> (e-mail,
>>> > SNMP, GUI).
>>> >
>>> >
>>> > On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>> >
>>> >> Mike,
>>> >>
>>> >> Please see my comments in-line below.
>>> >>
>>> >> Thanks,
>>> >> -John
>>> >>
>>> >> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>> >> wrote:
>>> >>
>>> >>> Comments below in red.
>>> >>>
>>> >>> Thanks
>>> >>>
>>> >>>
>>> >>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>>> >> wrote:
>>> >>>
>>> >>>> Mike,
>>> >>>>
>>> >>>> Overall, I agree with the steps to below for 4.2. However, we may
>>> want
>>> >> to
>>> >>>> throw an exception if we can not fulfill a requested QoS. If the
>>> user
>>> >> is
>>> >>>> expecting that the hypervisor will provide a particular QoS, and
>>> that is
>>> >>>> not possible, it seems like we should inform them rather than
>>> silently
>>> >>>> ignoring their request.
>>> >>>>
>>> >>>
>>> >>> Sure, that sounds reasonable.
>>> >>>
>>> >>> We'd have to come up with some way for the allocators to know about
>>> the
>>> >>> requested storage QoS and then query the candidate drivers.
>>> >>>
>>> >>> Any thoughts on how we might do that?
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> To collect my thoughts from previous parts of the thread, I am
>>> >>>> uncomfortable with the idea that the management server can
>>> overcommit a
>>> >>>> resource. You had mentioned querying the device for available IOPS.
>>> >> While
>>> >>>> that would be nice, it seems like we could fall back to a max IOPS
>>> and
>>> >>>> overcommit factor manually calculated and entered by the
>>> >>>> administrator/operator. I think such threshold and allocation rails
>>> >> should
>>> >>>> be added for both provisioned IOPS and throttled I/O -- it is a
>>> basic
>>> >>>> feature of any cloud orchestration platform.
>>> >>>>
>>> >>>
>>> >>> Are you thinking this ability would make it into 4.2? Just curious
>>> what
>>> >>> release we're talking about here. For the SolidFire SAN, you might
>>> have,
>>> >>> say, four separate storage nodes to start (200,000 IOPS) and then
>>> later
>>> >> add
>>> >>> a new node (now you're at 250,000 IOPS). CS would have to have a way
>>> to
>>> >>> know that the number of supported IOPS has increased.
>>> >>
>>> >> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>> example,
>>> >> we may only support expanding capacity in 4.2, and not handle any
>>> >> re-balance scenarios -- node failure, addition, etc. Extrapolating
>>> a
>>> >> bit, the throttled I/O enhancement seems like it needs a similar set
>>> of
>>> >> rails defined per host.
>>> >>
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>> >>>> implementation specific manner. I think we need to arrive at a
>>> general
>>> >>>> model that can be exploited by the allocators and planners. We
>>> should
>>> >>>> restrict implementation specific key-value pairs (call them details,
>>> >>>> extended data, whatever) to information that is unique to the
>>> driver and
>>> >>>> would provide no useful information to the management server's
>>> >>>> orchestration functions. A resource QoS does not seem to fit those
>>> >>>> criteria.
>>> >>>>
>>> >>>
>>> >>> I wonder if this would be a good discussion topic for Sunday's CS
>>> Collab
>>> >>> Conf hack day that Joe just sent out an e-mail about?
>>> >>
>>> >> Yes, it would -- I will put something in the wiki topic. It will
>>> also be
>>> >> part of my talk on Monday -- How to Run from Zombie which include
>>> some of
>>> >> my opinions on the topic.
>>> >>
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> Thanks,
>>> >>>> -John
>>> >>>>
>>> >>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>> >> mike.tutkowski@solidfire.com>
>>> >>>> wrote:
>>> >>>>
>>> >>>>> So, here's my suggestion for 4.2:
>>> >>>>>
>>> >>>>> Accept the values as they are currently required (four new fields
>>> for
>>> >>>> Wei's
>>> >>>>> feature or two new fields for mine).
>>> >>>>>
>>> >>>>> The Add Disk Offering dialog needs three new radio buttons:
>>> >>>>>
>>> >>>>> 1) No QoS
>>> >>>>>
>>> >>>>> 2) Hypervisor QoS
>>> >>>>>
>>> >>>>> 3) Storage Qos
>>> >>>>>
>>> >>>>> The admin needs to specify storage tags that only map to storage
>>> that
>>> >>>>> supports Storage QoS.
>>> >>>>>
>>> >>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>> >>>> hypervisors
>>> >>>>> in use support the new fields, they may not be enforced.
>>> >>>>>
>>> >>>>> Post 4.3:
>>> >>>>>
>>> >>>>> Come up with a way to more generally enter these parameters
>>> (probably
>>> >>>> just
>>> >>>>> key/value pairs sent to the drivers).
>>> >>>>>
>>> >>>>> Have the drivers expose their feature set so the allocators can
>>> >> consider
>>> >>>>> them more fully and throw an exception if there is not a sufficient
>>> >>>> match.
>>> >>>>>
>>> >>>>>
>>> >>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>> >>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>
>>> >>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>>> >>>> burden
>>> >>>>>> on the admin to configure a Disk Offering in a way that makes
>>> sense.
>>> >> For
>>> >>>>>> example, if he wants to use storage QoS with supported Min and Max
>>> >>>> values,
>>> >>>>>> he'll have to put in a storage tag that references the SolidFire
>>> >> primary
>>> >>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then
>>> he's
>>> >>>> not
>>> >>>>>> going to get the Min and Max feature. We could add help text to
>>> the
>>> >>>> pop-up
>>> >>>>>> dialog that's displayed when you click in the Min and Max text
>>> fields.
>>> >>>>>>
>>> >>>>>> Same idea for Wei's feature.
>>> >>>>>>
>>> >>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>> >>>>>> approach for post 4.2.
>>> >>>>>>
>>> >>>>>> Maybe in the future we could have the drivers advertise their
>>> >>>> capabilities
>>> >>>>>> and if the allocator feels a request is not being satisfied (say
>>> Min
>>> >> was
>>> >>>>>> entered, but it not's supported by any storage plug-in) it can
>>> throw
>>> >> an
>>> >>>>>> exception.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>> >>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>
>>> >>>>>>> Comments below in red.
>>> >>>>>>>
>>> >>>>>>> Thanks
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>> jburwell@basho.com>
>>> >>>> wrote:
>>> >>>>>>>
>>> >>>>>>>> Mike,
>>> >>>>>>>>
>>> >>>>>>>> Please see my comment in-line below.
>>> >>>>>>>>
>>> >>>>>>>> Thanks,
>>> >>>>>>>> -John
>>> >>>>>>>>
>>> >>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> Hi John,
>>> >>>>>>>>>
>>> >>>>>>>>> I've put comments below in red.
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks!
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>> jburwell@basho.com
>>> >>>
>>> >>>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>> Mike,
>>> >>>>>>>>>>
>>> >>>>>>>>>> First and foremost, we must ensure that these two features are
>>> >>>>>>>> mutually
>>> >>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>> >>>>>>>> contains both
>>> >>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>> >>>> non-deterministic
>>> >>>>>>>>>> operations.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> Agreed
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>>> >>>> oriented
>>> >>>>>>>>>> solves the problem in short term. As I understand storage
>>> tags,
>>> >> we
>>> >>>>>>>> have no
>>> >>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>> >>>>>>>> necessarily
>>> >>>>>>>>>> intending that we implement this allocation model in 4.3, but
>>> >>>> instead,
>>> >>>>>>>>>> determine if this type model would be one we would want to
>>> support
>>> >>>> in
>>> >>>>>>>> the
>>> >>>>>>>>>> future. If so, I would encourage us to ensure that the data
>>> model
>>> >>>> and
>>> >>>>>>>>>> current implementation would not preclude evolution in that
>>> >>>>>>>> direction. My
>>> >>>>>>>>>> view is that this type of allocation model is what user's
>>> expect
>>> >> of
>>> >>>>>>>> "cloud"
>>> >>>>>>>>>> systems -- selecting the best available resource set to
>>> fulfill a
>>> >>>>>>>> set of
>>> >>>>>>>>>> system requirements.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> I believe we have meet your requirement here in that what we've
>>> >>>>>>>> implemented
>>> >>>>>>>>> should not make refinement difficult in the future. If we don't
>>> >>>> modify
>>> >>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>>> simple
>>> >>>>>>>> changes
>>> >>>>>>>>> to enhance the current functioning of the system.
>>> >>>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> Looking through both patches, I have to say that the aggregated
>>> >> result
>>> >>>>>>>> seems a bit confusing. There are six new attributes for
>>> throttled
>>> >>>> I/O and
>>> >>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
>>> is
>>> >> not
>>> >>>>>>>> technical, but rather, about maintainability. At minimum,
>>> Javadoc
>>> >>>> should
>>> >>>>>>>> be added explaining the two sets of attributes and their mutual
>>> >>>> exclusion.
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>> exclusion.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> The other part that is interesting is that throttled I/O
>>> provides
>>> >> both
>>> >>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
>>> uses
>>> >> a
>>> >>>>>>>> range. In order to select the best available resource to
>>> fulfill a
>>> >>>> QoS, we
>>> >>>>>>>> would need to have the QoS expression normalized in terms of
>>> units
>>> >>>> (IOPS or
>>> >>>>>>>> bytes) and their expression (rate vs. range). If we want to
>>> >> achieve a
>>> >>>>>>>> model like I described, I think we would need to resolve this
>>> issue
>>> >>>> in 4.2
>>> >>>>>>>> to ensure a viable migration path.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I think we're not likely to be able to normalize the input for
>>> 4.2.
>>> >>>> Plus
>>> >>>>>>> people probably want to input the data in terms they're familiar
>>> with
>>> >>>> for
>>> >>>>>>> the products in question.
>>> >>>>>>>
>>> >>>>>>> Ideally we would fix the way we do storage tagging and let the
>>> user
>>> >>>> send
>>> >>>>>>> key/value pairs to each vendor that could be selected due to a
>>> given
>>> >>>>>>> storage tag. I'm still not sure that would solve it because what
>>> >>>> happens if
>>> >>>>>>> you change the storage tag of a given Primary Storage after
>>> having
>>> >>>> created
>>> >>>>>>> a Disk Offering?
>>> >>>>>>>
>>> >>>>>>> Basically storage tagging is kind of a mess and we should
>>> re-think
>>> >> it.
>>> >>>>>>>
>>> >>>>>>> Also, we need to have a way for the drivers to expose their
>>> supported
>>> >>>>>>> feature sets so the allocators can make good choices.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> As I think through the implications of these requirements and
>>> >>>> reflect
>>> >>>>>>>> on
>>> >>>>>>>>>> the reviews, I don't understand why they haven't already
>>> impacted
>>> >>>> the
>>> >>>>>>>>>> allocators and planners. As it stands, the current
>>> provisioned
>>> >> IOPS
>>> >>>>>>>> has no
>>> >>>>>>>>>> checks to ensure that the volumes are allocated to devices
>>> that
>>> >> have
>>> >>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>> understand
>>> >>>> the
>>> >>>>>>>>>> current patch, we can overcommit storage resources --
>>> potentially
>>> >>>>>>>> causing
>>> >>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>>> to me
>>> >>>>>>>> that a
>>> >>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>> maximum
>>> >>>> IOPS
>>> >>>>>>>> which
>>> >>>>>>>>>> it can support and some type of overcommitment factor. This
>>> >>>>>>>> information
>>> >>>>>>>>>> should be used by the storage allocators to determine the
>>> device
>>> >>>> best
>>> >>>>>>>> able
>>> >>>>>>>>>> to support the resources needs of a volume. It seems that a
>>> >> similar
>>> >>>>>>>> set of
>>> >>>>>>>>>> considerations would need to be added to the Hypervisor layer
>>> >> which
>>> >>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> Yeah, for this first release, we just followed the path that
>>> was
>>> >>>>>>>> previously
>>> >>>>>>>>> established for other properties you see on dialogs in CS: Just
>>> >>>> because
>>> >>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>>> >> supports
>>> >>>>>>>> them.
>>> >>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>> storage
>>> >> tag
>>> >>>>>>>> that
>>> >>>>>>>>> confines the deployment only to storage that supports the
>>> selected
>>> >>>>>>>>> features. This is not ideal, but it's kind of the way
>>> CloudStack
>>> >>>> works
>>> >>>>>>>>> today.
>>> >>>>>>>>
>>> >>>>>>>> I understand the tag functionality, and the need for the
>>> >> administrator
>>> >>>>>>>> to very carefully construct offerings. My concern is that we
>>> over
>>> >>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>> >>>> illustration,
>>> >>>>>>>> let's say we have a SolidFire, and the max IOPS for that device
>>> is
>>> >>>> 100000.
>>> >>>>>>>> We also know that we can safely oversubscribe by 50%.
>>> Therefore,
>>> >> we
>>> >>>>>>>> need to ensure that we don't allocate more than 150,000
>>> guaranteed
>>> >>>> IOPS
>>> >>>>>>>> from that device. Intuitively, it seems like the DataStore
>>> >>>> configuration
>>> >>>>>>>> should have a max assignable IOPS and overcommitment factor.
>>> As we
>>> >>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>> guarantee
>>> >>>> more IOPS
>>> >>>>>>>> exceed the configured maximum for a DataStore. Does that make
>>> >> sense?
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> I think that's a good idea for a future enhancement. I'm not even
>>> >> sure
>>> >>>> I
>>> >>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>>> have
>>> >> to
>>> >>>> get
>>> >>>>>>> all of the min values for all of the volumes on the SAN and total
>>> >> them
>>> >>>> up,
>>> >>>>>>> I suppose, and subtract it from the total (user facing) supported
>>> >> IOPS
>>> >>>> of
>>> >>>>>>> the system.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>> resources
>>> >>>> to
>>> >>>>>>>> be
>>> >>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>> >> provisioned
>>> >>>>>>>> IOPS, I
>>> >>>>>>>>>> think a side effect of the current implementation is SolidFire
>>> >>>> volumes
>>> >>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>>> >> seems
>>> >>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>>> with
>>> >> QoS
>>> >>>>>>>> VMs.
>>> >>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>>> starve
>>> >>>> out
>>> >>>>>>>> the
>>> >>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>>> >> collocated
>>> >>>>>>>> VMs
>>> >>>>>>>>>> from being fulfilled.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
>>> not
>>> >>>>>>>> specify
>>> >>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>> >>>> defaults
>>> >>>>>>>> we
>>> >>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you
>>> will
>>> >>>>>>>> have
>>> >>>>>>>>> your Mins met.
>>> >>>>>>>>>
>>> >>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>> exclusions
>>> >>>>>>>> list,
>>> >>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>>> >>>> volume
>>> >>>>>>>> to
>>> >>>>>>>>> storage with these following tags).
>>> >>>>>>>>
>>> >>>>>>>> I don't like the idea of a storage tags exclusion list as it
>>> would
>>> >>>>>>>> complicate component assembly. It would require a storage
>>> plugin to
>>> >>>>>>>> anticipate all of the possible component assemblies and
>>> determine
>>> >> the
>>> >>>>>>>> invalid relationships. I prefer that drivers express their
>>> >>>> capabilities
>>> >>>>>>>> which can be matched to a set of requested requirements.
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> I'm not sure why a storage plug-in would care about inclusion or
>>> >>>>>>> exclusion lists. It just needs to advertise its functionality in
>>> a
>>> >> way
>>> >>>> the
>>> >>>>>>> allocator understands.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>>> >> limits
>>> >>>>>>>> IOPS,
>>> >>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>>> with
>>> >>>>>>>> guaranteed
>>> >>>>>>>>> minimum IOPS does this.
>>> >>>>>>>>
>>> >>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>>> >> exist.
>>> >>>>>>>> However, I am concerned with the more general case as we
>>> supported
>>> >>>> more
>>> >>>>>>>> devices with provisioned IOPS.
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>>> values
>>> >> to
>>> >>>>>>> drivers. Min and Max and pretty industry standard for provisioned
>>> >>>> IOPS, but
>>> >>>>>>> what if you break them out by read and write or do something
>>> else? We
>>> >>>> need
>>> >>>>>>> a more general mechanism.
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>>> I/O
>>> >> and
>>> >>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> Agreed
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>> We also need to understand the implications of these QoS
>>> >> guarantees
>>> >>>> on
>>> >>>>>>>>>> operation of the system to ensure that the underlying hardware
>>> >>>>>>>> resources
>>> >>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>>> forced
>>> >> to
>>> >>>>>>>> make
>>> >>>>>>>>>> compromises to achieve these goals, and refine the
>>> implementation
>>> >> in
>>> >>>>>>>> 4.3.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> I agree, John. I also think you've come up with some great
>>> ideas
>>> >> for
>>> >>>>>>>> 4.3. :)
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> Thanks,
>>> >>>>>>>>>> -John
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>> >>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> Hi,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>>> storage
>>> >>>>>>>> tags).
>>> >>>>>>>>>> I
>>> >>>>>>>>>>> believe John was just throwing out an idea that - in
>>> addition to
>>> >>>>>>>> storage
>>> >>>>>>>>>>> tags - we could look into these allocators (storage QoS being
>>> >>>>>>>> preferred,
>>> >>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>>> >> hypervisor
>>> >>>>>>>> QoS
>>> >>>>>>>>>> is).
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I think John's concern is that you can enter in values for
>>> Wei's
>>> >>>> and
>>> >>>>>>>> my
>>> >>>>>>>>>>> feature that are not honored by other vendors (at least
>>> yet), so
>>> >> he
>>> >>>>>>>> was
>>> >>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
>>> >> prefer
>>> >>>>>>>> these
>>> >>>>>>>>>>> vendors when these fields are filled in. As it stands today
>>> in
>>> >>>>>>>>>> CloudStack,
>>> >>>>>>>>>>> we already have this kind of an issue with other features
>>> (fields
>>> >>>> in
>>> >>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
>>> post
>>> >>>> 4.2
>>> >>>>>>>> we
>>> >>>>>>>>>>> could look into generic name/value pairs (this is how
>>> OpenStack
>>> >>>>>>>> addresses
>>> >>>>>>>>>>> the issue).
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
>>> >> code
>>> >>>>>>>>>> freeze)
>>> >>>>>>>>>>> to go too deeply down that path in 4.2.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>>> fields
>>> >>>>>>>> and my
>>> >>>>>>>>>>> fields as is, make sure only one or the other feature has
>>> data
>>> >>>>>>>> entered
>>> >>>>>>>>>> for
>>> >>>>>>>>>>> it (or neither), and call it good for 4.2.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Then let's step back and look into a more general-purpose
>>> design
>>> >>>>>>>> that can
>>> >>>>>>>>>>> be applied throughout CloudStack where we have these
>>> situations.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> What do you think?
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>> >> jburwell@basho.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I just published my review @
>>> >> https://reviews.apache.org/r/11479/.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I apologize for the delay,
>>> >>>>>>>>>>>> -John
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> No problem, John.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> I still want to re-review it by myself before coming up
>>> with a
>>> >>>> new
>>> >>>>>>>>>> patch
>>> >>>>>>>>>>>>> file.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>> checked
>>> >>>> in
>>> >>>>>>>> and
>>> >>>>>>>>>>>>> merge those into mine before generating a new patch file?
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>> >>>> jburwell@basho.com
>>> >>>>>>>>>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>> >>>> offline
>>> >>>>>>>> ATM,
>>> >>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Sorry for the oversight,
>>> >>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>> >>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>> this :)
>>> >>>> ),
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>> areas
>>> >> we
>>> >>>>>>>> wanted
>>> >>>>>>>>>>>>>>> addressed.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>> >> afternoon,
>>> >>>>>>>> then
>>> >>>>>>>>>>>> send
>>> >>>>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>> another patch.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Sure, that sounds good.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> >>>>>>>> ustcweizhou@gmail.com>
>>> >>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Hi Mike,
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>>> Java
>>> >>>>>>>> code,
>>> >>>>>>>>>>>>>> except
>>> >>>>>>>>>>>>>>>>> the cloudstack UI.
>>> >>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>> branch
>>> >>>> into
>>> >>>>>>>>>>>> master
>>> >>>>>>>>>>>>>>>>> this
>>> >>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.com
>>> >
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Hey John,
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>> object_store
>>> >>>>>>>> branch,
>>> >>>>>>>>>> but
>>> >>>>>>>>>>>> -
>>> >>>>>>>>>>>>>> as
>>> >>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>> >>>>>>>> SolidFire
>>> >>>>>>>>>>>> branch
>>> >>>>>>>>>>>>>>>>> into
>>> >>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>> >>>> master.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>> >> merge
>>> >>>>>>>>>>>> strategy.
>>> >>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>> >>>>>>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>> >>>>>>>>>> disk_io_throttling,
>>> >>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>> of
>>> >>>>>>>>>> overlapping
>>> >>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>> >>>> patches
>>> >>>>>>>> to
>>> >>>>>>>>>> be
>>> >>>>>>>>>>>> as
>>> >>>>>>>>>>>>>>>>>>> follows:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>> >>>> SolidFire
>>> >>>>>>>> are
>>> >>>>>>>>>>>>>>>>> additive
>>> >>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>> >> dependency
>>> >>>>>>>>>> between
>>> >>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>> we
>>> >> can
>>> >>>>>>>>>>>> determine
>>> >>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>> into
>>> >> his
>>> >>>>>>>> branch
>>> >>>>>>>>>>>>>>>>> before
>>> >>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>> after his
>>> >>>>>>>> merge
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>> code in
>>> >>>> the
>>> >>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>> >> plug-in,
>>> >>>>>>>> which
>>> >>>>>>>>>>>>>>>>>>> currently
>>> >>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>> >> (although
>>> >>>>>>>> it
>>> >>>>>>>>>> does
>>> >>>>>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>>>> know
>>> >>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>> actually
>>> >> in
>>> >>>>>>>> use))
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>> the
>>> >> url
>>> >>>>>>>> field
>>> >>>>>>>>>>>> (if
>>> >>>>>>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>>> in
>>> >> the
>>> >>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>> hypervisor
>>> >> in
>>> >>>>>>>>>>>>>>>>> question, we
>>> >>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>> place
>>> >> of
>>> >>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>> >> checks
>>> >>>>>>>> for
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> managed
>>> >>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>> >> hypervisor
>>> >>>>>>>> data
>>> >>>>>>>>>>>>>>>>>>> structure is
>>> >>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>> executes to
>>> >>>>>>>> attach
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>> volume.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>> >>>> detach a
>>> >>>>>>>>>>>> volume,
>>> >>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> >>>>>>>> structure
>>> >>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>>>> removed.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>>> in
>>> >> 4.2
>>> >>>>>>>>>> unless
>>> >>>>>>>>>>>>>>>>> it is
>>> >>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>>> we
>>> >>>> have
>>> >>>>>>>> some
>>> >>>>>>>>>>>>>>>>> idea,
>>> >>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>> avoiding
>>> >>>>>>>>>>>> implementation
>>> >>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>> >> field.
>>> >>>> I
>>> >>>>>>>>>> think
>>> >>>>>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>> regarding
>>> >>>> the
>>> >>>>>>>>>>>>>>>>> expected
>>> >>>>>>>>>>>>>>>>>>> format
>>> >>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>> are
>>> >>>> not
>>> >>>>>>>>>>>>>>>>> returned on
>>> >>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>> >> listStoragePool
>>> >>>>>>>>>>>> response.
>>> >>>>>>>>>>>>>>>>>> Why
>>> >>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>> useful
>>> >>>> for
>>> >>>>>>>>>>>> clients
>>> >>>>>>>>>>>>>>>>> to
>>> >>>>>>>>>>>>>>>>>> be
>>> >>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>> field."
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>> here.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>> >>>> Offering-by-Disk
>>> >>>>>>>>>>>> Offering
>>> >>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>> to be
>>> >>>>>>>> able to
>>> >>>>>>>>>>>>>>>>>> associate
>>> >>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>> disk_offering_details
>>> >>>> table.
>>> >>>>>>>>>> Maybe
>>> >>>>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>>> could
>>> >>>>>>>>>>>>>>>>>>>>>> go there?
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>> IOPS
>>> >> in
>>> >>>>>>>> the
>>> >>>>>>>>>> GUI
>>> >>>>>>>>>>>>>>>>> if
>>> >>>>>>>>>>>>>>>>>>> it's
>>> >>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>>> DB.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>> cloud<
>>> >> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>> >
>>> >>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> --
>>> >>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>> o: 303.746.7302
>>> >>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>> *â„¢*
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> *Mike Tutkowski*
>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>> o: 303.746.7302
>>> >>>>>>> Advancing the way the world uses the cloud<
>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>> *â„¢*
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> *Mike Tutkowski*
>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>> o: 303.746.7302
>>> >>>>>> Advancing the way the world uses the cloud<
>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>> *â„¢*
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> *Mike Tutkowski*
>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>> e: mike.tutkowski@solidfire.com
>>> >>>>> o: 303.746.7302
>>> >>>>> Advancing the way the world uses the
>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>> *â„¢*
>>> >>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Mike Tutkowski*
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>> e: mike.tutkowski@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the
>>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>> *â„¢*
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
1) We want number of IOPS currently supported by the SAN.
2) We want the number of IOPS that are committed (sum of min IOPS for each
volume).
We could do the following to keep track of IOPS:
The plug-in could have a timer thread that goes off every, say, 1 minute.
It could query the SAN for the number of nodes that make up the SAN and
multiple this by 50,000. This is essentially the number of supported IOPS
of the SAN.
The next API call could be to get all of the volumes on the SAN. Iterate
through them all and add up their min IOPS values. This is the number of
IOPS the SAN is committed to.
These two numbers can then be updated in the storage_pool table (a column
for each value).
The allocators can get these values as needed (and they would be as
accurate as the last time the thread asked the SAN for this info).
These two fields, the min IOPS of the volume to create, and the overcommit
ratio of the plug-in would tell the allocator if it can select the given
storage pool.
What do you think?
On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> "As I mentioned previously, I am very reluctant for any feature to come
> into master that can exhaust resources."
>
> Just wanted to mention that, worst case, the SAN would fail creation of
> the volume before allowing a new volume to break the system.
>
>
> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Hi John,
>>
>> Are you thinking we add a column on to the storage pool table,
>> IOPS_Count, where we add and subtract committed IOPS?
>>
>> That is easy enough.
>>
>> How do you want to determine what the SAN is capable of supporting IOPS
>> wise? Remember we're dealing with a dynamic SAN here...as you add storage
>> nodes to the cluster, the number of IOPS increases. Do we have a thread we
>> can use to query external devices like this SAN to update the supported
>> number of IOPS?
>>
>> Also, how do you want to enforce the IOPS limit? Do we pass in an
>> overcommit ration to the plug-in when it's created? We would need to store
>> this in the storage_pool table, as well, I believe.
>>
>> We should also get Wei involved in this as his feature will need similar
>> functionality.
>>
>> Also, we should do this FAST as we have only two weeks left and many of
>> us will be out for several days for the CS Collab Conference.
>>
>> Thanks
>>
>>
>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jb...@basho.com>wrote:
>>
>>> Mike,
>>>
>>> Querying the SAN only indicates the number of IOPS currently in use.
>>> The allocator needs to check the number of IOPS committed which is tracked
>>> by CloudStack. For 4.2, we should be able to query the number of IOPS
>>> committed to a DataStore, and determine whether or not the number requested
>>> can be fulfilled by that device. It seems to be that a
>>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>>> DataStore's that don't support provisioned IOPS would return null.
>>>
>>> As I mentioned previously, I am very reluctant for any feature to come
>>> into master that can exhaust resources.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> > Yeah, I'm not sure I could come up with anything near an accurate
>>> > assessment of how many IOPS are currently available on the SAN (or
>>> even a
>>> > total number that are available for volumes). Not sure if there's yet
>>> an
>>> > API call for that.
>>> >
>>> > If I did know this number (total number of IOPS supported by the SAN),
>>> we'd
>>> > still have to keep track of the total number of volumes we've created
>>> from
>>> > CS on the SAN in terms of their IOPS. Also, if an admin issues an API
>>> call
>>> > directly to the SAN to tweak the number of IOPS on a given volume or
>>> set of
>>> > volumes (not supported from CS, but supported via the SolidFire API),
>>> our
>>> > numbers in CS would be off.
>>> >
>>> > I'm thinking verifying sufficient number of IOPS is a really good idea
>>> for
>>> > a future release.
>>> >
>>> > For 4.2 I think we should stick to having the allocator detect if
>>> storage
>>> > QoS is desired and if the storage pool in question supports that
>>> feature.
>>> >
>>> > If you really are over provisioned on your SAN in terms of IOPS or
>>> > capacity, the SAN can let the admin know in several different ways
>>> (e-mail,
>>> > SNMP, GUI).
>>> >
>>> >
>>> > On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>> >
>>> >> Mike,
>>> >>
>>> >> Please see my comments in-line below.
>>> >>
>>> >> Thanks,
>>> >> -John
>>> >>
>>> >> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>> >> wrote:
>>> >>
>>> >>> Comments below in red.
>>> >>>
>>> >>> Thanks
>>> >>>
>>> >>>
>>> >>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>>> >> wrote:
>>> >>>
>>> >>>> Mike,
>>> >>>>
>>> >>>> Overall, I agree with the steps to below for 4.2. However, we may
>>> want
>>> >> to
>>> >>>> throw an exception if we can not fulfill a requested QoS. If the
>>> user
>>> >> is
>>> >>>> expecting that the hypervisor will provide a particular QoS, and
>>> that is
>>> >>>> not possible, it seems like we should inform them rather than
>>> silently
>>> >>>> ignoring their request.
>>> >>>>
>>> >>>
>>> >>> Sure, that sounds reasonable.
>>> >>>
>>> >>> We'd have to come up with some way for the allocators to know about
>>> the
>>> >>> requested storage QoS and then query the candidate drivers.
>>> >>>
>>> >>> Any thoughts on how we might do that?
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> To collect my thoughts from previous parts of the thread, I am
>>> >>>> uncomfortable with the idea that the management server can
>>> overcommit a
>>> >>>> resource. You had mentioned querying the device for available IOPS.
>>> >> While
>>> >>>> that would be nice, it seems like we could fall back to a max IOPS
>>> and
>>> >>>> overcommit factor manually calculated and entered by the
>>> >>>> administrator/operator. I think such threshold and allocation rails
>>> >> should
>>> >>>> be added for both provisioned IOPS and throttled I/O -- it is a
>>> basic
>>> >>>> feature of any cloud orchestration platform.
>>> >>>>
>>> >>>
>>> >>> Are you thinking this ability would make it into 4.2? Just curious
>>> what
>>> >>> release we're talking about here. For the SolidFire SAN, you might
>>> have,
>>> >>> say, four separate storage nodes to start (200,000 IOPS) and then
>>> later
>>> >> add
>>> >>> a new node (now you're at 250,000 IOPS). CS would have to have a way
>>> to
>>> >>> know that the number of supported IOPS has increased.
>>> >>
>>> >> Yes, I think we need some *basic*/conservative rails in 4.2. For
>>> example,
>>> >> we may only support expanding capacity in 4.2, and not handle any
>>> >> re-balance scenarios -- node failure, addition, etc. Extrapolating
>>> a
>>> >> bit, the throttled I/O enhancement seems like it needs a similar set
>>> of
>>> >> rails defined per host.
>>> >>
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>> >>>> implementation specific manner. I think we need to arrive at a
>>> general
>>> >>>> model that can be exploited by the allocators and planners. We
>>> should
>>> >>>> restrict implementation specific key-value pairs (call them details,
>>> >>>> extended data, whatever) to information that is unique to the
>>> driver and
>>> >>>> would provide no useful information to the management server's
>>> >>>> orchestration functions. A resource QoS does not seem to fit those
>>> >>>> criteria.
>>> >>>>
>>> >>>
>>> >>> I wonder if this would be a good discussion topic for Sunday's CS
>>> Collab
>>> >>> Conf hack day that Joe just sent out an e-mail about?
>>> >>
>>> >> Yes, it would -- I will put something in the wiki topic. It will
>>> also be
>>> >> part of my talk on Monday -- How to Run from Zombie which include
>>> some of
>>> >> my opinions on the topic.
>>> >>
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> Thanks,
>>> >>>> -John
>>> >>>>
>>> >>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>>> >> mike.tutkowski@solidfire.com>
>>> >>>> wrote:
>>> >>>>
>>> >>>>> So, here's my suggestion for 4.2:
>>> >>>>>
>>> >>>>> Accept the values as they are currently required (four new fields
>>> for
>>> >>>> Wei's
>>> >>>>> feature or two new fields for mine).
>>> >>>>>
>>> >>>>> The Add Disk Offering dialog needs three new radio buttons:
>>> >>>>>
>>> >>>>> 1) No QoS
>>> >>>>>
>>> >>>>> 2) Hypervisor QoS
>>> >>>>>
>>> >>>>> 3) Storage Qos
>>> >>>>>
>>> >>>>> The admin needs to specify storage tags that only map to storage
>>> that
>>> >>>>> supports Storage QoS.
>>> >>>>>
>>> >>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>> >>>> hypervisors
>>> >>>>> in use support the new fields, they may not be enforced.
>>> >>>>>
>>> >>>>> Post 4.3:
>>> >>>>>
>>> >>>>> Come up with a way to more generally enter these parameters
>>> (probably
>>> >>>> just
>>> >>>>> key/value pairs sent to the drivers).
>>> >>>>>
>>> >>>>> Have the drivers expose their feature set so the allocators can
>>> >> consider
>>> >>>>> them more fully and throw an exception if there is not a sufficient
>>> >>>> match.
>>> >>>>>
>>> >>>>>
>>> >>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>> >>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>
>>> >>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>>> >>>> burden
>>> >>>>>> on the admin to configure a Disk Offering in a way that makes
>>> sense.
>>> >> For
>>> >>>>>> example, if he wants to use storage QoS with supported Min and Max
>>> >>>> values,
>>> >>>>>> he'll have to put in a storage tag that references the SolidFire
>>> >> primary
>>> >>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then
>>> he's
>>> >>>> not
>>> >>>>>> going to get the Min and Max feature. We could add help text to
>>> the
>>> >>>> pop-up
>>> >>>>>> dialog that's displayed when you click in the Min and Max text
>>> fields.
>>> >>>>>>
>>> >>>>>> Same idea for Wei's feature.
>>> >>>>>>
>>> >>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>> >>>>>> approach for post 4.2.
>>> >>>>>>
>>> >>>>>> Maybe in the future we could have the drivers advertise their
>>> >>>> capabilities
>>> >>>>>> and if the allocator feels a request is not being satisfied (say
>>> Min
>>> >> was
>>> >>>>>> entered, but it not's supported by any storage plug-in) it can
>>> throw
>>> >> an
>>> >>>>>> exception.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>> >>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>
>>> >>>>>>> Comments below in red.
>>> >>>>>>>
>>> >>>>>>> Thanks
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
>>> jburwell@basho.com>
>>> >>>> wrote:
>>> >>>>>>>
>>> >>>>>>>> Mike,
>>> >>>>>>>>
>>> >>>>>>>> Please see my comment in-line below.
>>> >>>>>>>>
>>> >>>>>>>> Thanks,
>>> >>>>>>>> -John
>>> >>>>>>>>
>>> >>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> Hi John,
>>> >>>>>>>>>
>>> >>>>>>>>> I've put comments below in red.
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks!
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>>> jburwell@basho.com
>>> >>>
>>> >>>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>> Mike,
>>> >>>>>>>>>>
>>> >>>>>>>>>> First and foremost, we must ensure that these two features are
>>> >>>>>>>> mutually
>>> >>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>> >>>>>>>> contains both
>>> >>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>> >>>> non-deterministic
>>> >>>>>>>>>> operations.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> Agreed
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>>> >>>> oriented
>>> >>>>>>>>>> solves the problem in short term. As I understand storage
>>> tags,
>>> >> we
>>> >>>>>>>> have no
>>> >>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>> >>>>>>>> necessarily
>>> >>>>>>>>>> intending that we implement this allocation model in 4.3, but
>>> >>>> instead,
>>> >>>>>>>>>> determine if this type model would be one we would want to
>>> support
>>> >>>> in
>>> >>>>>>>> the
>>> >>>>>>>>>> future. If so, I would encourage us to ensure that the data
>>> model
>>> >>>> and
>>> >>>>>>>>>> current implementation would not preclude evolution in that
>>> >>>>>>>> direction. My
>>> >>>>>>>>>> view is that this type of allocation model is what user's
>>> expect
>>> >> of
>>> >>>>>>>> "cloud"
>>> >>>>>>>>>> systems -- selecting the best available resource set to
>>> fulfill a
>>> >>>>>>>> set of
>>> >>>>>>>>>> system requirements.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> I believe we have meet your requirement here in that what we've
>>> >>>>>>>> implemented
>>> >>>>>>>>> should not make refinement difficult in the future. If we don't
>>> >>>> modify
>>> >>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>>> simple
>>> >>>>>>>> changes
>>> >>>>>>>>> to enhance the current functioning of the system.
>>> >>>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> Looking through both patches, I have to say that the aggregated
>>> >> result
>>> >>>>>>>> seems a bit confusing. There are six new attributes for
>>> throttled
>>> >>>> I/O and
>>> >>>>>>>> two for provisioned IOPS with no obvious grouping. My concern
>>> is
>>> >> not
>>> >>>>>>>> technical, but rather, about maintainability. At minimum,
>>> Javadoc
>>> >>>> should
>>> >>>>>>>> be added explaining the two sets of attributes and their mutual
>>> >>>> exclusion.
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> I agree: We need JavaDoc to explain them and their mutual
>>> exclusion.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> The other part that is interesting is that throttled I/O
>>> provides
>>> >> both
>>> >>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
>>> uses
>>> >> a
>>> >>>>>>>> range. In order to select the best available resource to
>>> fulfill a
>>> >>>> QoS, we
>>> >>>>>>>> would need to have the QoS expression normalized in terms of
>>> units
>>> >>>> (IOPS or
>>> >>>>>>>> bytes) and their expression (rate vs. range). If we want to
>>> >> achieve a
>>> >>>>>>>> model like I described, I think we would need to resolve this
>>> issue
>>> >>>> in 4.2
>>> >>>>>>>> to ensure a viable migration path.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I think we're not likely to be able to normalize the input for
>>> 4.2.
>>> >>>> Plus
>>> >>>>>>> people probably want to input the data in terms they're familiar
>>> with
>>> >>>> for
>>> >>>>>>> the products in question.
>>> >>>>>>>
>>> >>>>>>> Ideally we would fix the way we do storage tagging and let the
>>> user
>>> >>>> send
>>> >>>>>>> key/value pairs to each vendor that could be selected due to a
>>> given
>>> >>>>>>> storage tag. I'm still not sure that would solve it because what
>>> >>>> happens if
>>> >>>>>>> you change the storage tag of a given Primary Storage after
>>> having
>>> >>>> created
>>> >>>>>>> a Disk Offering?
>>> >>>>>>>
>>> >>>>>>> Basically storage tagging is kind of a mess and we should
>>> re-think
>>> >> it.
>>> >>>>>>>
>>> >>>>>>> Also, we need to have a way for the drivers to expose their
>>> supported
>>> >>>>>>> feature sets so the allocators can make good choices.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> As I think through the implications of these requirements and
>>> >>>> reflect
>>> >>>>>>>> on
>>> >>>>>>>>>> the reviews, I don't understand why they haven't already
>>> impacted
>>> >>>> the
>>> >>>>>>>>>> allocators and planners. As it stands, the current
>>> provisioned
>>> >> IOPS
>>> >>>>>>>> has no
>>> >>>>>>>>>> checks to ensure that the volumes are allocated to devices
>>> that
>>> >> have
>>> >>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>>> understand
>>> >>>> the
>>> >>>>>>>>>> current patch, we can overcommit storage resources --
>>> potentially
>>> >>>>>>>> causing
>>> >>>>>>>>>> none of the QoS obligations from being fulfilled. It seems
>>> to me
>>> >>>>>>>> that a
>>> >>>>>>>>>> DataStore supporting provisioned IOPS should express the
>>> maximum
>>> >>>> IOPS
>>> >>>>>>>> which
>>> >>>>>>>>>> it can support and some type of overcommitment factor. This
>>> >>>>>>>> information
>>> >>>>>>>>>> should be used by the storage allocators to determine the
>>> device
>>> >>>> best
>>> >>>>>>>> able
>>> >>>>>>>>>> to support the resources needs of a volume. It seems that a
>>> >> similar
>>> >>>>>>>> set of
>>> >>>>>>>>>> considerations would need to be added to the Hypervisor layer
>>> >> which
>>> >>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> Yeah, for this first release, we just followed the path that
>>> was
>>> >>>>>>>> previously
>>> >>>>>>>>> established for other properties you see on dialogs in CS: Just
>>> >>>> because
>>> >>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>>> >> supports
>>> >>>>>>>> them.
>>> >>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>>> storage
>>> >> tag
>>> >>>>>>>> that
>>> >>>>>>>>> confines the deployment only to storage that supports the
>>> selected
>>> >>>>>>>>> features. This is not ideal, but it's kind of the way
>>> CloudStack
>>> >>>> works
>>> >>>>>>>>> today.
>>> >>>>>>>>
>>> >>>>>>>> I understand the tag functionality, and the need for the
>>> >> administrator
>>> >>>>>>>> to very carefully construct offerings. My concern is that we
>>> over
>>> >>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>> >>>> illustration,
>>> >>>>>>>> let's say we have a SolidFire, and the max IOPS for that device
>>> is
>>> >>>> 100000.
>>> >>>>>>>> We also know that we can safely oversubscribe by 50%.
>>> Therefore,
>>> >> we
>>> >>>>>>>> need to ensure that we don't allocate more than 150,000
>>> guaranteed
>>> >>>> IOPS
>>> >>>>>>>> from that device. Intuitively, it seems like the DataStore
>>> >>>> configuration
>>> >>>>>>>> should have a max assignable IOPS and overcommitment factor.
>>> As we
>>> >>>>>>>> allocate volumes and attach VMs, we need to ensure that
>>> guarantee
>>> >>>> more IOPS
>>> >>>>>>>> exceed the configured maximum for a DataStore. Does that make
>>> >> sense?
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> I think that's a good idea for a future enhancement. I'm not even
>>> >> sure
>>> >>>> I
>>> >>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>>> have
>>> >> to
>>> >>>> get
>>> >>>>>>> all of the min values for all of the volumes on the SAN and total
>>> >> them
>>> >>>> up,
>>> >>>>>>> I suppose, and subtract it from the total (user facing) supported
>>> >> IOPS
>>> >>>> of
>>> >>>>>>> the system.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>>> resources
>>> >>>> to
>>> >>>>>>>> be
>>> >>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>>> >> provisioned
>>> >>>>>>>> IOPS, I
>>> >>>>>>>>>> think a side effect of the current implementation is SolidFire
>>> >>>> volumes
>>> >>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>>> >> seems
>>> >>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>>> with
>>> >> QoS
>>> >>>>>>>> VMs.
>>> >>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>>> starve
>>> >>>> out
>>> >>>>>>>> the
>>> >>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>>> >> collocated
>>> >>>>>>>> VMs
>>> >>>>>>>>>> from being fulfilled.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
>>> not
>>> >>>>>>>> specify
>>> >>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>> >>>> defaults
>>> >>>>>>>> we
>>> >>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you
>>> will
>>> >>>>>>>> have
>>> >>>>>>>>> your Mins met.
>>> >>>>>>>>>
>>> >>>>>>>>> It sounds like you're perhaps looking for a storage tags
>>> exclusions
>>> >>>>>>>> list,
>>> >>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>>> >>>> volume
>>> >>>>>>>> to
>>> >>>>>>>>> storage with these following tags).
>>> >>>>>>>>
>>> >>>>>>>> I don't like the idea of a storage tags exclusion list as it
>>> would
>>> >>>>>>>> complicate component assembly. It would require a storage
>>> plugin to
>>> >>>>>>>> anticipate all of the possible component assemblies and
>>> determine
>>> >> the
>>> >>>>>>>> invalid relationships. I prefer that drivers express their
>>> >>>> capabilities
>>> >>>>>>>> which can be matched to a set of requested requirements.
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> I'm not sure why a storage plug-in would care about inclusion or
>>> >>>>>>> exclusion lists. It just needs to advertise its functionality in
>>> a
>>> >> way
>>> >>>> the
>>> >>>>>>> allocator understands.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>>> >> limits
>>> >>>>>>>> IOPS,
>>> >>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system
>>> with
>>> >>>>>>>> guaranteed
>>> >>>>>>>>> minimum IOPS does this.
>>> >>>>>>>>
>>> >>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>>> >> exist.
>>> >>>>>>>> However, I am concerned with the more general case as we
>>> supported
>>> >>>> more
>>> >>>>>>>> devices with provisioned IOPS.
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>>> values
>>> >> to
>>> >>>>>>> drivers. Min and Max and pretty industry standard for provisioned
>>> >>>> IOPS, but
>>> >>>>>>> what if you break them out by read and write or do something
>>> else? We
>>> >>>> need
>>> >>>>>>> a more general mechanism.
>>> >>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled
>>> I/O
>>> >> and
>>> >>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> Agreed
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>> We also need to understand the implications of these QoS
>>> >> guarantees
>>> >>>> on
>>> >>>>>>>>>> operation of the system to ensure that the underlying hardware
>>> >>>>>>>> resources
>>> >>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>>> forced
>>> >> to
>>> >>>>>>>> make
>>> >>>>>>>>>> compromises to achieve these goals, and refine the
>>> implementation
>>> >> in
>>> >>>>>>>> 4.3.
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> I agree, John. I also think you've come up with some great
>>> ideas
>>> >> for
>>> >>>>>>>> 4.3. :)
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> Thanks,
>>> >>>>>>>>>> -John
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>> >>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> Hi,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>>> storage
>>> >>>>>>>> tags).
>>> >>>>>>>>>> I
>>> >>>>>>>>>>> believe John was just throwing out an idea that - in
>>> addition to
>>> >>>>>>>> storage
>>> >>>>>>>>>>> tags - we could look into these allocators (storage QoS being
>>> >>>>>>>> preferred,
>>> >>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>>> >> hypervisor
>>> >>>>>>>> QoS
>>> >>>>>>>>>> is).
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I think John's concern is that you can enter in values for
>>> Wei's
>>> >>>> and
>>> >>>>>>>> my
>>> >>>>>>>>>>> feature that are not honored by other vendors (at least
>>> yet), so
>>> >> he
>>> >>>>>>>> was
>>> >>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
>>> >> prefer
>>> >>>>>>>> these
>>> >>>>>>>>>>> vendors when these fields are filled in. As it stands today
>>> in
>>> >>>>>>>>>> CloudStack,
>>> >>>>>>>>>>> we already have this kind of an issue with other features
>>> (fields
>>> >>>> in
>>> >>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
>>> post
>>> >>>> 4.2
>>> >>>>>>>> we
>>> >>>>>>>>>>> could look into generic name/value pairs (this is how
>>> OpenStack
>>> >>>>>>>> addresses
>>> >>>>>>>>>>> the issue).
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
>>> >> code
>>> >>>>>>>>>> freeze)
>>> >>>>>>>>>>> to go too deeply down that path in 4.2.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>>> fields
>>> >>>>>>>> and my
>>> >>>>>>>>>>> fields as is, make sure only one or the other feature has
>>> data
>>> >>>>>>>> entered
>>> >>>>>>>>>> for
>>> >>>>>>>>>>> it (or neither), and call it good for 4.2.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Then let's step back and look into a more general-purpose
>>> design
>>> >>>>>>>> that can
>>> >>>>>>>>>>> be applied throughout CloudStack where we have these
>>> situations.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> What do you think?
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>>> >> jburwell@basho.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I just published my review @
>>> >> https://reviews.apache.org/r/11479/.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I apologize for the delay,
>>> >>>>>>>>>>>> -John
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> No problem, John.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> I still want to re-review it by myself before coming up
>>> with a
>>> >>>> new
>>> >>>>>>>>>> patch
>>> >>>>>>>>>>>>> file.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>>> checked
>>> >>>> in
>>> >>>>>>>> and
>>> >>>>>>>>>>>>> merge those into mine before generating a new patch file?
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>> >>>> jburwell@basho.com
>>> >>>>>>>>>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>> >>>> offline
>>> >>>>>>>> ATM,
>>> >>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Sorry for the oversight,
>>> >>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>> >>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>>> this :)
>>> >>>> ),
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>>> areas
>>> >> we
>>> >>>>>>>> wanted
>>> >>>>>>>>>>>>>>> addressed.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>>> >> afternoon,
>>> >>>>>>>> then
>>> >>>>>>>>>>>> send
>>> >>>>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>> another patch.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Sure, that sounds good.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> >>>>>>>> ustcweizhou@gmail.com>
>>> >>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Hi Mike,
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>>> Java
>>> >>>>>>>> code,
>>> >>>>>>>>>>>>>> except
>>> >>>>>>>>>>>>>>>>> the cloudstack UI.
>>> >>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>>> branch
>>> >>>> into
>>> >>>>>>>>>>>> master
>>> >>>>>>>>>>>>>>>>> this
>>> >>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkowski@solidfire.com
>>> >
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Hey John,
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
>>> object_store
>>> >>>>>>>> branch,
>>> >>>>>>>>>> but
>>> >>>>>>>>>>>> -
>>> >>>>>>>>>>>>>> as
>>> >>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>> >>>>>>>> SolidFire
>>> >>>>>>>>>>>> branch
>>> >>>>>>>>>>>>>>>>> into
>>> >>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>> >>>> master.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>>> >> merge
>>> >>>>>>>>>>>> strategy.
>>> >>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>> >>>>>>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>> >>>>>>>>>> disk_io_throttling,
>>> >>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number
>>> of
>>> >>>>>>>>>> overlapping
>>> >>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>> >>>> patches
>>> >>>>>>>> to
>>> >>>>>>>>>> be
>>> >>>>>>>>>>>> as
>>> >>>>>>>>>>>>>>>>>>> follows:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>> >>>> SolidFire
>>> >>>>>>>> are
>>> >>>>>>>>>>>>>>>>> additive
>>> >>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>>> >> dependency
>>> >>>>>>>>>> between
>>> >>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph,
>>> we
>>> >> can
>>> >>>>>>>>>>>> determine
>>> >>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code
>>> into
>>> >> his
>>> >>>>>>>> branch
>>> >>>>>>>>>>>>>>>>> before
>>> >>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code
>>> after his
>>> >>>>>>>> merge
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor
>>> code in
>>> >>>> the
>>> >>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>>> >> plug-in,
>>> >>>>>>>> which
>>> >>>>>>>>>>>>>>>>>>> currently
>>> >>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>>> >> (although
>>> >>>>>>>> it
>>> >>>>>>>>>> does
>>> >>>>>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>>>> know
>>> >>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>>> actually
>>> >> in
>>> >>>>>>>> use))
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>>> the
>>> >> url
>>> >>>>>>>> field
>>> >>>>>>>>>>>> (if
>>> >>>>>>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>>> in
>>> >> the
>>> >>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>>> hypervisor
>>> >> in
>>> >>>>>>>>>>>>>>>>> question, we
>>> >>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>>> place
>>> >> of
>>> >>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>>> >> checks
>>> >>>>>>>> for
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> managed
>>> >>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>>> >> hypervisor
>>> >>>>>>>> data
>>> >>>>>>>>>>>>>>>>>>> structure is
>>> >>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command
>>> executes to
>>> >>>>>>>> attach
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>> volume.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>> >>>> detach a
>>> >>>>>>>>>>>> volume,
>>> >>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> >>>>>>>> structure
>>> >>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>>>> removed.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>>> in
>>> >> 4.2
>>> >>>>>>>>>> unless
>>> >>>>>>>>>>>>>>>>> it is
>>> >>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>>> we
>>> >>>> have
>>> >>>>>>>> some
>>> >>>>>>>>>>>>>>>>> idea,
>>> >>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while
>>> avoiding
>>> >>>>>>>>>>>> implementation
>>> >>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>>> >> field.
>>> >>>> I
>>> >>>>>>>>>> think
>>> >>>>>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>>> regarding
>>> >>>> the
>>> >>>>>>>>>>>>>>>>> expected
>>> >>>>>>>>>>>>>>>>>>> format
>>> >>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>>> are
>>> >>>> not
>>> >>>>>>>>>>>>>>>>> returned on
>>> >>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>>> >> listStoragePool
>>> >>>>>>>>>>>> response.
>>> >>>>>>>>>>>>>>>>>> Why
>>> >>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>>> useful
>>> >>>> for
>>> >>>>>>>>>>>> clients
>>> >>>>>>>>>>>>>>>>> to
>>> >>>>>>>>>>>>>>>>>> be
>>> >>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details
>>> field."
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>>> here.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>> >>>> Offering-by-Disk
>>> >>>>>>>>>>>> Offering
>>> >>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have
>>> to be
>>> >>>>>>>> able to
>>> >>>>>>>>>>>>>>>>>> associate
>>> >>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
>>> disk_offering_details
>>> >>>> table.
>>> >>>>>>>>>> Maybe
>>> >>>>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>>> could
>>> >>>>>>>>>>>>>>>>>>>>>> go there?
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>>> IOPS
>>> >> in
>>> >>>>>>>> the
>>> >>>>>>>>>> GUI
>>> >>>>>>>>>>>>>>>>> if
>>> >>>>>>>>>>>>>>>>>>> it's
>>> >>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>>> DB.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>> cloud<
>>> >> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>> >
>>> >>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> --
>>> >>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>> o: 303.746.7302
>>> >>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>> *â„¢*
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> *Mike Tutkowski*
>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>> o: 303.746.7302
>>> >>>>>>> Advancing the way the world uses the cloud<
>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>> *â„¢*
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> *Mike Tutkowski*
>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>> o: 303.746.7302
>>> >>>>>> Advancing the way the world uses the cloud<
>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>> *â„¢*
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> *Mike Tutkowski*
>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>> e: mike.tutkowski@solidfire.com
>>> >>>>> o: 303.746.7302
>>> >>>>> Advancing the way the world uses the
>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>> *â„¢*
>>> >>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Mike Tutkowski*
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>> e: mike.tutkowski@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the
>>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>> *â„¢*
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"As I mentioned previously, I am very reluctant for any feature to come
into master that can exhaust resources."
Just wanted to mention that, worst case, the SAN would fail creation of the
volume before allowing a new volume to break the system.
On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Hi John,
>
> Are you thinking we add a column on to the storage pool table, IOPS_Count,
> where we add and subtract committed IOPS?
>
> That is easy enough.
>
> How do you want to determine what the SAN is capable of supporting IOPS
> wise? Remember we're dealing with a dynamic SAN here...as you add storage
> nodes to the cluster, the number of IOPS increases. Do we have a thread we
> can use to query external devices like this SAN to update the supported
> number of IOPS?
>
> Also, how do you want to enforce the IOPS limit? Do we pass in an
> overcommit ration to the plug-in when it's created? We would need to store
> this in the storage_pool table, as well, I believe.
>
> We should also get Wei involved in this as his feature will need similar
> functionality.
>
> Also, we should do this FAST as we have only two weeks left and many of us
> will be out for several days for the CS Collab Conference.
>
> Thanks
>
>
> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Querying the SAN only indicates the number of IOPS currently in use. The
>> allocator needs to check the number of IOPS committed which is tracked by
>> CloudStack. For 4.2, we should be able to query the number of IOPS
>> committed to a DataStore, and determine whether or not the number requested
>> can be fulfilled by that device. It seems to be that a
>> DataStore#getCommittedIOPS() : Long method would be sufficient.
>> DataStore's that don't support provisioned IOPS would return null.
>>
>> As I mentioned previously, I am very reluctant for any feature to come
>> into master that can exhaust resources.
>>
>> Thanks,
>> -John
>>
>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > Yeah, I'm not sure I could come up with anything near an accurate
>> > assessment of how many IOPS are currently available on the SAN (or even
>> a
>> > total number that are available for volumes). Not sure if there's yet an
>> > API call for that.
>> >
>> > If I did know this number (total number of IOPS supported by the SAN),
>> we'd
>> > still have to keep track of the total number of volumes we've created
>> from
>> > CS on the SAN in terms of their IOPS. Also, if an admin issues an API
>> call
>> > directly to the SAN to tweak the number of IOPS on a given volume or
>> set of
>> > volumes (not supported from CS, but supported via the SolidFire API),
>> our
>> > numbers in CS would be off.
>> >
>> > I'm thinking verifying sufficient number of IOPS is a really good idea
>> for
>> > a future release.
>> >
>> > For 4.2 I think we should stick to having the allocator detect if
>> storage
>> > QoS is desired and if the storage pool in question supports that
>> feature.
>> >
>> > If you really are over provisioned on your SAN in terms of IOPS or
>> > capacity, the SAN can let the admin know in several different ways
>> (e-mail,
>> > SNMP, GUI).
>> >
>> >
>> > On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Mike,
>> >>
>> >> Please see my comments in-line below.
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >> wrote:
>> >>
>> >>> Comments below in red.
>> >>>
>> >>> Thanks
>> >>>
>> >>>
>> >>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> Overall, I agree with the steps to below for 4.2. However, we may
>> want
>> >> to
>> >>>> throw an exception if we can not fulfill a requested QoS. If the
>> user
>> >> is
>> >>>> expecting that the hypervisor will provide a particular QoS, and
>> that is
>> >>>> not possible, it seems like we should inform them rather than
>> silently
>> >>>> ignoring their request.
>> >>>>
>> >>>
>> >>> Sure, that sounds reasonable.
>> >>>
>> >>> We'd have to come up with some way for the allocators to know about
>> the
>> >>> requested storage QoS and then query the candidate drivers.
>> >>>
>> >>> Any thoughts on how we might do that?
>> >>>
>> >>>
>> >>>>
>> >>>> To collect my thoughts from previous parts of the thread, I am
>> >>>> uncomfortable with the idea that the management server can
>> overcommit a
>> >>>> resource. You had mentioned querying the device for available IOPS.
>> >> While
>> >>>> that would be nice, it seems like we could fall back to a max IOPS
>> and
>> >>>> overcommit factor manually calculated and entered by the
>> >>>> administrator/operator. I think such threshold and allocation rails
>> >> should
>> >>>> be added for both provisioned IOPS and throttled I/O -- it is a basic
>> >>>> feature of any cloud orchestration platform.
>> >>>>
>> >>>
>> >>> Are you thinking this ability would make it into 4.2? Just curious
>> what
>> >>> release we're talking about here. For the SolidFire SAN, you might
>> have,
>> >>> say, four separate storage nodes to start (200,000 IOPS) and then
>> later
>> >> add
>> >>> a new node (now you're at 250,000 IOPS). CS would have to have a way
>> to
>> >>> know that the number of supported IOPS has increased.
>> >>
>> >> Yes, I think we need some *basic*/conservative rails in 4.2. For
>> example,
>> >> we may only support expanding capacity in 4.2, and not handle any
>> >> re-balance scenarios -- node failure, addition, etc. Extrapolating a
>> >> bit, the throttled I/O enhancement seems like it needs a similar set of
>> >> rails defined per host.
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>> >>>> implementation specific manner. I think we need to arrive at a
>> general
>> >>>> model that can be exploited by the allocators and planners. We
>> should
>> >>>> restrict implementation specific key-value pairs (call them details,
>> >>>> extended data, whatever) to information that is unique to the driver
>> and
>> >>>> would provide no useful information to the management server's
>> >>>> orchestration functions. A resource QoS does not seem to fit those
>> >>>> criteria.
>> >>>>
>> >>>
>> >>> I wonder if this would be a good discussion topic for Sunday's CS
>> Collab
>> >>> Conf hack day that Joe just sent out an e-mail about?
>> >>
>> >> Yes, it would -- I will put something in the wiki topic. It will also
>> be
>> >> part of my talk on Monday -- How to Run from Zombie which include some
>> of
>> >> my opinions on the topic.
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com>
>> >>>> wrote:
>> >>>>
>> >>>>> So, here's my suggestion for 4.2:
>> >>>>>
>> >>>>> Accept the values as they are currently required (four new fields
>> for
>> >>>> Wei's
>> >>>>> feature or two new fields for mine).
>> >>>>>
>> >>>>> The Add Disk Offering dialog needs three new radio buttons:
>> >>>>>
>> >>>>> 1) No QoS
>> >>>>>
>> >>>>> 2) Hypervisor QoS
>> >>>>>
>> >>>>> 3) Storage Qos
>> >>>>>
>> >>>>> The admin needs to specify storage tags that only map to storage
>> that
>> >>>>> supports Storage QoS.
>> >>>>>
>> >>>>> The admin needs to be aware for Hypervisor QoS that unless all
>> >>>> hypervisors
>> >>>>> in use support the new fields, they may not be enforced.
>> >>>>>
>> >>>>> Post 4.3:
>> >>>>>
>> >>>>> Come up with a way to more generally enter these parameters
>> (probably
>> >>>> just
>> >>>>> key/value pairs sent to the drivers).
>> >>>>>
>> >>>>> Have the drivers expose their feature set so the allocators can
>> >> consider
>> >>>>> them more fully and throw an exception if there is not a sufficient
>> >>>> match.
>> >>>>>
>> >>>>>
>> >>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>
>> >>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>> >>>> burden
>> >>>>>> on the admin to configure a Disk Offering in a way that makes
>> sense.
>> >> For
>> >>>>>> example, if he wants to use storage QoS with supported Min and Max
>> >>>> values,
>> >>>>>> he'll have to put in a storage tag that references the SolidFire
>> >> primary
>> >>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then
>> he's
>> >>>> not
>> >>>>>> going to get the Min and Max feature. We could add help text to the
>> >>>> pop-up
>> >>>>>> dialog that's displayed when you click in the Min and Max text
>> fields.
>> >>>>>>
>> >>>>>> Same idea for Wei's feature.
>> >>>>>>
>> >>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>> >>>>>> approach for post 4.2.
>> >>>>>>
>> >>>>>> Maybe in the future we could have the drivers advertise their
>> >>>> capabilities
>> >>>>>> and if the allocator feels a request is not being satisfied (say
>> Min
>> >> was
>> >>>>>> entered, but it not's supported by any storage plug-in) it can
>> throw
>> >> an
>> >>>>>> exception.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>
>> >>>>>>> Comments below in red.
>> >>>>>>>
>> >>>>>>> Thanks
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jburwell@basho.com
>> >
>> >>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> Please see my comment in-line below.
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi John,
>> >>>>>>>>>
>> >>>>>>>>> I've put comments below in red.
>> >>>>>>>>>
>> >>>>>>>>> Thanks!
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
>> jburwell@basho.com
>> >>>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Mike,
>> >>>>>>>>>>
>> >>>>>>>>>> First and foremost, we must ensure that these two features are
>> >>>>>>>> mutually
>> >>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>> >>>>>>>> contains both
>> >>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>> >>>> non-deterministic
>> >>>>>>>>>> operations.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Agreed
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>> >>>> oriented
>> >>>>>>>>>> solves the problem in short term. As I understand storage
>> tags,
>> >> we
>> >>>>>>>> have no
>> >>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>> >>>>>>>> necessarily
>> >>>>>>>>>> intending that we implement this allocation model in 4.3, but
>> >>>> instead,
>> >>>>>>>>>> determine if this type model would be one we would want to
>> support
>> >>>> in
>> >>>>>>>> the
>> >>>>>>>>>> future. If so, I would encourage us to ensure that the data
>> model
>> >>>> and
>> >>>>>>>>>> current implementation would not preclude evolution in that
>> >>>>>>>> direction. My
>> >>>>>>>>>> view is that this type of allocation model is what user's
>> expect
>> >> of
>> >>>>>>>> "cloud"
>> >>>>>>>>>> systems -- selecting the best available resource set to
>> fulfill a
>> >>>>>>>> set of
>> >>>>>>>>>> system requirements.
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> I believe we have meet your requirement here in that what we've
>> >>>>>>>> implemented
>> >>>>>>>>> should not make refinement difficult in the future. If we don't
>> >>>> modify
>> >>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
>> simple
>> >>>>>>>> changes
>> >>>>>>>>> to enhance the current functioning of the system.
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Looking through both patches, I have to say that the aggregated
>> >> result
>> >>>>>>>> seems a bit confusing. There are six new attributes for
>> throttled
>> >>>> I/O and
>> >>>>>>>> two for provisioned IOPS with no obvious grouping. My concern is
>> >> not
>> >>>>>>>> technical, but rather, about maintainability. At minimum,
>> Javadoc
>> >>>> should
>> >>>>>>>> be added explaining the two sets of attributes and their mutual
>> >>>> exclusion.
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> I agree: We need JavaDoc to explain them and their mutual
>> exclusion.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>> The other part that is interesting is that throttled I/O provides
>> >> both
>> >>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
>> uses
>> >> a
>> >>>>>>>> range. In order to select the best available resource to
>> fulfill a
>> >>>> QoS, we
>> >>>>>>>> would need to have the QoS expression normalized in terms of
>> units
>> >>>> (IOPS or
>> >>>>>>>> bytes) and their expression (rate vs. range). If we want to
>> >> achieve a
>> >>>>>>>> model like I described, I think we would need to resolve this
>> issue
>> >>>> in 4.2
>> >>>>>>>> to ensure a viable migration path.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I think we're not likely to be able to normalize the input for
>> 4.2.
>> >>>> Plus
>> >>>>>>> people probably want to input the data in terms they're familiar
>> with
>> >>>> for
>> >>>>>>> the products in question.
>> >>>>>>>
>> >>>>>>> Ideally we would fix the way we do storage tagging and let the
>> user
>> >>>> send
>> >>>>>>> key/value pairs to each vendor that could be selected due to a
>> given
>> >>>>>>> storage tag. I'm still not sure that would solve it because what
>> >>>> happens if
>> >>>>>>> you change the storage tag of a given Primary Storage after having
>> >>>> created
>> >>>>>>> a Disk Offering?
>> >>>>>>>
>> >>>>>>> Basically storage tagging is kind of a mess and we should re-think
>> >> it.
>> >>>>>>>
>> >>>>>>> Also, we need to have a way for the drivers to expose their
>> supported
>> >>>>>>> feature sets so the allocators can make good choices.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> As I think through the implications of these requirements and
>> >>>> reflect
>> >>>>>>>> on
>> >>>>>>>>>> the reviews, I don't understand why they haven't already
>> impacted
>> >>>> the
>> >>>>>>>>>> allocators and planners. As it stands, the current provisioned
>> >> IOPS
>> >>>>>>>> has no
>> >>>>>>>>>> checks to ensure that the volumes are allocated to devices that
>> >> have
>> >>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
>> understand
>> >>>> the
>> >>>>>>>>>> current patch, we can overcommit storage resources --
>> potentially
>> >>>>>>>> causing
>> >>>>>>>>>> none of the QoS obligations from being fulfilled. It seems to
>> me
>> >>>>>>>> that a
>> >>>>>>>>>> DataStore supporting provisioned IOPS should express the
>> maximum
>> >>>> IOPS
>> >>>>>>>> which
>> >>>>>>>>>> it can support and some type of overcommitment factor. This
>> >>>>>>>> information
>> >>>>>>>>>> should be used by the storage allocators to determine the
>> device
>> >>>> best
>> >>>>>>>> able
>> >>>>>>>>>> to support the resources needs of a volume. It seems that a
>> >> similar
>> >>>>>>>> set of
>> >>>>>>>>>> considerations would need to be added to the Hypervisor layer
>> >> which
>> >>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Yeah, for this first release, we just followed the path that was
>> >>>>>>>> previously
>> >>>>>>>>> established for other properties you see on dialogs in CS: Just
>> >>>> because
>> >>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>> >> supports
>> >>>>>>>> them.
>> >>>>>>>>> It is then up to the admin to make sure he inputs, say, a
>> storage
>> >> tag
>> >>>>>>>> that
>> >>>>>>>>> confines the deployment only to storage that supports the
>> selected
>> >>>>>>>>> features. This is not ideal, but it's kind of the way CloudStack
>> >>>> works
>> >>>>>>>>> today.
>> >>>>>>>>
>> >>>>>>>> I understand the tag functionality, and the need for the
>> >> administrator
>> >>>>>>>> to very carefully construct offerings. My concern is that we
>> over
>> >>>>>>>> guarantee a resource's available IOPS. For the purposes of
>> >>>> illustration,
>> >>>>>>>> let's say we have a SolidFire, and the max IOPS for that device
>> is
>> >>>> 100000.
>> >>>>>>>> We also know that we can safely oversubscribe by 50%. Therefore,
>> >> we
>> >>>>>>>> need to ensure that we don't allocate more than 150,000
>> guaranteed
>> >>>> IOPS
>> >>>>>>>> from that device. Intuitively, it seems like the DataStore
>> >>>> configuration
>> >>>>>>>> should have a max assignable IOPS and overcommitment factor. As
>> we
>> >>>>>>>> allocate volumes and attach VMs, we need to ensure that guarantee
>> >>>> more IOPS
>> >>>>>>>> exceed the configured maximum for a DataStore. Does that make
>> >> sense?
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> I think that's a good idea for a future enhancement. I'm not even
>> >> sure
>> >>>> I
>> >>>>>>> can query the SAN to find out how many IOPS safely remain. I'd
>> have
>> >> to
>> >>>> get
>> >>>>>>> all of the min values for all of the volumes on the SAN and total
>> >> them
>> >>>> up,
>> >>>>>>> I suppose, and subtract it from the total (user facing) supported
>> >> IOPS
>> >>>> of
>> >>>>>>> the system.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Another question occurs to me -- should we allow non-QoS
>> resources
>> >>>> to
>> >>>>>>>> be
>> >>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>> >> provisioned
>> >>>>>>>> IOPS, I
>> >>>>>>>>>> think a side effect of the current implementation is SolidFire
>> >>>> volumes
>> >>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>> >> seems
>> >>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side
>> with
>> >> QoS
>> >>>>>>>> VMs.
>> >>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
>> starve
>> >>>> out
>> >>>>>>>> the
>> >>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>> >> collocated
>> >>>>>>>> VMs
>> >>>>>>>>>> from being fulfilled.
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and
>> not
>> >>>>>>>> specify
>> >>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>> >>>> defaults
>> >>>>>>>> we
>> >>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you
>> will
>> >>>>>>>> have
>> >>>>>>>>> your Mins met.
>> >>>>>>>>>
>> >>>>>>>>> It sounds like you're perhaps looking for a storage tags
>> exclusions
>> >>>>>>>> list,
>> >>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>> >>>> volume
>> >>>>>>>> to
>> >>>>>>>>> storage with these following tags).
>> >>>>>>>>
>> >>>>>>>> I don't like the idea of a storage tags exclusion list as it
>> would
>> >>>>>>>> complicate component assembly. It would require a storage
>> plugin to
>> >>>>>>>> anticipate all of the possible component assemblies and determine
>> >> the
>> >>>>>>>> invalid relationships. I prefer that drivers express their
>> >>>> capabilities
>> >>>>>>>> which can be matched to a set of requested requirements.
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> I'm not sure why a storage plug-in would care about inclusion or
>> >>>>>>> exclusion lists. It just needs to advertise its functionality in a
>> >> way
>> >>>> the
>> >>>>>>> allocator understands.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>> >> limits
>> >>>>>>>> IOPS,
>> >>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system with
>> >>>>>>>> guaranteed
>> >>>>>>>>> minimum IOPS does this.
>> >>>>>>>>
>> >>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>> >> exist.
>> >>>>>>>> However, I am concerned with the more general case as we
>> supported
>> >>>> more
>> >>>>>>>> devices with provisioned IOPS.
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
>> values
>> >> to
>> >>>>>>> drivers. Min and Max and pretty industry standard for provisioned
>> >>>> IOPS, but
>> >>>>>>> what if you break them out by read and write or do something
>> else? We
>> >>>> need
>> >>>>>>> a more general mechanism.
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled I/O
>> >> and
>> >>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Agreed
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>> We also need to understand the implications of these QoS
>> >> guarantees
>> >>>> on
>> >>>>>>>>>> operation of the system to ensure that the underlying hardware
>> >>>>>>>> resources
>> >>>>>>>>>> can fulfill them. Given the time frame, we will likely be
>> forced
>> >> to
>> >>>>>>>> make
>> >>>>>>>>>> compromises to achieve these goals, and refine the
>> implementation
>> >> in
>> >>>>>>>> 4.3.
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> I agree, John. I also think you've come up with some great ideas
>> >> for
>> >>>>>>>> 4.3. :)
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> -John
>> >>>>>>>>>>
>> >>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
>> storage
>> >>>>>>>> tags).
>> >>>>>>>>>> I
>> >>>>>>>>>>> believe John was just throwing out an idea that - in addition
>> to
>> >>>>>>>> storage
>> >>>>>>>>>>> tags - we could look into these allocators (storage QoS being
>> >>>>>>>> preferred,
>> >>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>> >> hypervisor
>> >>>>>>>> QoS
>> >>>>>>>>>> is).
>> >>>>>>>>>>>
>> >>>>>>>>>>> I think John's concern is that you can enter in values for
>> Wei's
>> >>>> and
>> >>>>>>>> my
>> >>>>>>>>>>> feature that are not honored by other vendors (at least yet),
>> so
>> >> he
>> >>>>>>>> was
>> >>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
>> >> prefer
>> >>>>>>>> these
>> >>>>>>>>>>> vendors when these fields are filled in. As it stands today in
>> >>>>>>>>>> CloudStack,
>> >>>>>>>>>>> we already have this kind of an issue with other features
>> (fields
>> >>>> in
>> >>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
>> post
>> >>>> 4.2
>> >>>>>>>> we
>> >>>>>>>>>>> could look into generic name/value pairs (this is how
>> OpenStack
>> >>>>>>>> addresses
>> >>>>>>>>>>> the issue).
>> >>>>>>>>>>>
>> >>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
>> >> code
>> >>>>>>>>>> freeze)
>> >>>>>>>>>>> to go too deeply down that path in 4.2.
>> >>>>>>>>>>>
>> >>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
>> fields
>> >>>>>>>> and my
>> >>>>>>>>>>> fields as is, make sure only one or the other feature has data
>> >>>>>>>> entered
>> >>>>>>>>>> for
>> >>>>>>>>>>> it (or neither), and call it good for 4.2.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Then let's step back and look into a more general-purpose
>> design
>> >>>>>>>> that can
>> >>>>>>>>>>> be applied throughout CloudStack where we have these
>> situations.
>> >>>>>>>>>>>
>> >>>>>>>>>>> What do you think?
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>> >> jburwell@basho.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I just published my review @
>> >> https://reviews.apache.org/r/11479/.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I apologize for the delay,
>> >>>>>>>>>>>> -John
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> No problem, John.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I still want to re-review it by myself before coming up
>> with a
>> >>>> new
>> >>>>>>>>>> patch
>> >>>>>>>>>>>>> file.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
>> checked
>> >>>> in
>> >>>>>>>> and
>> >>>>>>>>>>>>> merge those into mine before generating a new patch file?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> >>>> jburwell@basho.com
>> >>>>>>>>>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>> >>>> offline
>> >>>>>>>> ATM,
>> >>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Sorry for the oversight,
>> >>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> >>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading
>> this :)
>> >>>> ),
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the
>> areas
>> >> we
>> >>>>>>>> wanted
>> >>>>>>>>>>>>>>> addressed.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>> >> afternoon,
>> >>>>>>>> then
>> >>>>>>>>>>>> send
>> >>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>> another patch.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Sure, that sounds good.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> >>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Hi Mike,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
>> Java
>> >>>>>>>> code,
>> >>>>>>>>>>>>>> except
>> >>>>>>>>>>>>>>>>> the cloudstack UI.
>> >>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
>> branch
>> >>>> into
>> >>>>>>>>>>>> master
>> >>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Hey John,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>> >>>>>>>> branch,
>> >>>>>>>>>> but
>> >>>>>>>>>>>> -
>> >>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>> >>>>>>>> SolidFire
>> >>>>>>>>>>>> branch
>> >>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>> >>>> master.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>> >> merge
>> >>>>>>>>>>>> strategy.
>> >>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> >>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>> >>>>>>>>>> disk_io_throttling,
>> >>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>> >>>>>>>>>> overlapping
>> >>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>> >>>> patches
>> >>>>>>>> to
>> >>>>>>>>>> be
>> >>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>> follows:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>> >>>> SolidFire
>> >>>>>>>> are
>> >>>>>>>>>>>>>>>>> additive
>> >>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>> >> dependency
>> >>>>>>>>>> between
>> >>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
>> >> can
>> >>>>>>>>>>>> determine
>> >>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
>> >> his
>> >>>>>>>> branch
>> >>>>>>>>>>>>>>>>> before
>> >>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after
>> his
>> >>>>>>>> merge
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code
>> in
>> >>>> the
>> >>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>> >> plug-in,
>> >>>>>>>> which
>> >>>>>>>>>>>>>>>>>>> currently
>> >>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>> >> (although
>> >>>>>>>> it
>> >>>>>>>>>> does
>> >>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>> know
>> >>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
>> actually
>> >> in
>> >>>>>>>> use))
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in
>> the
>> >> url
>> >>>>>>>> field
>> >>>>>>>>>>>> (if
>> >>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored
>> in
>> >> the
>> >>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
>> hypervisor
>> >> in
>> >>>>>>>>>>>>>>>>> question, we
>> >>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the
>> place
>> >> of
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>> >> checks
>> >>>>>>>> for
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> managed
>> >>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>> >> hypervisor
>> >>>>>>>> data
>> >>>>>>>>>>>>>>>>>>> structure is
>> >>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes
>> to
>> >>>>>>>> attach
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> volume.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>> >>>> detach a
>> >>>>>>>>>>>> volume,
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>> >>>>>>>> structure
>> >>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>> removed.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS
>> in
>> >> 4.2
>> >>>>>>>>>> unless
>> >>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If
>> we
>> >>>> have
>> >>>>>>>> some
>> >>>>>>>>>>>>>>>>> idea,
>> >>>>>>>>>>>>>>>>>>>>> that'd be cool.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>> >>>>>>>>>>>> implementation
>> >>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>> >> field.
>> >>>> I
>> >>>>>>>>>> think
>> >>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
>> regarding
>> >>>> the
>> >>>>>>>>>>>>>>>>> expected
>> >>>>>>>>>>>>>>>>>>> format
>> >>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
>> are
>> >>>> not
>> >>>>>>>>>>>>>>>>> returned on
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>> >> listStoragePool
>> >>>>>>>>>>>> response.
>> >>>>>>>>>>>>>>>>>> Why
>> >>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
>> useful
>> >>>> for
>> >>>>>>>>>>>> clients
>> >>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
>> here.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> >>>> Offering-by-Disk
>> >>>>>>>>>>>> Offering
>> >>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to
>> be
>> >>>>>>>> able to
>> >>>>>>>>>>>>>>>>>> associate
>> >>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>> >>>> table.
>> >>>>>>>>>> Maybe
>> >>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>> could
>> >>>>>>>>>>>>>>>>>>>>>> go there?
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
>> IOPS
>> >> in
>> >>>>>>>> the
>> >>>>>>>>>> GUI
>> >>>>>>>>>>>>>>>>> if
>> >>>>>>>>>>>>>>>>>>> it's
>> >>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
>> DB.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>> cloud<
>> >> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
Are you thinking we add a column on to the storage pool table, IOPS_Count,
where we add and subtract committed IOPS?
That is easy enough.
How do you want to determine what the SAN is capable of supporting IOPS
wise? Remember we're dealing with a dynamic SAN here...as you add storage
nodes to the cluster, the number of IOPS increases. Do we have a thread we
can use to query external devices like this SAN to update the supported
number of IOPS?
Also, how do you want to enforce the IOPS limit? Do we pass in an
overcommit ration to the plug-in when it's created? We would need to store
this in the storage_pool table, as well, I believe.
We should also get Wei involved in this as his feature will need similar
functionality.
Also, we should do this FAST as we have only two weeks left and many of us
will be out for several days for the CS Collab Conference.
Thanks
On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Querying the SAN only indicates the number of IOPS currently in use. The
> allocator needs to check the number of IOPS committed which is tracked by
> CloudStack. For 4.2, we should be able to query the number of IOPS
> committed to a DataStore, and determine whether or not the number requested
> can be fulfilled by that device. It seems to be that a
> DataStore#getCommittedIOPS() : Long method would be sufficient.
> DataStore's that don't support provisioned IOPS would return null.
>
> As I mentioned previously, I am very reluctant for any feature to come
> into master that can exhaust resources.
>
> Thanks,
> -John
>
> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Yeah, I'm not sure I could come up with anything near an accurate
> > assessment of how many IOPS are currently available on the SAN (or even a
> > total number that are available for volumes). Not sure if there's yet an
> > API call for that.
> >
> > If I did know this number (total number of IOPS supported by the SAN),
> we'd
> > still have to keep track of the total number of volumes we've created
> from
> > CS on the SAN in terms of their IOPS. Also, if an admin issues an API
> call
> > directly to the SAN to tweak the number of IOPS on a given volume or set
> of
> > volumes (not supported from CS, but supported via the SolidFire API), our
> > numbers in CS would be off.
> >
> > I'm thinking verifying sufficient number of IOPS is a really good idea
> for
> > a future release.
> >
> > For 4.2 I think we should stick to having the allocator detect if storage
> > QoS is desired and if the storage pool in question supports that feature.
> >
> > If you really are over provisioned on your SAN in terms of IOPS or
> > capacity, the SAN can let the admin know in several different ways
> (e-mail,
> > SNMP, GUI).
> >
> >
> > On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Please see my comments in-line below.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Comments below in red.
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Overall, I agree with the steps to below for 4.2. However, we may
> want
> >> to
> >>>> throw an exception if we can not fulfill a requested QoS. If the user
> >> is
> >>>> expecting that the hypervisor will provide a particular QoS, and that
> is
> >>>> not possible, it seems like we should inform them rather than silently
> >>>> ignoring their request.
> >>>>
> >>>
> >>> Sure, that sounds reasonable.
> >>>
> >>> We'd have to come up with some way for the allocators to know about the
> >>> requested storage QoS and then query the candidate drivers.
> >>>
> >>> Any thoughts on how we might do that?
> >>>
> >>>
> >>>>
> >>>> To collect my thoughts from previous parts of the thread, I am
> >>>> uncomfortable with the idea that the management server can overcommit
> a
> >>>> resource. You had mentioned querying the device for available IOPS.
> >> While
> >>>> that would be nice, it seems like we could fall back to a max IOPS and
> >>>> overcommit factor manually calculated and entered by the
> >>>> administrator/operator. I think such threshold and allocation rails
> >> should
> >>>> be added for both provisioned IOPS and throttled I/O -- it is a basic
> >>>> feature of any cloud orchestration platform.
> >>>>
> >>>
> >>> Are you thinking this ability would make it into 4.2? Just curious what
> >>> release we're talking about here. For the SolidFire SAN, you might
> have,
> >>> say, four separate storage nodes to start (200,000 IOPS) and then later
> >> add
> >>> a new node (now you're at 250,000 IOPS). CS would have to have a way to
> >>> know that the number of supported IOPS has increased.
> >>
> >> Yes, I think we need some *basic*/conservative rails in 4.2. For
> example,
> >> we may only support expanding capacity in 4.2, and not handle any
> >> re-balance scenarios -- node failure, addition, etc. Extrapolating a
> >> bit, the throttled I/O enhancement seems like it needs a similar set of
> >> rails defined per host.
> >>
> >>>
> >>>
> >>>>
> >>>> For 4.3, I don't like the idea that a QoS would be expressed in a
> >>>> implementation specific manner. I think we need to arrive at a
> general
> >>>> model that can be exploited by the allocators and planners. We should
> >>>> restrict implementation specific key-value pairs (call them details,
> >>>> extended data, whatever) to information that is unique to the driver
> and
> >>>> would provide no useful information to the management server's
> >>>> orchestration functions. A resource QoS does not seem to fit those
> >>>> criteria.
> >>>>
> >>>
> >>> I wonder if this would be a good discussion topic for Sunday's CS
> Collab
> >>> Conf hack day that Joe just sent out an e-mail about?
> >>
> >> Yes, it would -- I will put something in the wiki topic. It will also
> be
> >> part of my talk on Monday -- How to Run from Zombie which include some
> of
> >> my opinions on the topic.
> >>
> >>>
> >>>
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> So, here's my suggestion for 4.2:
> >>>>>
> >>>>> Accept the values as they are currently required (four new fields for
> >>>> Wei's
> >>>>> feature or two new fields for mine).
> >>>>>
> >>>>> The Add Disk Offering dialog needs three new radio buttons:
> >>>>>
> >>>>> 1) No QoS
> >>>>>
> >>>>> 2) Hypervisor QoS
> >>>>>
> >>>>> 3) Storage Qos
> >>>>>
> >>>>> The admin needs to specify storage tags that only map to storage that
> >>>>> supports Storage QoS.
> >>>>>
> >>>>> The admin needs to be aware for Hypervisor QoS that unless all
> >>>> hypervisors
> >>>>> in use support the new fields, they may not be enforced.
> >>>>>
> >>>>> Post 4.3:
> >>>>>
> >>>>> Come up with a way to more generally enter these parameters (probably
> >>>> just
> >>>>> key/value pairs sent to the drivers).
> >>>>>
> >>>>> Have the drivers expose their feature set so the allocators can
> >> consider
> >>>>> them more fully and throw an exception if there is not a sufficient
> >>>> match.
> >>>>>
> >>>>>
> >>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
> >>>> burden
> >>>>>> on the admin to configure a Disk Offering in a way that makes sense.
> >> For
> >>>>>> example, if he wants to use storage QoS with supported Min and Max
> >>>> values,
> >>>>>> he'll have to put in a storage tag that references the SolidFire
> >> primary
> >>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then
> he's
> >>>> not
> >>>>>> going to get the Min and Max feature. We could add help text to the
> >>>> pop-up
> >>>>>> dialog that's displayed when you click in the Min and Max text
> fields.
> >>>>>>
> >>>>>> Same idea for Wei's feature.
> >>>>>>
> >>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
> >>>>>> approach for post 4.2.
> >>>>>>
> >>>>>> Maybe in the future we could have the drivers advertise their
> >>>> capabilities
> >>>>>> and if the allocator feels a request is not being satisfied (say Min
> >> was
> >>>>>> entered, but it not's supported by any storage plug-in) it can throw
> >> an
> >>>>>> exception.
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Comments below in red.
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> Please see my comment in-line below.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi John,
> >>>>>>>>>
> >>>>>>>>> I've put comments below in red.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> jburwell@basho.com
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> First and foremost, we must ensure that these two features are
> >>>>>>>> mutually
> >>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
> >>>>>>>> contains both
> >>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >>>> non-deterministic
> >>>>>>>>>> operations.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Agreed
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
> >>>> oriented
> >>>>>>>>>> solves the problem in short term. As I understand storage tags,
> >> we
> >>>>>>>> have no
> >>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
> >>>>>>>> necessarily
> >>>>>>>>>> intending that we implement this allocation model in 4.3, but
> >>>> instead,
> >>>>>>>>>> determine if this type model would be one we would want to
> support
> >>>> in
> >>>>>>>> the
> >>>>>>>>>> future. If so, I would encourage us to ensure that the data
> model
> >>>> and
> >>>>>>>>>> current implementation would not preclude evolution in that
> >>>>>>>> direction. My
> >>>>>>>>>> view is that this type of allocation model is what user's expect
> >> of
> >>>>>>>> "cloud"
> >>>>>>>>>> systems -- selecting the best available resource set to fulfill
> a
> >>>>>>>> set of
> >>>>>>>>>> system requirements.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I believe we have meet your requirement here in that what we've
> >>>>>>>> implemented
> >>>>>>>>> should not make refinement difficult in the future. If we don't
> >>>> modify
> >>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively
> simple
> >>>>>>>> changes
> >>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> Looking through both patches, I have to say that the aggregated
> >> result
> >>>>>>>> seems a bit confusing. There are six new attributes for throttled
> >>>> I/O and
> >>>>>>>> two for provisioned IOPS with no obvious grouping. My concern is
> >> not
> >>>>>>>> technical, but rather, about maintainability. At minimum,
> Javadoc
> >>>> should
> >>>>>>>> be added explaining the two sets of attributes and their mutual
> >>>> exclusion.
> >>>>>>>>
> >>>>>>>
> >>>>>>> I agree: We need JavaDoc to explain them and their mutual
> exclusion.
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> The other part that is interesting is that throttled I/O provides
> >> both
> >>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS
> uses
> >> a
> >>>>>>>> range. In order to select the best available resource to fulfill
> a
> >>>> QoS, we
> >>>>>>>> would need to have the QoS expression normalized in terms of units
> >>>> (IOPS or
> >>>>>>>> bytes) and their expression (rate vs. range). If we want to
> >> achieve a
> >>>>>>>> model like I described, I think we would need to resolve this
> issue
> >>>> in 4.2
> >>>>>>>> to ensure a viable migration path.
> >>>>>>>
> >>>>>>>
> >>>>>>> I think we're not likely to be able to normalize the input for 4.2.
> >>>> Plus
> >>>>>>> people probably want to input the data in terms they're familiar
> with
> >>>> for
> >>>>>>> the products in question.
> >>>>>>>
> >>>>>>> Ideally we would fix the way we do storage tagging and let the user
> >>>> send
> >>>>>>> key/value pairs to each vendor that could be selected due to a
> given
> >>>>>>> storage tag. I'm still not sure that would solve it because what
> >>>> happens if
> >>>>>>> you change the storage tag of a given Primary Storage after having
> >>>> created
> >>>>>>> a Disk Offering?
> >>>>>>>
> >>>>>>> Basically storage tagging is kind of a mess and we should re-think
> >> it.
> >>>>>>>
> >>>>>>> Also, we need to have a way for the drivers to expose their
> supported
> >>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> As I think through the implications of these requirements and
> >>>> reflect
> >>>>>>>> on
> >>>>>>>>>> the reviews, I don't understand why they haven't already
> impacted
> >>>> the
> >>>>>>>>>> allocators and planners. As it stands, the current provisioned
> >> IOPS
> >>>>>>>> has no
> >>>>>>>>>> checks to ensure that the volumes are allocated to devices that
> >> have
> >>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I
> understand
> >>>> the
> >>>>>>>>>> current patch, we can overcommit storage resources --
> potentially
> >>>>>>>> causing
> >>>>>>>>>> none of the QoS obligations from being fulfilled. It seems to
> me
> >>>>>>>> that a
> >>>>>>>>>> DataStore supporting provisioned IOPS should express the maximum
> >>>> IOPS
> >>>>>>>> which
> >>>>>>>>>> it can support and some type of overcommitment factor. This
> >>>>>>>> information
> >>>>>>>>>> should be used by the storage allocators to determine the device
> >>>> best
> >>>>>>>> able
> >>>>>>>>>> to support the resources needs of a volume. It seems that a
> >> similar
> >>>>>>>> set of
> >>>>>>>>>> considerations would need to be added to the Hypervisor layer
> >> which
> >>>>>>>>>> allocating a VM to a host to prevent oversubscription.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Yeah, for this first release, we just followed the path that was
> >>>>>>>> previously
> >>>>>>>>> established for other properties you see on dialogs in CS: Just
> >>>> because
> >>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
> >> supports
> >>>>>>>> them.
> >>>>>>>>> It is then up to the admin to make sure he inputs, say, a storage
> >> tag
> >>>>>>>> that
> >>>>>>>>> confines the deployment only to storage that supports the
> selected
> >>>>>>>>> features. This is not ideal, but it's kind of the way CloudStack
> >>>> works
> >>>>>>>>> today.
> >>>>>>>>
> >>>>>>>> I understand the tag functionality, and the need for the
> >> administrator
> >>>>>>>> to very carefully construct offerings. My concern is that we over
> >>>>>>>> guarantee a resource's available IOPS. For the purposes of
> >>>> illustration,
> >>>>>>>> let's say we have a SolidFire, and the max IOPS for that device is
> >>>> 100000.
> >>>>>>>> We also know that we can safely oversubscribe by 50%. Therefore,
> >> we
> >>>>>>>> need to ensure that we don't allocate more than 150,000 guaranteed
> >>>> IOPS
> >>>>>>>> from that device. Intuitively, it seems like the DataStore
> >>>> configuration
> >>>>>>>> should have a max assignable IOPS and overcommitment factor. As
> we
> >>>>>>>> allocate volumes and attach VMs, we need to ensure that guarantee
> >>>> more IOPS
> >>>>>>>> exceed the configured maximum for a DataStore. Does that make
> >> sense?
> >>>>>>>>
> >>>>>>>
> >>>>>>> I think that's a good idea for a future enhancement. I'm not even
> >> sure
> >>>> I
> >>>>>>> can query the SAN to find out how many IOPS safely remain. I'd have
> >> to
> >>>> get
> >>>>>>> all of the min values for all of the volumes on the SAN and total
> >> them
> >>>> up,
> >>>>>>> I suppose, and subtract it from the total (user facing) supported
> >> IOPS
> >>>> of
> >>>>>>> the system.
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Another question occurs to me -- should we allow non-QoS
> resources
> >>>> to
> >>>>>>>> be
> >>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
> >> provisioned
> >>>>>>>> IOPS, I
> >>>>>>>>>> think a side effect of the current implementation is SolidFire
> >>>> volumes
> >>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
> >> seems
> >>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side with
> >> QoS
> >>>>>>>> VMs.
> >>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially
> starve
> >>>> out
> >>>>>>>> the
> >>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
> >> collocated
> >>>>>>>> VMs
> >>>>>>>>>> from being fulfilled.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and not
> >>>>>>>> specify
> >>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
> >>>> defaults
> >>>>>>>> we
> >>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you
> will
> >>>>>>>> have
> >>>>>>>>> your Mins met.
> >>>>>>>>>
> >>>>>>>>> It sounds like you're perhaps looking for a storage tags
> exclusions
> >>>>>>>> list,
> >>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
> >>>> volume
> >>>>>>>> to
> >>>>>>>>> storage with these following tags).
> >>>>>>>>
> >>>>>>>> I don't like the idea of a storage tags exclusion list as it would
> >>>>>>>> complicate component assembly. It would require a storage plugin
> to
> >>>>>>>> anticipate all of the possible component assemblies and determine
> >> the
> >>>>>>>> invalid relationships. I prefer that drivers express their
> >>>> capabilities
> >>>>>>>> which can be matched to a set of requested requirements.
> >>>>>>>>
> >>>>>>>
> >>>>>>> I'm not sure why a storage plug-in would care about inclusion or
> >>>>>>> exclusion lists. It just needs to advertise its functionality in a
> >> way
> >>>> the
> >>>>>>> allocator understands.
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
> >> limits
> >>>>>>>> IOPS,
> >>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system with
> >>>>>>>> guaranteed
> >>>>>>>>> minimum IOPS does this.
> >>>>>>>>
> >>>>>>>> As I said, for SolidFire, it sounds like this problem does not
> >> exist.
> >>>>>>>> However, I am concerned with the more general case as we supported
> >>>> more
> >>>>>>>> devices with provisioned IOPS.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific
> values
> >> to
> >>>>>>> drivers. Min and Max and pretty industry standard for provisioned
> >>>> IOPS, but
> >>>>>>> what if you break them out by read and write or do something else?
> We
> >>>> need
> >>>>>>> a more general mechanism.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled I/O
> >> and
> >>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Agreed
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> We also need to understand the implications of these QoS
> >> guarantees
> >>>> on
> >>>>>>>>>> operation of the system to ensure that the underlying hardware
> >>>>>>>> resources
> >>>>>>>>>> can fulfill them. Given the time frame, we will likely be
> forced
> >> to
> >>>>>>>> make
> >>>>>>>>>> compromises to achieve these goals, and refine the
> implementation
> >> in
> >>>>>>>> 4.3.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I agree, John. I also think you've come up with some great ideas
> >> for
> >>>>>>>> 4.3. :)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with
> storage
> >>>>>>>> tags).
> >>>>>>>>>> I
> >>>>>>>>>>> believe John was just throwing out an idea that - in addition
> to
> >>>>>>>> storage
> >>>>>>>>>>> tags - we could look into these allocators (storage QoS being
> >>>>>>>> preferred,
> >>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
> >> hypervisor
> >>>>>>>> QoS
> >>>>>>>>>> is).
> >>>>>>>>>>>
> >>>>>>>>>>> I think John's concern is that you can enter in values for
> Wei's
> >>>> and
> >>>>>>>> my
> >>>>>>>>>>> feature that are not honored by other vendors (at least yet),
> so
> >> he
> >>>>>>>> was
> >>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
> >> prefer
> >>>>>>>> these
> >>>>>>>>>>> vendors when these fields are filled in. As it stands today in
> >>>>>>>>>> CloudStack,
> >>>>>>>>>>> we already have this kind of an issue with other features
> (fields
> >>>> in
> >>>>>>>>>>> dialogs for features that not all vendors support). Perhaps
> post
> >>>> 4.2
> >>>>>>>> we
> >>>>>>>>>>> could look into generic name/value pairs (this is how OpenStack
> >>>>>>>> addresses
> >>>>>>>>>>> the issue).
> >>>>>>>>>>>
> >>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
> >> code
> >>>>>>>>>> freeze)
> >>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>
> >>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's
> fields
> >>>>>>>> and my
> >>>>>>>>>>> fields as is, make sure only one or the other feature has data
> >>>>>>>> entered
> >>>>>>>>>> for
> >>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>
> >>>>>>>>>>> Then let's step back and look into a more general-purpose
> design
> >>>>>>>> that can
> >>>>>>>>>>> be applied throughout CloudStack where we have these
> situations.
> >>>>>>>>>>>
> >>>>>>>>>>> What do you think?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >> jburwell@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I just published my review @
> >> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I still want to re-review it by myself before coming up with
> a
> >>>> new
> >>>>>>>>>> patch
> >>>>>>>>>>>>> file.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be
> checked
> >>>> in
> >>>>>>>> and
> >>>>>>>>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >>>> jburwell@basho.com
> >>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> >>>> offline
> >>>>>>>> ATM,
> >>>>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this
> :)
> >>>> ),
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
> >> we
> >>>>>>>> wanted
> >>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> >> afternoon,
> >>>>>>>> then
> >>>>>>>>>>>> send
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in
> Java
> >>>>>>>> code,
> >>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling
> branch
> >>>> into
> >>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> >>>>>>>> branch,
> >>>>>>>>>> but
> >>>>>>>>>>>> -
> >>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> >>>>>>>> SolidFire
> >>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
> >>>> master.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
> >> merge
> >>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> >>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> >>>> patches
> >>>>>>>> to
> >>>>>>>>>> be
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> >>>> SolidFire
> >>>>>>>> are
> >>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> >> dependency
> >>>>>>>>>> between
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
> >> can
> >>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
> >> his
> >>>>>>>> branch
> >>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after
> his
> >>>>>>>> merge
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code
> in
> >>>> the
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> >> plug-in,
> >>>>>>>> which
> >>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> >> (although
> >>>>>>>> it
> >>>>>>>>>> does
> >>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is
> actually
> >> in
> >>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
> >> url
> >>>>>>>> field
> >>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
> >> the
> >>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the
> hypervisor
> >> in
> >>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
> >> of
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> >> checks
> >>>>>>>> for
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> >> hypervisor
> >>>>>>>> data
> >>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes
> to
> >>>>>>>> attach
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> >>>> detach a
> >>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> >>>>>>>> structure
> >>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
> >> 4.2
> >>>>>>>>>> unless
> >>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> >>>> have
> >>>>>>>> some
> >>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> >> field.
> >>>> I
> >>>>>>>>>> think
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation
> regarding
> >>>> the
> >>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details
> are
> >>>> not
> >>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> >> listStoragePool
> >>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be
> useful
> >>>> for
> >>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS
> here.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >>>> Offering-by-Disk
> >>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to
> be
> >>>>>>>> able to
> >>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> >>>> table.
> >>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst
> IOPS
> >> in
> >>>>>>>> the
> >>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the
> DB.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Querying the SAN only indicates the number of IOPS currently in use. The allocator needs to check the number of IOPS committed which is tracked by CloudStack. For 4.2, we should be able to query the number of IOPS committed to a DataStore, and determine whether or not the number requested can be fulfilled by that device. It seems to be that a DataStore#getCommittedIOPS() : Long method would be sufficient. DataStore's that don't support provisioned IOPS would return null.
As I mentioned previously, I am very reluctant for any feature to come into master that can exhaust resources.
Thanks,
-John
On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Yeah, I'm not sure I could come up with anything near an accurate
> assessment of how many IOPS are currently available on the SAN (or even a
> total number that are available for volumes). Not sure if there's yet an
> API call for that.
>
> If I did know this number (total number of IOPS supported by the SAN), we'd
> still have to keep track of the total number of volumes we've created from
> CS on the SAN in terms of their IOPS. Also, if an admin issues an API call
> directly to the SAN to tweak the number of IOPS on a given volume or set of
> volumes (not supported from CS, but supported via the SolidFire API), our
> numbers in CS would be off.
>
> I'm thinking verifying sufficient number of IOPS is a really good idea for
> a future release.
>
> For 4.2 I think we should stick to having the allocator detect if storage
> QoS is desired and if the storage pool in question supports that feature.
>
> If you really are over provisioned on your SAN in terms of IOPS or
> capacity, the SAN can let the admin know in several different ways (e-mail,
> SNMP, GUI).
>
>
> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Please see my comments in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Comments below in red.
>>>
>>> Thanks
>>>
>>>
>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> Overall, I agree with the steps to below for 4.2. However, we may want
>> to
>>>> throw an exception if we can not fulfill a requested QoS. If the user
>> is
>>>> expecting that the hypervisor will provide a particular QoS, and that is
>>>> not possible, it seems like we should inform them rather than silently
>>>> ignoring their request.
>>>>
>>>
>>> Sure, that sounds reasonable.
>>>
>>> We'd have to come up with some way for the allocators to know about the
>>> requested storage QoS and then query the candidate drivers.
>>>
>>> Any thoughts on how we might do that?
>>>
>>>
>>>>
>>>> To collect my thoughts from previous parts of the thread, I am
>>>> uncomfortable with the idea that the management server can overcommit a
>>>> resource. You had mentioned querying the device for available IOPS.
>> While
>>>> that would be nice, it seems like we could fall back to a max IOPS and
>>>> overcommit factor manually calculated and entered by the
>>>> administrator/operator. I think such threshold and allocation rails
>> should
>>>> be added for both provisioned IOPS and throttled I/O -- it is a basic
>>>> feature of any cloud orchestration platform.
>>>>
>>>
>>> Are you thinking this ability would make it into 4.2? Just curious what
>>> release we're talking about here. For the SolidFire SAN, you might have,
>>> say, four separate storage nodes to start (200,000 IOPS) and then later
>> add
>>> a new node (now you're at 250,000 IOPS). CS would have to have a way to
>>> know that the number of supported IOPS has increased.
>>
>> Yes, I think we need some *basic*/conservative rails in 4.2. For example,
>> we may only support expanding capacity in 4.2, and not handle any
>> re-balance scenarios -- node failure, addition, etc. Extrapolating a
>> bit, the throttled I/O enhancement seems like it needs a similar set of
>> rails defined per host.
>>
>>>
>>>
>>>>
>>>> For 4.3, I don't like the idea that a QoS would be expressed in a
>>>> implementation specific manner. I think we need to arrive at a general
>>>> model that can be exploited by the allocators and planners. We should
>>>> restrict implementation specific key-value pairs (call them details,
>>>> extended data, whatever) to information that is unique to the driver and
>>>> would provide no useful information to the management server's
>>>> orchestration functions. A resource QoS does not seem to fit those
>>>> criteria.
>>>>
>>>
>>> I wonder if this would be a good discussion topic for Sunday's CS Collab
>>> Conf hack day that Joe just sent out an e-mail about?
>>
>> Yes, it would -- I will put something in the wiki topic. It will also be
>> part of my talk on Monday -- How to Run from Zombie which include some of
>> my opinions on the topic.
>>
>>>
>>>
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> So, here's my suggestion for 4.2:
>>>>>
>>>>> Accept the values as they are currently required (four new fields for
>>>> Wei's
>>>>> feature or two new fields for mine).
>>>>>
>>>>> The Add Disk Offering dialog needs three new radio buttons:
>>>>>
>>>>> 1) No QoS
>>>>>
>>>>> 2) Hypervisor QoS
>>>>>
>>>>> 3) Storage Qos
>>>>>
>>>>> The admin needs to specify storage tags that only map to storage that
>>>>> supports Storage QoS.
>>>>>
>>>>> The admin needs to be aware for Hypervisor QoS that unless all
>>>> hypervisors
>>>>> in use support the new fields, they may not be enforced.
>>>>>
>>>>> Post 4.3:
>>>>>
>>>>> Come up with a way to more generally enter these parameters (probably
>>>> just
>>>>> key/value pairs sent to the drivers).
>>>>>
>>>>> Have the drivers expose their feature set so the allocators can
>> consider
>>>>> them more fully and throw an exception if there is not a sufficient
>>>> match.
>>>>>
>>>>>
>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>>>> burden
>>>>>> on the admin to configure a Disk Offering in a way that makes sense.
>> For
>>>>>> example, if he wants to use storage QoS with supported Min and Max
>>>> values,
>>>>>> he'll have to put in a storage tag that references the SolidFire
>> primary
>>>>>> storage (plug-in). If he puts in a storage tag that doesn't, then he's
>>>> not
>>>>>> going to get the Min and Max feature. We could add help text to the
>>>> pop-up
>>>>>> dialog that's displayed when you click in the Min and Max text fields.
>>>>>>
>>>>>> Same idea for Wei's feature.
>>>>>>
>>>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>>>>> approach for post 4.2.
>>>>>>
>>>>>> Maybe in the future we could have the drivers advertise their
>>>> capabilities
>>>>>> and if the allocator feels a request is not being satisfied (say Min
>> was
>>>>>> entered, but it not's supported by any storage plug-in) it can throw
>> an
>>>>>> exception.
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> Comments below in red.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> Please see my comment in-line below.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Hi John,
>>>>>>>>>
>>>>>>>>> I've put comments below in red.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jburwell@basho.com
>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> First and foremost, we must ensure that these two features are
>>>>>>>> mutually
>>>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>>>>>>> contains both
>>>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>>>> non-deterministic
>>>>>>>>>> operations.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Agreed
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>>>> oriented
>>>>>>>>>> solves the problem in short term. As I understand storage tags,
>> we
>>>>>>>> have no
>>>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>>>>>>> necessarily
>>>>>>>>>> intending that we implement this allocation model in 4.3, but
>>>> instead,
>>>>>>>>>> determine if this type model would be one we would want to support
>>>> in
>>>>>>>> the
>>>>>>>>>> future. If so, I would encourage us to ensure that the data model
>>>> and
>>>>>>>>>> current implementation would not preclude evolution in that
>>>>>>>> direction. My
>>>>>>>>>> view is that this type of allocation model is what user's expect
>> of
>>>>>>>> "cloud"
>>>>>>>>>> systems -- selecting the best available resource set to fulfill a
>>>>>>>> set of
>>>>>>>>>> system requirements.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I believe we have meet your requirement here in that what we've
>>>>>>>> implemented
>>>>>>>>> should not make refinement difficult in the future. If we don't
>>>> modify
>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
>>>>>>>> changes
>>>>>>>>> to enhance the current functioning of the system.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Looking through both patches, I have to say that the aggregated
>> result
>>>>>>>> seems a bit confusing. There are six new attributes for throttled
>>>> I/O and
>>>>>>>> two for provisioned IOPS with no obvious grouping. My concern is
>> not
>>>>>>>> technical, but rather, about maintainability. At minimum, Javadoc
>>>> should
>>>>>>>> be added explaining the two sets of attributes and their mutual
>>>> exclusion.
>>>>>>>>
>>>>>>>
>>>>>>> I agree: We need JavaDoc to explain them and their mutual exclusion.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> The other part that is interesting is that throttled I/O provides
>> both
>>>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses
>> a
>>>>>>>> range. In order to select the best available resource to fulfill a
>>>> QoS, we
>>>>>>>> would need to have the QoS expression normalized in terms of units
>>>> (IOPS or
>>>>>>>> bytes) and their expression (rate vs. range). If we want to
>> achieve a
>>>>>>>> model like I described, I think we would need to resolve this issue
>>>> in 4.2
>>>>>>>> to ensure a viable migration path.
>>>>>>>
>>>>>>>
>>>>>>> I think we're not likely to be able to normalize the input for 4.2.
>>>> Plus
>>>>>>> people probably want to input the data in terms they're familiar with
>>>> for
>>>>>>> the products in question.
>>>>>>>
>>>>>>> Ideally we would fix the way we do storage tagging and let the user
>>>> send
>>>>>>> key/value pairs to each vendor that could be selected due to a given
>>>>>>> storage tag. I'm still not sure that would solve it because what
>>>> happens if
>>>>>>> you change the storage tag of a given Primary Storage after having
>>>> created
>>>>>>> a Disk Offering?
>>>>>>>
>>>>>>> Basically storage tagging is kind of a mess and we should re-think
>> it.
>>>>>>>
>>>>>>> Also, we need to have a way for the drivers to expose their supported
>>>>>>> feature sets so the allocators can make good choices.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> As I think through the implications of these requirements and
>>>> reflect
>>>>>>>> on
>>>>>>>>>> the reviews, I don't understand why they haven't already impacted
>>>> the
>>>>>>>>>> allocators and planners. As it stands, the current provisioned
>> IOPS
>>>>>>>> has no
>>>>>>>>>> checks to ensure that the volumes are allocated to devices that
>> have
>>>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I understand
>>>> the
>>>>>>>>>> current patch, we can overcommit storage resources -- potentially
>>>>>>>> causing
>>>>>>>>>> none of the QoS obligations from being fulfilled. It seems to me
>>>>>>>> that a
>>>>>>>>>> DataStore supporting provisioned IOPS should express the maximum
>>>> IOPS
>>>>>>>> which
>>>>>>>>>> it can support and some type of overcommitment factor. This
>>>>>>>> information
>>>>>>>>>> should be used by the storage allocators to determine the device
>>>> best
>>>>>>>> able
>>>>>>>>>> to support the resources needs of a volume. It seems that a
>> similar
>>>>>>>> set of
>>>>>>>>>> considerations would need to be added to the Hypervisor layer
>> which
>>>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yeah, for this first release, we just followed the path that was
>>>>>>>> previously
>>>>>>>>> established for other properties you see on dialogs in CS: Just
>>>> because
>>>>>>>>> they're there doesn't mean the vendor your VM is deployed to
>> supports
>>>>>>>> them.
>>>>>>>>> It is then up to the admin to make sure he inputs, say, a storage
>> tag
>>>>>>>> that
>>>>>>>>> confines the deployment only to storage that supports the selected
>>>>>>>>> features. This is not ideal, but it's kind of the way CloudStack
>>>> works
>>>>>>>>> today.
>>>>>>>>
>>>>>>>> I understand the tag functionality, and the need for the
>> administrator
>>>>>>>> to very carefully construct offerings. My concern is that we over
>>>>>>>> guarantee a resource's available IOPS. For the purposes of
>>>> illustration,
>>>>>>>> let's say we have a SolidFire, and the max IOPS for that device is
>>>> 100000.
>>>>>>>> We also know that we can safely oversubscribe by 50%. Therefore,
>> we
>>>>>>>> need to ensure that we don't allocate more than 150,000 guaranteed
>>>> IOPS
>>>>>>>> from that device. Intuitively, it seems like the DataStore
>>>> configuration
>>>>>>>> should have a max assignable IOPS and overcommitment factor. As we
>>>>>>>> allocate volumes and attach VMs, we need to ensure that guarantee
>>>> more IOPS
>>>>>>>> exceed the configured maximum for a DataStore. Does that make
>> sense?
>>>>>>>>
>>>>>>>
>>>>>>> I think that's a good idea for a future enhancement. I'm not even
>> sure
>>>> I
>>>>>>> can query the SAN to find out how many IOPS safely remain. I'd have
>> to
>>>> get
>>>>>>> all of the min values for all of the volumes on the SAN and total
>> them
>>>> up,
>>>>>>> I suppose, and subtract it from the total (user facing) supported
>> IOPS
>>>> of
>>>>>>> the system.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Another question occurs to me -- should we allow non-QoS resources
>>>> to
>>>>>>>> be
>>>>>>>>>> assigned to hosts/storage devices that ensure QoS? For
>> provisioned
>>>>>>>> IOPS, I
>>>>>>>>>> think a side effect of the current implementation is SolidFire
>>>> volumes
>>>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
>> seems
>>>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side with
>> QoS
>>>>>>>> VMs.
>>>>>>>>>> In this scenario, a greedy, unbounded VM could potentially starve
>>>> out
>>>>>>>> the
>>>>>>>>>> other VMs on the host -- preventing the QoSes defined the
>> collocated
>>>>>>>> VMs
>>>>>>>>>> from being fulfilled.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> You can make SolidFire volumes (inside and outside of CS) and not
>>>>>>>> specify
>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>>>> defaults
>>>>>>>> we
>>>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will
>>>>>>>> have
>>>>>>>>> your Mins met.
>>>>>>>>>
>>>>>>>>> It sounds like you're perhaps looking for a storage tags exclusions
>>>>>>>> list,
>>>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>>>> volume
>>>>>>>> to
>>>>>>>>> storage with these following tags).
>>>>>>>>
>>>>>>>> I don't like the idea of a storage tags exclusion list as it would
>>>>>>>> complicate component assembly. It would require a storage plugin to
>>>>>>>> anticipate all of the possible component assemblies and determine
>> the
>>>>>>>> invalid relationships. I prefer that drivers express their
>>>> capabilities
>>>>>>>> which can be matched to a set of requested requirements.
>>>>>>>>
>>>>>>>
>>>>>>> I'm not sure why a storage plug-in would care about inclusion or
>>>>>>> exclusion lists. It just needs to advertise its functionality in a
>> way
>>>> the
>>>>>>> allocator understands.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
>> limits
>>>>>>>> IOPS,
>>>>>>>>> it does not solve the Noisy Neighbor problem. Only a system with
>>>>>>>> guaranteed
>>>>>>>>> minimum IOPS does this.
>>>>>>>>
>>>>>>>> As I said, for SolidFire, it sounds like this problem does not
>> exist.
>>>>>>>> However, I am concerned with the more general case as we supported
>>>> more
>>>>>>>> devices with provisioned IOPS.
>>>>>>>>
>>>>>>>
>>>>>>> Post 4.2 we need to investigate a way to pass vendor-specific values
>> to
>>>>>>> drivers. Min and Max and pretty industry standard for provisioned
>>>> IOPS, but
>>>>>>> what if you break them out by read and write or do something else? We
>>>> need
>>>>>>> a more general mechanism.
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> In my opinion, we need to ensure that hypervisor throttled I/O
>> and
>>>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Agreed
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> We also need to understand the implications of these QoS
>> guarantees
>>>> on
>>>>>>>>>> operation of the system to ensure that the underlying hardware
>>>>>>>> resources
>>>>>>>>>> can fulfill them. Given the time frame, we will likely be forced
>> to
>>>>>>>> make
>>>>>>>>>> compromises to achieve these goals, and refine the implementation
>> in
>>>>>>>> 4.3.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I agree, John. I also think you've come up with some great ideas
>> for
>>>>>>>> 4.3. :)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
>>>>>>>> tags).
>>>>>>>>>> I
>>>>>>>>>>> believe John was just throwing out an idea that - in addition to
>>>>>>>> storage
>>>>>>>>>>> tags - we could look into these allocators (storage QoS being
>>>>>>>> preferred,
>>>>>>>>>>> then hypervisor QoS if storage QoS is not available, but
>> hypervisor
>>>>>>>> QoS
>>>>>>>>>> is).
>>>>>>>>>>>
>>>>>>>>>>> I think John's concern is that you can enter in values for Wei's
>>>> and
>>>>>>>> my
>>>>>>>>>>> feature that are not honored by other vendors (at least yet), so
>> he
>>>>>>>> was
>>>>>>>>>>> hoping - in addition to storage tags - for the allocators to
>> prefer
>>>>>>>> these
>>>>>>>>>>> vendors when these fields are filled in. As it stands today in
>>>>>>>>>> CloudStack,
>>>>>>>>>>> we already have this kind of an issue with other features (fields
>>>> in
>>>>>>>>>>> dialogs for features that not all vendors support). Perhaps post
>>>> 4.2
>>>>>>>> we
>>>>>>>>>>> could look into generic name/value pairs (this is how OpenStack
>>>>>>>> addresses
>>>>>>>>>>> the issue).
>>>>>>>>>>>
>>>>>>>>>>> Honestly, I think we're too late in the game (two weeks until
>> code
>>>>>>>>>> freeze)
>>>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>>>
>>>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields
>>>>>>>> and my
>>>>>>>>>>> fields as is, make sure only one or the other feature has data
>>>>>>>> entered
>>>>>>>>>> for
>>>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>>>
>>>>>>>>>>> Then let's step back and look into a more general-purpose design
>>>>>>>> that can
>>>>>>>>>>> be applied throughout CloudStack where we have these situations.
>>>>>>>>>>>
>>>>>>>>>>> What do you think?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
>> jburwell@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> I just published my review @
>> https://reviews.apache.org/r/11479/.
>>>>>>>>>>>>
>>>>>>>>>>>> I apologize for the delay,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> No problem, John.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I still want to re-review it by myself before coming up with a
>>>> new
>>>>>>>>>> patch
>>>>>>>>>>>>> file.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
>>>> in
>>>>>>>> and
>>>>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>>>> jburwell@basho.com
>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>>>> offline
>>>>>>>> ATM,
>>>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
>>>> ),
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
>> we
>>>>>>>> wanted
>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I plan to review the code again tomorrow morning or
>> afternoon,
>>>>>>>> then
>>>>>>>>>>>> send
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>>>>>>>> code,
>>>>>>>>>>>>>> except
>>>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
>>>> into
>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>>>>>>>> branch,
>>>>>>>>>> but
>>>>>>>>>>>> -
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>>>>>> SolidFire
>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>>>> master.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
>> merge
>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>>>>>>>>> overlapping
>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>>>> patches
>>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>>>> SolidFire
>>>>>>>> are
>>>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>>>> to the object_store branch or there are circular
>> dependency
>>>>>>>>>> between
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
>> can
>>>>>>>>>>>> determine
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
>> his
>>>>>>>> branch
>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>>>>>>> merge
>>>>>>>>>> and
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
>>>> the
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
>> plug-in,
>>>>>>>> which
>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
>> (although
>>>>>>>> it
>>>>>>>>>> does
>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
>> in
>>>>>>>> use))
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
>> url
>>>>>>>> field
>>>>>>>>>>>> (if
>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
>> the
>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
>> in
>>>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
>> of
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
>> checks
>>>>>>>> for
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
>> hypervisor
>>>>>>>> data
>>>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>>>>>>>> attach
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>>>> detach a
>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>>>>>>> structure
>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
>> 4.2
>>>>>>>>>> unless
>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
>>>> have
>>>>>>>> some
>>>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
>> field.
>>>> I
>>>>>>>>>> think
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
>>>> the
>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
>>>> not
>>>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
>> listStoragePool
>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
>>>> for
>>>>>>>>>>>> clients
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>>>> Offering-by-Disk
>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>>>>>>>> able to
>>>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>>>> table.
>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
>> in
>>>>>>>> the
>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, I'm not sure I could come up with anything near an accurate
assessment of how many IOPS are currently available on the SAN (or even a
total number that are available for volumes). Not sure if there's yet an
API call for that.
If I did know this number (total number of IOPS supported by the SAN), we'd
still have to keep track of the total number of volumes we've created from
CS on the SAN in terms of their IOPS. Also, if an admin issues an API call
directly to the SAN to tweak the number of IOPS on a given volume or set of
volumes (not supported from CS, but supported via the SolidFire API), our
numbers in CS would be off.
I'm thinking verifying sufficient number of IOPS is a really good idea for
a future release.
For 4.2 I think we should stick to having the allocator detect if storage
QoS is desired and if the storage pool in question supports that feature.
If you really are over provisioned on your SAN in terms of IOPS or
capacity, the SAN can let the admin know in several different ways (e-mail,
SNMP, GUI).
On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my comments in-line below.
>
> Thanks,
> -John
>
> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Comments below in red.
> >
> > Thanks
> >
> >
> > On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Overall, I agree with the steps to below for 4.2. However, we may want
> to
> >> throw an exception if we can not fulfill a requested QoS. If the user
> is
> >> expecting that the hypervisor will provide a particular QoS, and that is
> >> not possible, it seems like we should inform them rather than silently
> >> ignoring their request.
> >>
> >
> > Sure, that sounds reasonable.
> >
> > We'd have to come up with some way for the allocators to know about the
> > requested storage QoS and then query the candidate drivers.
> >
> > Any thoughts on how we might do that?
> >
> >
> >>
> >> To collect my thoughts from previous parts of the thread, I am
> >> uncomfortable with the idea that the management server can overcommit a
> >> resource. You had mentioned querying the device for available IOPS.
> While
> >> that would be nice, it seems like we could fall back to a max IOPS and
> >> overcommit factor manually calculated and entered by the
> >> administrator/operator. I think such threshold and allocation rails
> should
> >> be added for both provisioned IOPS and throttled I/O -- it is a basic
> >> feature of any cloud orchestration platform.
> >>
> >
> > Are you thinking this ability would make it into 4.2? Just curious what
> > release we're talking about here. For the SolidFire SAN, you might have,
> > say, four separate storage nodes to start (200,000 IOPS) and then later
> add
> > a new node (now you're at 250,000 IOPS). CS would have to have a way to
> > know that the number of supported IOPS has increased.
>
> Yes, I think we need some *basic*/conservative rails in 4.2. For example,
> we may only support expanding capacity in 4.2, and not handle any
> re-balance scenarios -- node failure, addition, etc. Extrapolating a
> bit, the throttled I/O enhancement seems like it needs a similar set of
> rails defined per host.
>
> >
> >
> >>
> >> For 4.3, I don't like the idea that a QoS would be expressed in a
> >> implementation specific manner. I think we need to arrive at a general
> >> model that can be exploited by the allocators and planners. We should
> >> restrict implementation specific key-value pairs (call them details,
> >> extended data, whatever) to information that is unique to the driver and
> >> would provide no useful information to the management server's
> >> orchestration functions. A resource QoS does not seem to fit those
> >> criteria.
> >>
> >
> > I wonder if this would be a good discussion topic for Sunday's CS Collab
> > Conf hack day that Joe just sent out an e-mail about?
>
> Yes, it would -- I will put something in the wiki topic. It will also be
> part of my talk on Monday -- How to Run from Zombie which include some of
> my opinions on the topic.
>
> >
> >
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> So, here's my suggestion for 4.2:
> >>>
> >>> Accept the values as they are currently required (four new fields for
> >> Wei's
> >>> feature or two new fields for mine).
> >>>
> >>> The Add Disk Offering dialog needs three new radio buttons:
> >>>
> >>> 1) No QoS
> >>>
> >>> 2) Hypervisor QoS
> >>>
> >>> 3) Storage Qos
> >>>
> >>> The admin needs to specify storage tags that only map to storage that
> >>> supports Storage QoS.
> >>>
> >>> The admin needs to be aware for Hypervisor QoS that unless all
> >> hypervisors
> >>> in use support the new fields, they may not be enforced.
> >>>
> >>> Post 4.3:
> >>>
> >>> Come up with a way to more generally enter these parameters (probably
> >> just
> >>> key/value pairs sent to the drivers).
> >>>
> >>> Have the drivers expose their feature set so the allocators can
> consider
> >>> them more fully and throw an exception if there is not a sufficient
> >> match.
> >>>
> >>>
> >>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> My thinking is, for 4.2, while not ideal, we will need to put some
> >> burden
> >>>> on the admin to configure a Disk Offering in a way that makes sense.
> For
> >>>> example, if he wants to use storage QoS with supported Min and Max
> >> values,
> >>>> he'll have to put in a storage tag that references the SolidFire
> primary
> >>>> storage (plug-in). If he puts in a storage tag that doesn't, then he's
> >> not
> >>>> going to get the Min and Max feature. We could add help text to the
> >> pop-up
> >>>> dialog that's displayed when you click in the Min and Max text fields.
> >>>>
> >>>> Same idea for Wei's feature.
> >>>>
> >>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
> >>>> approach for post 4.2.
> >>>>
> >>>> Maybe in the future we could have the drivers advertise their
> >> capabilities
> >>>> and if the allocator feels a request is not being satisfied (say Min
> was
> >>>> entered, but it not's supported by any storage plug-in) it can throw
> an
> >>>> exception.
> >>>>
> >>>>
> >>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> Comments below in red.
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>>
> >>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Please see my comment in-line below.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Hi John,
> >>>>>>>
> >>>>>>> I've put comments below in red.
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jburwell@basho.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> First and foremost, we must ensure that these two features are
> >>>>>> mutually
> >>>>>>>> exclusive in 4.2. We don't want to find a configuration that
> >>>>>> contains both
> >>>>>>>> hypervisor and storage IOPS guarantees that leads to
> >> non-deterministic
> >>>>>>>> operations.
> >>>>>>>
> >>>>>>>
> >>>>>>> Agreed
> >>>>>>>
> >>>>>>>
> >>>>>>>> Restricting QoS expression to be either hypervisor or storage
> >> oriented
> >>>>>>>> solves the problem in short term. As I understand storage tags,
> we
> >>>>>> have no
> >>>>>>>> means of expressing this type of mutual exclusion. I wasn't
> >>>>>> necessarily
> >>>>>>>> intending that we implement this allocation model in 4.3, but
> >> instead,
> >>>>>>>> determine if this type model would be one we would want to support
> >> in
> >>>>>> the
> >>>>>>>> future. If so, I would encourage us to ensure that the data model
> >> and
> >>>>>>>> current implementation would not preclude evolution in that
> >>>>>> direction. My
> >>>>>>>> view is that this type of allocation model is what user's expect
> of
> >>>>>> "cloud"
> >>>>>>>> systems -- selecting the best available resource set to fulfill a
> >>>>>> set of
> >>>>>>>> system requirements.
> >>>>>>>>
> >>>>>>>
> >>>>>>> I believe we have meet your requirement here in that what we've
> >>>>>> implemented
> >>>>>>> should not make refinement difficult in the future. If we don't
> >> modify
> >>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
> >>>>>> changes
> >>>>>>> to enhance the current functioning of the system.
> >>>>>>>
> >>>>>>
> >>>>>> Looking through both patches, I have to say that the aggregated
> result
> >>>>>> seems a bit confusing. There are six new attributes for throttled
> >> I/O and
> >>>>>> two for provisioned IOPS with no obvious grouping. My concern is
> not
> >>>>>> technical, but rather, about maintainability. At minimum, Javadoc
> >> should
> >>>>>> be added explaining the two sets of attributes and their mutual
> >> exclusion.
> >>>>>>
> >>>>>
> >>>>> I agree: We need JavaDoc to explain them and their mutual exclusion.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> The other part that is interesting is that throttled I/O provides
> both
> >>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses
> a
> >>>>>> range. In order to select the best available resource to fulfill a
> >> QoS, we
> >>>>>> would need to have the QoS expression normalized in terms of units
> >> (IOPS or
> >>>>>> bytes) and their expression (rate vs. range). If we want to
> achieve a
> >>>>>> model like I described, I think we would need to resolve this issue
> >> in 4.2
> >>>>>> to ensure a viable migration path.
> >>>>>
> >>>>>
> >>>>> I think we're not likely to be able to normalize the input for 4.2.
> >> Plus
> >>>>> people probably want to input the data in terms they're familiar with
> >> for
> >>>>> the products in question.
> >>>>>
> >>>>> Ideally we would fix the way we do storage tagging and let the user
> >> send
> >>>>> key/value pairs to each vendor that could be selected due to a given
> >>>>> storage tag. I'm still not sure that would solve it because what
> >> happens if
> >>>>> you change the storage tag of a given Primary Storage after having
> >> created
> >>>>> a Disk Offering?
> >>>>>
> >>>>> Basically storage tagging is kind of a mess and we should re-think
> it.
> >>>>>
> >>>>> Also, we need to have a way for the drivers to expose their supported
> >>>>> feature sets so the allocators can make good choices.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> As I think through the implications of these requirements and
> >> reflect
> >>>>>> on
> >>>>>>>> the reviews, I don't understand why they haven't already impacted
> >> the
> >>>>>>>> allocators and planners. As it stands, the current provisioned
> IOPS
> >>>>>> has no
> >>>>>>>> checks to ensure that the volumes are allocated to devices that
> have
> >>>>>>>> capacity to fulfill the requested QoS. Therefore, as I understand
> >> the
> >>>>>>>> current patch, we can overcommit storage resources -- potentially
> >>>>>> causing
> >>>>>>>> none of the QoS obligations from being fulfilled. It seems to me
> >>>>>> that a
> >>>>>>>> DataStore supporting provisioned IOPS should express the maximum
> >> IOPS
> >>>>>> which
> >>>>>>>> it can support and some type of overcommitment factor. This
> >>>>>> information
> >>>>>>>> should be used by the storage allocators to determine the device
> >> best
> >>>>>> able
> >>>>>>>> to support the resources needs of a volume. It seems that a
> similar
> >>>>>> set of
> >>>>>>>> considerations would need to be added to the Hypervisor layer
> which
> >>>>>>>> allocating a VM to a host to prevent oversubscription.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Yeah, for this first release, we just followed the path that was
> >>>>>> previously
> >>>>>>> established for other properties you see on dialogs in CS: Just
> >> because
> >>>>>>> they're there doesn't mean the vendor your VM is deployed to
> supports
> >>>>>> them.
> >>>>>>> It is then up to the admin to make sure he inputs, say, a storage
> tag
> >>>>>> that
> >>>>>>> confines the deployment only to storage that supports the selected
> >>>>>>> features. This is not ideal, but it's kind of the way CloudStack
> >> works
> >>>>>>> today.
> >>>>>>
> >>>>>> I understand the tag functionality, and the need for the
> administrator
> >>>>>> to very carefully construct offerings. My concern is that we over
> >>>>>> guarantee a resource's available IOPS. For the purposes of
> >> illustration,
> >>>>>> let's say we have a SolidFire, and the max IOPS for that device is
> >> 100000.
> >>>>>> We also know that we can safely oversubscribe by 50%. Therefore,
> we
> >>>>>> need to ensure that we don't allocate more than 150,000 guaranteed
> >> IOPS
> >>>>>> from that device. Intuitively, it seems like the DataStore
> >> configuration
> >>>>>> should have a max assignable IOPS and overcommitment factor. As we
> >>>>>> allocate volumes and attach VMs, we need to ensure that guarantee
> >> more IOPS
> >>>>>> exceed the configured maximum for a DataStore. Does that make
> sense?
> >>>>>>
> >>>>>
> >>>>> I think that's a good idea for a future enhancement. I'm not even
> sure
> >> I
> >>>>> can query the SAN to find out how many IOPS safely remain. I'd have
> to
> >> get
> >>>>> all of the min values for all of the volumes on the SAN and total
> them
> >> up,
> >>>>> I suppose, and subtract it from the total (user facing) supported
> IOPS
> >> of
> >>>>> the system.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Another question occurs to me -- should we allow non-QoS resources
> >> to
> >>>>>> be
> >>>>>>>> assigned to hosts/storage devices that ensure QoS? For
> provisioned
> >>>>>> IOPS, I
> >>>>>>>> think a side effect of the current implementation is SolidFire
> >> volumes
> >>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it
> seems
> >>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side with
> QoS
> >>>>>> VMs.
> >>>>>>>> In this scenario, a greedy, unbounded VM could potentially starve
> >> out
> >>>>>> the
> >>>>>>>> other VMs on the host -- preventing the QoSes defined the
> collocated
> >>>>>> VMs
> >>>>>>>> from being fulfilled.
> >>>>>>>>
> >>>>>>>
> >>>>>>> You can make SolidFire volumes (inside and outside of CS) and not
> >>>>>> specify
> >>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
> >> defaults
> >>>>>> we
> >>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will
> >>>>>> have
> >>>>>>> your Mins met.
> >>>>>>>
> >>>>>>> It sounds like you're perhaps looking for a storage tags exclusions
> >>>>>> list,
> >>>>>>> which might be nice to have at some point (i.e. don't deploy my
> >> volume
> >>>>>> to
> >>>>>>> storage with these following tags).
> >>>>>>
> >>>>>> I don't like the idea of a storage tags exclusion list as it would
> >>>>>> complicate component assembly. It would require a storage plugin to
> >>>>>> anticipate all of the possible component assemblies and determine
> the
> >>>>>> invalid relationships. I prefer that drivers express their
> >> capabilities
> >>>>>> which can be matched to a set of requested requirements.
> >>>>>>
> >>>>>
> >>>>> I'm not sure why a storage plug-in would care about inclusion or
> >>>>> exclusion lists. It just needs to advertise its functionality in a
> way
> >> the
> >>>>> allocator understands.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> I agree with your assessment of Hypervisor QoS. Since it only
> limits
> >>>>>> IOPS,
> >>>>>>> it does not solve the Noisy Neighbor problem. Only a system with
> >>>>>> guaranteed
> >>>>>>> minimum IOPS does this.
> >>>>>>
> >>>>>> As I said, for SolidFire, it sounds like this problem does not
> exist.
> >>>>>> However, I am concerned with the more general case as we supported
> >> more
> >>>>>> devices with provisioned IOPS.
> >>>>>>
> >>>>>
> >>>>> Post 4.2 we need to investigate a way to pass vendor-specific values
> to
> >>>>> drivers. Min and Max and pretty industry standard for provisioned
> >> IOPS, but
> >>>>> what if you break them out by read and write or do something else? We
> >> need
> >>>>> a more general mechanism.
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> In my opinion, we need to ensure that hypervisor throttled I/O
> and
> >>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
> >>>>>>>
> >>>>>>>
> >>>>>>> Agreed
> >>>>>>>
> >>>>>>>
> >>>>>>>> We also need to understand the implications of these QoS
> guarantees
> >> on
> >>>>>>>> operation of the system to ensure that the underlying hardware
> >>>>>> resources
> >>>>>>>> can fulfill them. Given the time frame, we will likely be forced
> to
> >>>>>> make
> >>>>>>>> compromises to achieve these goals, and refine the implementation
> in
> >>>>>> 4.3.
> >>>>>>>>
> >>>>>>>
> >>>>>>> I agree, John. I also think you've come up with some great ideas
> for
> >>>>>> 4.3. :)
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
> >>>>>> tags).
> >>>>>>>> I
> >>>>>>>>> believe John was just throwing out an idea that - in addition to
> >>>>>> storage
> >>>>>>>>> tags - we could look into these allocators (storage QoS being
> >>>>>> preferred,
> >>>>>>>>> then hypervisor QoS if storage QoS is not available, but
> hypervisor
> >>>>>> QoS
> >>>>>>>> is).
> >>>>>>>>>
> >>>>>>>>> I think John's concern is that you can enter in values for Wei's
> >> and
> >>>>>> my
> >>>>>>>>> feature that are not honored by other vendors (at least yet), so
> he
> >>>>>> was
> >>>>>>>>> hoping - in addition to storage tags - for the allocators to
> prefer
> >>>>>> these
> >>>>>>>>> vendors when these fields are filled in. As it stands today in
> >>>>>>>> CloudStack,
> >>>>>>>>> we already have this kind of an issue with other features (fields
> >> in
> >>>>>>>>> dialogs for features that not all vendors support). Perhaps post
> >> 4.2
> >>>>>> we
> >>>>>>>>> could look into generic name/value pairs (this is how OpenStack
> >>>>>> addresses
> >>>>>>>>> the issue).
> >>>>>>>>>
> >>>>>>>>> Honestly, I think we're too late in the game (two weeks until
> code
> >>>>>>>> freeze)
> >>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>
> >>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields
> >>>>>> and my
> >>>>>>>>> fields as is, make sure only one or the other feature has data
> >>>>>> entered
> >>>>>>>> for
> >>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>
> >>>>>>>>> Then let's step back and look into a more general-purpose design
> >>>>>> that can
> >>>>>>>>> be applied throughout CloudStack where we have these situations.
> >>>>>>>>>
> >>>>>>>>> What do you think?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> jburwell@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> I just published my review @
> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>
> >>>>>>>>>> I apologize for the delay,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> No problem, John.
> >>>>>>>>>>>
> >>>>>>>>>>> I still want to re-review it by myself before coming up with a
> >> new
> >>>>>>>> patch
> >>>>>>>>>>> file.
> >>>>>>>>>>>
> >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
> >> in
> >>>>>> and
> >>>>>>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> >> jburwell@basho.com
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I just realized that I forgot to publish my review. I am
> >> offline
> >>>>>> ATM,
> >>>>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
> >> ),
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas
> we
> >>>>>> wanted
> >>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I plan to review the code again tomorrow morning or
> afternoon,
> >>>>>> then
> >>>>>>>>>> send
> >>>>>>>>>>>> in
> >>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
> >>>>>> code,
> >>>>>>>>>>>> except
> >>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
> >> into
> >>>>>>>>>> master
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> >>>>>> branch,
> >>>>>>>> but
> >>>>>>>>>> -
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> >>>>>> SolidFire
> >>>>>>>>>> branch
> >>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>> the object_store branch before object_store goes into
> >> master.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this
> merge
> >>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> >>>>>>>> overlapping
> >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
> >> patches
> >>>>>> to
> >>>>>>>> be
> >>>>>>>>>> as
> >>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
> >> SolidFire
> >>>>>> are
> >>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>> to the object_store branch or there are circular
> dependency
> >>>>>>>> between
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we
> can
> >>>>>>>>>> determine
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into
> his
> >>>>>> branch
> >>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
> >>>>>> merge
> >>>>>>>> and
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
> >> the
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage
> plug-in,
> >>>>>> which
> >>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use
> (although
> >>>>>> it
> >>>>>>>> does
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually
> in
> >>>>>> use))
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the
> url
> >>>>>> field
> >>>>>>>>>> (if
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in
> the
> >>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor
> in
> >>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place
> of
> >>>>>> the
> >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor
> checks
> >>>>>> for
> >>>>>>>> the
> >>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary
> hypervisor
> >>>>>> data
> >>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
> >>>>>> attach
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> >> detach a
> >>>>>>>>>> volume,
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> >>>>>> structure
> >>>>>>>> is
> >>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in
> 4.2
> >>>>>>>> unless
> >>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> >> have
> >>>>>> some
> >>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details
> field.
> >> I
> >>>>>>>> think
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
> >> the
> >>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
> >> not
> >>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or
> listStoragePool
> >>>>>>>>>> response.
> >>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
> >> for
> >>>>>>>>>> clients
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> >> Offering-by-Disk
> >>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
> >>>>>> able to
> >>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> >> table.
> >>>>>>>> Maybe
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS
> in
> >>>>>> the
> >>>>>>>> GUI
> >>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Please see my comments in-line below.
Thanks,
-John
On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Comments below in red.
>
> Thanks
>
>
> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Overall, I agree with the steps to below for 4.2. However, we may want to
>> throw an exception if we can not fulfill a requested QoS. If the user is
>> expecting that the hypervisor will provide a particular QoS, and that is
>> not possible, it seems like we should inform them rather than silently
>> ignoring their request.
>>
>
> Sure, that sounds reasonable.
>
> We'd have to come up with some way for the allocators to know about the
> requested storage QoS and then query the candidate drivers.
>
> Any thoughts on how we might do that?
>
>
>>
>> To collect my thoughts from previous parts of the thread, I am
>> uncomfortable with the idea that the management server can overcommit a
>> resource. You had mentioned querying the device for available IOPS. While
>> that would be nice, it seems like we could fall back to a max IOPS and
>> overcommit factor manually calculated and entered by the
>> administrator/operator. I think such threshold and allocation rails should
>> be added for both provisioned IOPS and throttled I/O -- it is a basic
>> feature of any cloud orchestration platform.
>>
>
> Are you thinking this ability would make it into 4.2? Just curious what
> release we're talking about here. For the SolidFire SAN, you might have,
> say, four separate storage nodes to start (200,000 IOPS) and then later add
> a new node (now you're at 250,000 IOPS). CS would have to have a way to
> know that the number of supported IOPS has increased.
Yes, I think we need some *basic*/conservative rails in 4.2. For example, we may only support expanding capacity in 4.2, and not handle any re-balance scenarios -- node failure, addition, etc. Extrapolating a bit, the throttled I/O enhancement seems like it needs a similar set of rails defined per host.
>
>
>>
>> For 4.3, I don't like the idea that a QoS would be expressed in a
>> implementation specific manner. I think we need to arrive at a general
>> model that can be exploited by the allocators and planners. We should
>> restrict implementation specific key-value pairs (call them details,
>> extended data, whatever) to information that is unique to the driver and
>> would provide no useful information to the management server's
>> orchestration functions. A resource QoS does not seem to fit those
>> criteria.
>>
>
> I wonder if this would be a good discussion topic for Sunday's CS Collab
> Conf hack day that Joe just sent out an e-mail about?
Yes, it would -- I will put something in the wiki topic. It will also be part of my talk on Monday -- How to Run from Zombie which include some of my opinions on the topic.
>
>
>>
>> Thanks,
>> -John
>>
>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> So, here's my suggestion for 4.2:
>>>
>>> Accept the values as they are currently required (four new fields for
>> Wei's
>>> feature or two new fields for mine).
>>>
>>> The Add Disk Offering dialog needs three new radio buttons:
>>>
>>> 1) No QoS
>>>
>>> 2) Hypervisor QoS
>>>
>>> 3) Storage Qos
>>>
>>> The admin needs to specify storage tags that only map to storage that
>>> supports Storage QoS.
>>>
>>> The admin needs to be aware for Hypervisor QoS that unless all
>> hypervisors
>>> in use support the new fields, they may not be enforced.
>>>
>>> Post 4.3:
>>>
>>> Come up with a way to more generally enter these parameters (probably
>> just
>>> key/value pairs sent to the drivers).
>>>
>>> Have the drivers expose their feature set so the allocators can consider
>>> them more fully and throw an exception if there is not a sufficient
>> match.
>>>
>>>
>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> My thinking is, for 4.2, while not ideal, we will need to put some
>> burden
>>>> on the admin to configure a Disk Offering in a way that makes sense. For
>>>> example, if he wants to use storage QoS with supported Min and Max
>> values,
>>>> he'll have to put in a storage tag that references the SolidFire primary
>>>> storage (plug-in). If he puts in a storage tag that doesn't, then he's
>> not
>>>> going to get the Min and Max feature. We could add help text to the
>> pop-up
>>>> dialog that's displayed when you click in the Min and Max text fields.
>>>>
>>>> Same idea for Wei's feature.
>>>>
>>>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>>>> approach for post 4.2.
>>>>
>>>> Maybe in the future we could have the drivers advertise their
>> capabilities
>>>> and if the allocator feels a request is not being satisfied (say Min was
>>>> entered, but it not's supported by any storage plug-in) it can throw an
>>>> exception.
>>>>
>>>>
>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Comments below in red.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Please see my comment in-line below.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> Hi John,
>>>>>>>
>>>>>>> I've put comments below in red.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> First and foremost, we must ensure that these two features are
>>>>>> mutually
>>>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>>>>> contains both
>>>>>>>> hypervisor and storage IOPS guarantees that leads to
>> non-deterministic
>>>>>>>> operations.
>>>>>>>
>>>>>>>
>>>>>>> Agreed
>>>>>>>
>>>>>>>
>>>>>>>> Restricting QoS expression to be either hypervisor or storage
>> oriented
>>>>>>>> solves the problem in short term. As I understand storage tags, we
>>>>>> have no
>>>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>>>>> necessarily
>>>>>>>> intending that we implement this allocation model in 4.3, but
>> instead,
>>>>>>>> determine if this type model would be one we would want to support
>> in
>>>>>> the
>>>>>>>> future. If so, I would encourage us to ensure that the data model
>> and
>>>>>>>> current implementation would not preclude evolution in that
>>>>>> direction. My
>>>>>>>> view is that this type of allocation model is what user's expect of
>>>>>> "cloud"
>>>>>>>> systems -- selecting the best available resource set to fulfill a
>>>>>> set of
>>>>>>>> system requirements.
>>>>>>>>
>>>>>>>
>>>>>>> I believe we have meet your requirement here in that what we've
>>>>>> implemented
>>>>>>> should not make refinement difficult in the future. If we don't
>> modify
>>>>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
>>>>>> changes
>>>>>>> to enhance the current functioning of the system.
>>>>>>>
>>>>>>
>>>>>> Looking through both patches, I have to say that the aggregated result
>>>>>> seems a bit confusing. There are six new attributes for throttled
>> I/O and
>>>>>> two for provisioned IOPS with no obvious grouping. My concern is not
>>>>>> technical, but rather, about maintainability. At minimum, Javadoc
>> should
>>>>>> be added explaining the two sets of attributes and their mutual
>> exclusion.
>>>>>>
>>>>>
>>>>> I agree: We need JavaDoc to explain them and their mutual exclusion.
>>>>>
>>>>>
>>>>>>
>>>>>> The other part that is interesting is that throttled I/O provides both
>>>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses a
>>>>>> range. In order to select the best available resource to fulfill a
>> QoS, we
>>>>>> would need to have the QoS expression normalized in terms of units
>> (IOPS or
>>>>>> bytes) and their expression (rate vs. range). If we want to achieve a
>>>>>> model like I described, I think we would need to resolve this issue
>> in 4.2
>>>>>> to ensure a viable migration path.
>>>>>
>>>>>
>>>>> I think we're not likely to be able to normalize the input for 4.2.
>> Plus
>>>>> people probably want to input the data in terms they're familiar with
>> for
>>>>> the products in question.
>>>>>
>>>>> Ideally we would fix the way we do storage tagging and let the user
>> send
>>>>> key/value pairs to each vendor that could be selected due to a given
>>>>> storage tag. I'm still not sure that would solve it because what
>> happens if
>>>>> you change the storage tag of a given Primary Storage after having
>> created
>>>>> a Disk Offering?
>>>>>
>>>>> Basically storage tagging is kind of a mess and we should re-think it.
>>>>>
>>>>> Also, we need to have a way for the drivers to expose their supported
>>>>> feature sets so the allocators can make good choices.
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> As I think through the implications of these requirements and
>> reflect
>>>>>> on
>>>>>>>> the reviews, I don't understand why they haven't already impacted
>> the
>>>>>>>> allocators and planners. As it stands, the current provisioned IOPS
>>>>>> has no
>>>>>>>> checks to ensure that the volumes are allocated to devices that have
>>>>>>>> capacity to fulfill the requested QoS. Therefore, as I understand
>> the
>>>>>>>> current patch, we can overcommit storage resources -- potentially
>>>>>> causing
>>>>>>>> none of the QoS obligations from being fulfilled. It seems to me
>>>>>> that a
>>>>>>>> DataStore supporting provisioned IOPS should express the maximum
>> IOPS
>>>>>> which
>>>>>>>> it can support and some type of overcommitment factor. This
>>>>>> information
>>>>>>>> should be used by the storage allocators to determine the device
>> best
>>>>>> able
>>>>>>>> to support the resources needs of a volume. It seems that a similar
>>>>>> set of
>>>>>>>> considerations would need to be added to the Hypervisor layer which
>>>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>>>
>>>>>>>
>>>>>>> Yeah, for this first release, we just followed the path that was
>>>>>> previously
>>>>>>> established for other properties you see on dialogs in CS: Just
>> because
>>>>>>> they're there doesn't mean the vendor your VM is deployed to supports
>>>>>> them.
>>>>>>> It is then up to the admin to make sure he inputs, say, a storage tag
>>>>>> that
>>>>>>> confines the deployment only to storage that supports the selected
>>>>>>> features. This is not ideal, but it's kind of the way CloudStack
>> works
>>>>>>> today.
>>>>>>
>>>>>> I understand the tag functionality, and the need for the administrator
>>>>>> to very carefully construct offerings. My concern is that we over
>>>>>> guarantee a resource's available IOPS. For the purposes of
>> illustration,
>>>>>> let's say we have a SolidFire, and the max IOPS for that device is
>> 100000.
>>>>>> We also know that we can safely oversubscribe by 50%. Therefore, we
>>>>>> need to ensure that we don't allocate more than 150,000 guaranteed
>> IOPS
>>>>>> from that device. Intuitively, it seems like the DataStore
>> configuration
>>>>>> should have a max assignable IOPS and overcommitment factor. As we
>>>>>> allocate volumes and attach VMs, we need to ensure that guarantee
>> more IOPS
>>>>>> exceed the configured maximum for a DataStore. Does that make sense?
>>>>>>
>>>>>
>>>>> I think that's a good idea for a future enhancement. I'm not even sure
>> I
>>>>> can query the SAN to find out how many IOPS safely remain. I'd have to
>> get
>>>>> all of the min values for all of the volumes on the SAN and total them
>> up,
>>>>> I suppose, and subtract it from the total (user facing) supported IOPS
>> of
>>>>> the system.
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Another question occurs to me -- should we allow non-QoS resources
>> to
>>>>>> be
>>>>>>>> assigned to hosts/storage devices that ensure QoS? For provisioned
>>>>>> IOPS, I
>>>>>>>> think a side effect of the current implementation is SolidFire
>> volumes
>>>>>>>> always have a QoS. However, for hypervisor throttled I/O, it seems
>>>>>>>> entirely possible for non-QoS VMs to allocated side-by-side with QoS
>>>>>> VMs.
>>>>>>>> In this scenario, a greedy, unbounded VM could potentially starve
>> out
>>>>>> the
>>>>>>>> other VMs on the host -- preventing the QoSes defined the collocated
>>>>>> VMs
>>>>>>>> from being fulfilled.
>>>>>>>>
>>>>>>>
>>>>>>> You can make SolidFire volumes (inside and outside of CS) and not
>>>>>> specify
>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
>> defaults
>>>>>> we
>>>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will
>>>>>> have
>>>>>>> your Mins met.
>>>>>>>
>>>>>>> It sounds like you're perhaps looking for a storage tags exclusions
>>>>>> list,
>>>>>>> which might be nice to have at some point (i.e. don't deploy my
>> volume
>>>>>> to
>>>>>>> storage with these following tags).
>>>>>>
>>>>>> I don't like the idea of a storage tags exclusion list as it would
>>>>>> complicate component assembly. It would require a storage plugin to
>>>>>> anticipate all of the possible component assemblies and determine the
>>>>>> invalid relationships. I prefer that drivers express their
>> capabilities
>>>>>> which can be matched to a set of requested requirements.
>>>>>>
>>>>>
>>>>> I'm not sure why a storage plug-in would care about inclusion or
>>>>> exclusion lists. It just needs to advertise its functionality in a way
>> the
>>>>> allocator understands.
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> I agree with your assessment of Hypervisor QoS. Since it only limits
>>>>>> IOPS,
>>>>>>> it does not solve the Noisy Neighbor problem. Only a system with
>>>>>> guaranteed
>>>>>>> minimum IOPS does this.
>>>>>>
>>>>>> As I said, for SolidFire, it sounds like this problem does not exist.
>>>>>> However, I am concerned with the more general case as we supported
>> more
>>>>>> devices with provisioned IOPS.
>>>>>>
>>>>>
>>>>> Post 4.2 we need to investigate a way to pass vendor-specific values to
>>>>> drivers. Min and Max and pretty industry standard for provisioned
>> IOPS, but
>>>>> what if you break them out by read and write or do something else? We
>> need
>>>>> a more general mechanism.
>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> In my opinion, we need to ensure that hypervisor throttled I/O and
>>>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>>>>
>>>>>>>
>>>>>>> Agreed
>>>>>>>
>>>>>>>
>>>>>>>> We also need to understand the implications of these QoS guarantees
>> on
>>>>>>>> operation of the system to ensure that the underlying hardware
>>>>>> resources
>>>>>>>> can fulfill them. Given the time frame, we will likely be forced to
>>>>>> make
>>>>>>>> compromises to achieve these goals, and refine the implementation in
>>>>>> 4.3.
>>>>>>>>
>>>>>>>
>>>>>>> I agree, John. I also think you've come up with some great ideas for
>>>>>> 4.3. :)
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
>>>>>> tags).
>>>>>>>> I
>>>>>>>>> believe John was just throwing out an idea that - in addition to
>>>>>> storage
>>>>>>>>> tags - we could look into these allocators (storage QoS being
>>>>>> preferred,
>>>>>>>>> then hypervisor QoS if storage QoS is not available, but hypervisor
>>>>>> QoS
>>>>>>>> is).
>>>>>>>>>
>>>>>>>>> I think John's concern is that you can enter in values for Wei's
>> and
>>>>>> my
>>>>>>>>> feature that are not honored by other vendors (at least yet), so he
>>>>>> was
>>>>>>>>> hoping - in addition to storage tags - for the allocators to prefer
>>>>>> these
>>>>>>>>> vendors when these fields are filled in. As it stands today in
>>>>>>>> CloudStack,
>>>>>>>>> we already have this kind of an issue with other features (fields
>> in
>>>>>>>>> dialogs for features that not all vendors support). Perhaps post
>> 4.2
>>>>>> we
>>>>>>>>> could look into generic name/value pairs (this is how OpenStack
>>>>>> addresses
>>>>>>>>> the issue).
>>>>>>>>>
>>>>>>>>> Honestly, I think we're too late in the game (two weeks until code
>>>>>>>> freeze)
>>>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>>>
>>>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields
>>>>>> and my
>>>>>>>>> fields as is, make sure only one or the other feature has data
>>>>>> entered
>>>>>>>> for
>>>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>>>
>>>>>>>>> Then let's step back and look into a more general-purpose design
>>>>>> that can
>>>>>>>>> be applied throughout CloudStack where we have these situations.
>>>>>>>>>
>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> I just published my review @ https://reviews.apache.org/r/11479/.
>>>>>>>>>>
>>>>>>>>>> I apologize for the delay,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> No problem, John.
>>>>>>>>>>>
>>>>>>>>>>> I still want to re-review it by myself before coming up with a
>> new
>>>>>>>> patch
>>>>>>>>>>> file.
>>>>>>>>>>>
>>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
>> in
>>>>>> and
>>>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
>> jburwell@basho.com
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> I just realized that I forgot to publish my review. I am
>> offline
>>>>>> ATM,
>>>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>>>
>>>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>>>
>>>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
>> ),
>>>>>>>>>>>>>
>>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
>>>>>> wanted
>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
>>>>>> then
>>>>>>>>>> send
>>>>>>>>>>>> in
>>>>>>>>>>>>> another patch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>>>>>> code,
>>>>>>>>>>>> except
>>>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
>> into
>>>>>>>>>> master
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>>>>>> branch,
>>>>>>>> but
>>>>>>>>>> -
>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>>>> SolidFire
>>>>>>>>>> branch
>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>> the object_store branch before object_store goes into
>> master.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>>>>>>> overlapping
>>>>>>>>>>>>>>>>> elements. I understand the dependencies between the
>> patches
>>>>>> to
>>>>>>>> be
>>>>>>>>>> as
>>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Am I correct that the device management aspects of
>> SolidFire
>>>>>> are
>>>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>>>> to the object_store branch or there are circular dependency
>>>>>>>> between
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>>>>>>>> determine
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>>>>>> branch
>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>>>>> merge
>>>>>>>> and
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
>> the
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>>>>>> which
>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although
>>>>>> it
>>>>>>>> does
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>>>>>> use))
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>>>>>> field
>>>>>>>>>> (if
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of
>>>>>> the
>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
>>>>>> for
>>>>>>>> the
>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>>>>>> data
>>>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>>>>>> attach
>>>>>>>> the
>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
>> detach a
>>>>>>>>>> volume,
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>>>>> structure
>>>>>>>> is
>>>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>>>>>>>> unless
>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
>> have
>>>>>> some
>>>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details field.
>> I
>>>>>>>> think
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
>> the
>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
>> not
>>>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>>>>>>>> response.
>>>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
>> for
>>>>>>>>>> clients
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
>> Offering-by-Disk
>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>>>>>> able to
>>>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
>> table.
>>>>>>>> Maybe
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
>>>>>> the
>>>>>>>> GUI
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Comments below in red.
Thanks
On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Overall, I agree with the steps to below for 4.2. However, we may want to
> throw an exception if we can not fulfill a requested QoS. If the user is
> expecting that the hypervisor will provide a particular QoS, and that is
> not possible, it seems like we should inform them rather than silently
> ignoring their request.
>
Sure, that sounds reasonable.
We'd have to come up with some way for the allocators to know about the
requested storage QoS and then query the candidate drivers.
Any thoughts on how we might do that?
>
> To collect my thoughts from previous parts of the thread, I am
> uncomfortable with the idea that the management server can overcommit a
> resource. You had mentioned querying the device for available IOPS. While
> that would be nice, it seems like we could fall back to a max IOPS and
> overcommit factor manually calculated and entered by the
> administrator/operator. I think such threshold and allocation rails should
> be added for both provisioned IOPS and throttled I/O -- it is a basic
> feature of any cloud orchestration platform.
>
Are you thinking this ability would make it into 4.2? Just curious what
release we're talking about here. For the SolidFire SAN, you might have,
say, four separate storage nodes to start (200,000 IOPS) and then later add
a new node (now you're at 250,000 IOPS). CS would have to have a way to
know that the number of supported IOPS has increased.
>
> For 4.3, I don't like the idea that a QoS would be expressed in a
> implementation specific manner. I think we need to arrive at a general
> model that can be exploited by the allocators and planners. We should
> restrict implementation specific key-value pairs (call them details,
> extended data, whatever) to information that is unique to the driver and
> would provide no useful information to the management server's
> orchestration functions. A resource QoS does not seem to fit those
> criteria.
>
I wonder if this would be a good discussion topic for Sunday's CS Collab
Conf hack day that Joe just sent out an e-mail about?
>
> Thanks,
> -John
>
> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > So, here's my suggestion for 4.2:
> >
> > Accept the values as they are currently required (four new fields for
> Wei's
> > feature or two new fields for mine).
> >
> > The Add Disk Offering dialog needs three new radio buttons:
> >
> > 1) No QoS
> >
> > 2) Hypervisor QoS
> >
> > 3) Storage Qos
> >
> > The admin needs to specify storage tags that only map to storage that
> > supports Storage QoS.
> >
> > The admin needs to be aware for Hypervisor QoS that unless all
> hypervisors
> > in use support the new fields, they may not be enforced.
> >
> > Post 4.3:
> >
> > Come up with a way to more generally enter these parameters (probably
> just
> > key/value pairs sent to the drivers).
> >
> > Have the drivers expose their feature set so the allocators can consider
> > them more fully and throw an exception if there is not a sufficient
> match.
> >
> >
> > On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> My thinking is, for 4.2, while not ideal, we will need to put some
> burden
> >> on the admin to configure a Disk Offering in a way that makes sense. For
> >> example, if he wants to use storage QoS with supported Min and Max
> values,
> >> he'll have to put in a storage tag that references the SolidFire primary
> >> storage (plug-in). If he puts in a storage tag that doesn't, then he's
> not
> >> going to get the Min and Max feature. We could add help text to the
> pop-up
> >> dialog that's displayed when you click in the Min and Max text fields.
> >>
> >> Same idea for Wei's feature.
> >>
> >> Not idea, true...perhaps we can brainstorm on a more comprehensive
> >> approach for post 4.2.
> >>
> >> Maybe in the future we could have the drivers advertise their
> capabilities
> >> and if the allocator feels a request is not being satisfied (say Min was
> >> entered, but it not's supported by any storage plug-in) it can throw an
> >> exception.
> >>
> >>
> >> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> Comments below in red.
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com>
> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Please see my comment in-line below.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> Hi John,
> >>>>>
> >>>>> I've put comments below in red.
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> First and foremost, we must ensure that these two features are
> >>>> mutually
> >>>>>> exclusive in 4.2. We don't want to find a configuration that
> >>>> contains both
> >>>>>> hypervisor and storage IOPS guarantees that leads to
> non-deterministic
> >>>>>> operations.
> >>>>>
> >>>>>
> >>>>> Agreed
> >>>>>
> >>>>>
> >>>>>> Restricting QoS expression to be either hypervisor or storage
> oriented
> >>>>>> solves the problem in short term. As I understand storage tags, we
> >>>> have no
> >>>>>> means of expressing this type of mutual exclusion. I wasn't
> >>>> necessarily
> >>>>>> intending that we implement this allocation model in 4.3, but
> instead,
> >>>>>> determine if this type model would be one we would want to support
> in
> >>>> the
> >>>>>> future. If so, I would encourage us to ensure that the data model
> and
> >>>>>> current implementation would not preclude evolution in that
> >>>> direction. My
> >>>>>> view is that this type of allocation model is what user's expect of
> >>>> "cloud"
> >>>>>> systems -- selecting the best available resource set to fulfill a
> >>>> set of
> >>>>>> system requirements.
> >>>>>>
> >>>>>
> >>>>> I believe we have meet your requirement here in that what we've
> >>>> implemented
> >>>>> should not make refinement difficult in the future. If we don't
> modify
> >>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
> >>>> changes
> >>>>> to enhance the current functioning of the system.
> >>>>>
> >>>>
> >>>> Looking through both patches, I have to say that the aggregated result
> >>>> seems a bit confusing. There are six new attributes for throttled
> I/O and
> >>>> two for provisioned IOPS with no obvious grouping. My concern is not
> >>>> technical, but rather, about maintainability. At minimum, Javadoc
> should
> >>>> be added explaining the two sets of attributes and their mutual
> exclusion.
> >>>>
> >>>
> >>> I agree: We need JavaDoc to explain them and their mutual exclusion.
> >>>
> >>>
> >>>>
> >>>> The other part that is interesting is that throttled I/O provides both
> >>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses a
> >>>> range. In order to select the best available resource to fulfill a
> QoS, we
> >>>> would need to have the QoS expression normalized in terms of units
> (IOPS or
> >>>> bytes) and their expression (rate vs. range). If we want to achieve a
> >>>> model like I described, I think we would need to resolve this issue
> in 4.2
> >>>> to ensure a viable migration path.
> >>>
> >>>
> >>> I think we're not likely to be able to normalize the input for 4.2.
> Plus
> >>> people probably want to input the data in terms they're familiar with
> for
> >>> the products in question.
> >>>
> >>> Ideally we would fix the way we do storage tagging and let the user
> send
> >>> key/value pairs to each vendor that could be selected due to a given
> >>> storage tag. I'm still not sure that would solve it because what
> happens if
> >>> you change the storage tag of a given Primary Storage after having
> created
> >>> a Disk Offering?
> >>>
> >>> Basically storage tagging is kind of a mess and we should re-think it.
> >>>
> >>> Also, we need to have a way for the drivers to expose their supported
> >>> feature sets so the allocators can make good choices.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>> As I think through the implications of these requirements and
> reflect
> >>>> on
> >>>>>> the reviews, I don't understand why they haven't already impacted
> the
> >>>>>> allocators and planners. As it stands, the current provisioned IOPS
> >>>> has no
> >>>>>> checks to ensure that the volumes are allocated to devices that have
> >>>>>> capacity to fulfill the requested QoS. Therefore, as I understand
> the
> >>>>>> current patch, we can overcommit storage resources -- potentially
> >>>> causing
> >>>>>> none of the QoS obligations from being fulfilled. It seems to me
> >>>> that a
> >>>>>> DataStore supporting provisioned IOPS should express the maximum
> IOPS
> >>>> which
> >>>>>> it can support and some type of overcommitment factor. This
> >>>> information
> >>>>>> should be used by the storage allocators to determine the device
> best
> >>>> able
> >>>>>> to support the resources needs of a volume. It seems that a similar
> >>>> set of
> >>>>>> considerations would need to be added to the Hypervisor layer which
> >>>>>> allocating a VM to a host to prevent oversubscription.
> >>>>>>
> >>>>>
> >>>>> Yeah, for this first release, we just followed the path that was
> >>>> previously
> >>>>> established for other properties you see on dialogs in CS: Just
> because
> >>>>> they're there doesn't mean the vendor your VM is deployed to supports
> >>>> them.
> >>>>> It is then up to the admin to make sure he inputs, say, a storage tag
> >>>> that
> >>>>> confines the deployment only to storage that supports the selected
> >>>>> features. This is not ideal, but it's kind of the way CloudStack
> works
> >>>>> today.
> >>>>
> >>>> I understand the tag functionality, and the need for the administrator
> >>>> to very carefully construct offerings. My concern is that we over
> >>>> guarantee a resource's available IOPS. For the purposes of
> illustration,
> >>>> let's say we have a SolidFire, and the max IOPS for that device is
> 100000.
> >>>> We also know that we can safely oversubscribe by 50%. Therefore, we
> >>>> need to ensure that we don't allocate more than 150,000 guaranteed
> IOPS
> >>>> from that device. Intuitively, it seems like the DataStore
> configuration
> >>>> should have a max assignable IOPS and overcommitment factor. As we
> >>>> allocate volumes and attach VMs, we need to ensure that guarantee
> more IOPS
> >>>> exceed the configured maximum for a DataStore. Does that make sense?
> >>>>
> >>>
> >>> I think that's a good idea for a future enhancement. I'm not even sure
> I
> >>> can query the SAN to find out how many IOPS safely remain. I'd have to
> get
> >>> all of the min values for all of the volumes on the SAN and total them
> up,
> >>> I suppose, and subtract it from the total (user facing) supported IOPS
> of
> >>> the system.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Another question occurs to me -- should we allow non-QoS resources
> to
> >>>> be
> >>>>>> assigned to hosts/storage devices that ensure QoS? For provisioned
> >>>> IOPS, I
> >>>>>> think a side effect of the current implementation is SolidFire
> volumes
> >>>>>> always have a QoS. However, for hypervisor throttled I/O, it seems
> >>>>>> entirely possible for non-QoS VMs to allocated side-by-side with QoS
> >>>> VMs.
> >>>>>> In this scenario, a greedy, unbounded VM could potentially starve
> out
> >>>> the
> >>>>>> other VMs on the host -- preventing the QoSes defined the collocated
> >>>> VMs
> >>>>>> from being fulfilled.
> >>>>>>
> >>>>>
> >>>>> You can make SolidFire volumes (inside and outside of CS) and not
> >>>> specify
> >>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the
> defaults
> >>>> we
> >>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will
> >>>> have
> >>>>> your Mins met.
> >>>>>
> >>>>> It sounds like you're perhaps looking for a storage tags exclusions
> >>>> list,
> >>>>> which might be nice to have at some point (i.e. don't deploy my
> volume
> >>>> to
> >>>>> storage with these following tags).
> >>>>
> >>>> I don't like the idea of a storage tags exclusion list as it would
> >>>> complicate component assembly. It would require a storage plugin to
> >>>> anticipate all of the possible component assemblies and determine the
> >>>> invalid relationships. I prefer that drivers express their
> capabilities
> >>>> which can be matched to a set of requested requirements.
> >>>>
> >>>
> >>> I'm not sure why a storage plug-in would care about inclusion or
> >>> exclusion lists. It just needs to advertise its functionality in a way
> the
> >>> allocator understands.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>> I agree with your assessment of Hypervisor QoS. Since it only limits
> >>>> IOPS,
> >>>>> it does not solve the Noisy Neighbor problem. Only a system with
> >>>> guaranteed
> >>>>> minimum IOPS does this.
> >>>>
> >>>> As I said, for SolidFire, it sounds like this problem does not exist.
> >>>> However, I am concerned with the more general case as we supported
> more
> >>>> devices with provisioned IOPS.
> >>>>
> >>>
> >>> Post 4.2 we need to investigate a way to pass vendor-specific values to
> >>> drivers. Min and Max and pretty industry standard for provisioned
> IOPS, but
> >>> what if you break them out by read and write or do something else? We
> need
> >>> a more general mechanism.
> >>>
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> In my opinion, we need to ensure that hypervisor throttled I/O and
> >>>>>> storage provisioned IOPS are mutually exclusive per volume.
> >>>>>
> >>>>>
> >>>>> Agreed
> >>>>>
> >>>>>
> >>>>>> We also need to understand the implications of these QoS guarantees
> on
> >>>>>> operation of the system to ensure that the underlying hardware
> >>>> resources
> >>>>>> can fulfill them. Given the time frame, we will likely be forced to
> >>>> make
> >>>>>> compromises to achieve these goals, and refine the implementation in
> >>>> 4.3.
> >>>>>>
> >>>>>
> >>>>> I agree, John. I also think you've come up with some great ideas for
> >>>> 4.3. :)
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
> >>>> tags).
> >>>>>> I
> >>>>>>> believe John was just throwing out an idea that - in addition to
> >>>> storage
> >>>>>>> tags - we could look into these allocators (storage QoS being
> >>>> preferred,
> >>>>>>> then hypervisor QoS if storage QoS is not available, but hypervisor
> >>>> QoS
> >>>>>> is).
> >>>>>>>
> >>>>>>> I think John's concern is that you can enter in values for Wei's
> and
> >>>> my
> >>>>>>> feature that are not honored by other vendors (at least yet), so he
> >>>> was
> >>>>>>> hoping - in addition to storage tags - for the allocators to prefer
> >>>> these
> >>>>>>> vendors when these fields are filled in. As it stands today in
> >>>>>> CloudStack,
> >>>>>>> we already have this kind of an issue with other features (fields
> in
> >>>>>>> dialogs for features that not all vendors support). Perhaps post
> 4.2
> >>>> we
> >>>>>>> could look into generic name/value pairs (this is how OpenStack
> >>>> addresses
> >>>>>>> the issue).
> >>>>>>>
> >>>>>>> Honestly, I think we're too late in the game (two weeks until code
> >>>>>> freeze)
> >>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>
> >>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields
> >>>> and my
> >>>>>>> fields as is, make sure only one or the other feature has data
> >>>> entered
> >>>>>> for
> >>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>
> >>>>>>> Then let's step back and look into a more general-purpose design
> >>>> that can
> >>>>>>> be applied throughout CloudStack where we have these situations.
> >>>>>>>
> >>>>>>> What do you think?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> I just published my review @ https://reviews.apache.org/r/11479/.
> >>>>>>>>
> >>>>>>>> I apologize for the delay,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> No problem, John.
> >>>>>>>>>
> >>>>>>>>> I still want to re-review it by myself before coming up with a
> new
> >>>>>> patch
> >>>>>>>>> file.
> >>>>>>>>>
> >>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked
> in
> >>>> and
> >>>>>>>>> merge those into mine before generating a new patch file?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <
> jburwell@basho.com
> >>>>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> I just realized that I forgot to publish my review. I am
> offline
> >>>> ATM,
> >>>>>>>>>> but I will publish it in the next couple of hours.
> >>>>>>>>>>
> >>>>>>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>>>>>
> >>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :)
> ),
> >>>>>>>>>>>
> >>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
> >>>> wanted
> >>>>>>>>>>> addressed.
> >>>>>>>>>>>
> >>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
> >>>> then
> >>>>>>>> send
> >>>>>>>>>> in
> >>>>>>>>>>> another patch.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>> ustcweizhou@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
> >>>> code,
> >>>>>>>>>> except
> >>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch
> into
> >>>>>>>> master
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
> >>>> branch,
> >>>>>> but
> >>>>>>>> -
> >>>>>>>>>> as
> >>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
> >>>> SolidFire
> >>>>>>>> branch
> >>>>>>>>>>>>> into
> >>>>>>>>>>>>>> the object_store branch before object_store goes into
> master.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
> >>>>>>>> strategy.
> >>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
> >>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
> >>>>>> overlapping
> >>>>>>>>>>>>>>> elements. I understand the dependencies between the
> patches
> >>>> to
> >>>>>> be
> >>>>>>>> as
> >>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Am I correct that the device management aspects of
> SolidFire
> >>>> are
> >>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>> to the object_store branch or there are circular dependency
> >>>>>> between
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
> >>>>>>>> determine
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
> >>>> branch
> >>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
> >>>> merge
> >>>>>> and
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in
> the
> >>>>>>>> storage
> >>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
> >>>> which
> >>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although
> >>>> it
> >>>>>> does
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
> >>>> use))
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
> >>>> field
> >>>>>>>> (if
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of
> >>>> the
> >>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
> >>>> for
> >>>>>> the
> >>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
> >>>> data
> >>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
> >>>> attach
> >>>>>> the
> >>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to
> detach a
> >>>>>>>> volume,
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
> >>>> structure
> >>>>>> is
> >>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
> >>>>>> unless
> >>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we
> have
> >>>> some
> >>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>>>>>> implementation
> >>>>>>>>>>>>>>>>>> attributes. I always wondered about the details field.
> I
> >>>>>> think
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding
> the
> >>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are
> not
> >>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> >>>>>>>> response.
> >>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful
> for
> >>>>>>>> clients
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk
> Offering-by-Disk
> >>>>>>>> Offering
> >>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
> >>>> able to
> >>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details
> table.
> >>>>>> Maybe
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
> >>>> the
> >>>>>> GUI
> >>>>>>>>>>>>> if
> >>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Overall, I agree with the steps to below for 4.2. However, we may want to throw an exception if we can not fulfill a requested QoS. If the user is expecting that the hypervisor will provide a particular QoS, and that is not possible, it seems like we should inform them rather than silently ignoring their request.
To collect my thoughts from previous parts of the thread, I am uncomfortable with the idea that the management server can overcommit a resource. You had mentioned querying the device for available IOPS. While that would be nice, it seems like we could fall back to a max IOPS and overcommit factor manually calculated and entered by the administrator/operator. I think such threshold and allocation rails should be added for both provisioned IOPS and throttled I/O -- it is a basic feature of any cloud orchestration platform.
For 4.3, I don't like the idea that a QoS would be expressed in a implementation specific manner. I think we need to arrive at a general model that can be exploited by the allocators and planners. We should restrict implementation specific key-value pairs (call them details, extended data, whatever) to information that is unique to the driver and would provide no useful information to the management server's orchestration functions. A resource QoS does not seem to fit those criteria.
Thanks,
-John
On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> So, here's my suggestion for 4.2:
>
> Accept the values as they are currently required (four new fields for Wei's
> feature or two new fields for mine).
>
> The Add Disk Offering dialog needs three new radio buttons:
>
> 1) No QoS
>
> 2) Hypervisor QoS
>
> 3) Storage Qos
>
> The admin needs to specify storage tags that only map to storage that
> supports Storage QoS.
>
> The admin needs to be aware for Hypervisor QoS that unless all hypervisors
> in use support the new fields, they may not be enforced.
>
> Post 4.3:
>
> Come up with a way to more generally enter these parameters (probably just
> key/value pairs sent to the drivers).
>
> Have the drivers expose their feature set so the allocators can consider
> them more fully and throw an exception if there is not a sufficient match.
>
>
> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> My thinking is, for 4.2, while not ideal, we will need to put some burden
>> on the admin to configure a Disk Offering in a way that makes sense. For
>> example, if he wants to use storage QoS with supported Min and Max values,
>> he'll have to put in a storage tag that references the SolidFire primary
>> storage (plug-in). If he puts in a storage tag that doesn't, then he's not
>> going to get the Min and Max feature. We could add help text to the pop-up
>> dialog that's displayed when you click in the Min and Max text fields.
>>
>> Same idea for Wei's feature.
>>
>> Not idea, true...perhaps we can brainstorm on a more comprehensive
>> approach for post 4.2.
>>
>> Maybe in the future we could have the drivers advertise their capabilities
>> and if the allocator feels a request is not being satisfied (say Min was
>> entered, but it not's supported by any storage plug-in) it can throw an
>> exception.
>>
>>
>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Comments below in red.
>>>
>>> Thanks
>>>
>>>
>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com> wrote:
>>>
>>>> Mike,
>>>>
>>>> Please see my comment in-line below.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> I've put comments below in red.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> First and foremost, we must ensure that these two features are
>>>> mutually
>>>>>> exclusive in 4.2. We don't want to find a configuration that
>>>> contains both
>>>>>> hypervisor and storage IOPS guarantees that leads to non-deterministic
>>>>>> operations.
>>>>>
>>>>>
>>>>> Agreed
>>>>>
>>>>>
>>>>>> Restricting QoS expression to be either hypervisor or storage oriented
>>>>>> solves the problem in short term. As I understand storage tags, we
>>>> have no
>>>>>> means of expressing this type of mutual exclusion. I wasn't
>>>> necessarily
>>>>>> intending that we implement this allocation model in 4.3, but instead,
>>>>>> determine if this type model would be one we would want to support in
>>>> the
>>>>>> future. If so, I would encourage us to ensure that the data model and
>>>>>> current implementation would not preclude evolution in that
>>>> direction. My
>>>>>> view is that this type of allocation model is what user's expect of
>>>> "cloud"
>>>>>> systems -- selecting the best available resource set to fulfill a
>>>> set of
>>>>>> system requirements.
>>>>>>
>>>>>
>>>>> I believe we have meet your requirement here in that what we've
>>>> implemented
>>>>> should not make refinement difficult in the future. If we don't modify
>>>>> allocators for 4.2, but we do for 4.3, we've made relatively simple
>>>> changes
>>>>> to enhance the current functioning of the system.
>>>>>
>>>>
>>>> Looking through both patches, I have to say that the aggregated result
>>>> seems a bit confusing. There are six new attributes for throttled I/O and
>>>> two for provisioned IOPS with no obvious grouping. My concern is not
>>>> technical, but rather, about maintainability. At minimum, Javadoc should
>>>> be added explaining the two sets of attributes and their mutual exclusion.
>>>>
>>>
>>> I agree: We need JavaDoc to explain them and their mutual exclusion.
>>>
>>>
>>>>
>>>> The other part that is interesting is that throttled I/O provides both
>>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses a
>>>> range. In order to select the best available resource to fulfill a QoS, we
>>>> would need to have the QoS expression normalized in terms of units (IOPS or
>>>> bytes) and their expression (rate vs. range). If we want to achieve a
>>>> model like I described, I think we would need to resolve this issue in 4.2
>>>> to ensure a viable migration path.
>>>
>>>
>>> I think we're not likely to be able to normalize the input for 4.2. Plus
>>> people probably want to input the data in terms they're familiar with for
>>> the products in question.
>>>
>>> Ideally we would fix the way we do storage tagging and let the user send
>>> key/value pairs to each vendor that could be selected due to a given
>>> storage tag. I'm still not sure that would solve it because what happens if
>>> you change the storage tag of a given Primary Storage after having created
>>> a Disk Offering?
>>>
>>> Basically storage tagging is kind of a mess and we should re-think it.
>>>
>>> Also, we need to have a way for the drivers to expose their supported
>>> feature sets so the allocators can make good choices.
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> As I think through the implications of these requirements and reflect
>>>> on
>>>>>> the reviews, I don't understand why they haven't already impacted the
>>>>>> allocators and planners. As it stands, the current provisioned IOPS
>>>> has no
>>>>>> checks to ensure that the volumes are allocated to devices that have
>>>>>> capacity to fulfill the requested QoS. Therefore, as I understand the
>>>>>> current patch, we can overcommit storage resources -- potentially
>>>> causing
>>>>>> none of the QoS obligations from being fulfilled. It seems to me
>>>> that a
>>>>>> DataStore supporting provisioned IOPS should express the maximum IOPS
>>>> which
>>>>>> it can support and some type of overcommitment factor. This
>>>> information
>>>>>> should be used by the storage allocators to determine the device best
>>>> able
>>>>>> to support the resources needs of a volume. It seems that a similar
>>>> set of
>>>>>> considerations would need to be added to the Hypervisor layer which
>>>>>> allocating a VM to a host to prevent oversubscription.
>>>>>>
>>>>>
>>>>> Yeah, for this first release, we just followed the path that was
>>>> previously
>>>>> established for other properties you see on dialogs in CS: Just because
>>>>> they're there doesn't mean the vendor your VM is deployed to supports
>>>> them.
>>>>> It is then up to the admin to make sure he inputs, say, a storage tag
>>>> that
>>>>> confines the deployment only to storage that supports the selected
>>>>> features. This is not ideal, but it's kind of the way CloudStack works
>>>>> today.
>>>>
>>>> I understand the tag functionality, and the need for the administrator
>>>> to very carefully construct offerings. My concern is that we over
>>>> guarantee a resource's available IOPS. For the purposes of illustration,
>>>> let's say we have a SolidFire, and the max IOPS for that device is 100000.
>>>> We also know that we can safely oversubscribe by 50%. Therefore, we
>>>> need to ensure that we don't allocate more than 150,000 guaranteed IOPS
>>>> from that device. Intuitively, it seems like the DataStore configuration
>>>> should have a max assignable IOPS and overcommitment factor. As we
>>>> allocate volumes and attach VMs, we need to ensure that guarantee more IOPS
>>>> exceed the configured maximum for a DataStore. Does that make sense?
>>>>
>>>
>>> I think that's a good idea for a future enhancement. I'm not even sure I
>>> can query the SAN to find out how many IOPS safely remain. I'd have to get
>>> all of the min values for all of the volumes on the SAN and total them up,
>>> I suppose, and subtract it from the total (user facing) supported IOPS of
>>> the system.
>>>
>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Another question occurs to me -- should we allow non-QoS resources to
>>>> be
>>>>>> assigned to hosts/storage devices that ensure QoS? For provisioned
>>>> IOPS, I
>>>>>> think a side effect of the current implementation is SolidFire volumes
>>>>>> always have a QoS. However, for hypervisor throttled I/O, it seems
>>>>>> entirely possible for non-QoS VMs to allocated side-by-side with QoS
>>>> VMs.
>>>>>> In this scenario, a greedy, unbounded VM could potentially starve out
>>>> the
>>>>>> other VMs on the host -- preventing the QoSes defined the collocated
>>>> VMs
>>>>>> from being fulfilled.
>>>>>>
>>>>>
>>>>> You can make SolidFire volumes (inside and outside of CS) and not
>>>> specify
>>>>> IOPS. You'll still get guaranteed IOPS, but it will be at the defaults
>>>> we
>>>>> choose. Unless you over-provision IOPS on a SolidFire SAN, you will
>>>> have
>>>>> your Mins met.
>>>>>
>>>>> It sounds like you're perhaps looking for a storage tags exclusions
>>>> list,
>>>>> which might be nice to have at some point (i.e. don't deploy my volume
>>>> to
>>>>> storage with these following tags).
>>>>
>>>> I don't like the idea of a storage tags exclusion list as it would
>>>> complicate component assembly. It would require a storage plugin to
>>>> anticipate all of the possible component assemblies and determine the
>>>> invalid relationships. I prefer that drivers express their capabilities
>>>> which can be matched to a set of requested requirements.
>>>>
>>>
>>> I'm not sure why a storage plug-in would care about inclusion or
>>> exclusion lists. It just needs to advertise its functionality in a way the
>>> allocator understands.
>>>
>>>
>>>>
>>>>>
>>>>> I agree with your assessment of Hypervisor QoS. Since it only limits
>>>> IOPS,
>>>>> it does not solve the Noisy Neighbor problem. Only a system with
>>>> guaranteed
>>>>> minimum IOPS does this.
>>>>
>>>> As I said, for SolidFire, it sounds like this problem does not exist.
>>>> However, I am concerned with the more general case as we supported more
>>>> devices with provisioned IOPS.
>>>>
>>>
>>> Post 4.2 we need to investigate a way to pass vendor-specific values to
>>> drivers. Min and Max and pretty industry standard for provisioned IOPS, but
>>> what if you break them out by read and write or do something else? We need
>>> a more general mechanism.
>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> In my opinion, we need to ensure that hypervisor throttled I/O and
>>>>>> storage provisioned IOPS are mutually exclusive per volume.
>>>>>
>>>>>
>>>>> Agreed
>>>>>
>>>>>
>>>>>> We also need to understand the implications of these QoS guarantees on
>>>>>> operation of the system to ensure that the underlying hardware
>>>> resources
>>>>>> can fulfill them. Given the time frame, we will likely be forced to
>>>> make
>>>>>> compromises to achieve these goals, and refine the implementation in
>>>> 4.3.
>>>>>>
>>>>>
>>>>> I agree, John. I also think you've come up with some great ideas for
>>>> 4.3. :)
>>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Yeah, Alex, I think that's the way we were planning (with storage
>>>> tags).
>>>>>> I
>>>>>>> believe John was just throwing out an idea that - in addition to
>>>> storage
>>>>>>> tags - we could look into these allocators (storage QoS being
>>>> preferred,
>>>>>>> then hypervisor QoS if storage QoS is not available, but hypervisor
>>>> QoS
>>>>>> is).
>>>>>>>
>>>>>>> I think John's concern is that you can enter in values for Wei's and
>>>> my
>>>>>>> feature that are not honored by other vendors (at least yet), so he
>>>> was
>>>>>>> hoping - in addition to storage tags - for the allocators to prefer
>>>> these
>>>>>>> vendors when these fields are filled in. As it stands today in
>>>>>> CloudStack,
>>>>>>> we already have this kind of an issue with other features (fields in
>>>>>>> dialogs for features that not all vendors support). Perhaps post 4.2
>>>> we
>>>>>>> could look into generic name/value pairs (this is how OpenStack
>>>> addresses
>>>>>>> the issue).
>>>>>>>
>>>>>>> Honestly, I think we're too late in the game (two weeks until code
>>>>>> freeze)
>>>>>>> to go too deeply down that path in 4.2.
>>>>>>>
>>>>>>> It's probably better if we - at least for 4.2 - keep Wei's fields
>>>> and my
>>>>>>> fields as is, make sure only one or the other feature has data
>>>> entered
>>>>>> for
>>>>>>> it (or neither), and call it good for 4.2.
>>>>>>>
>>>>>>> Then let's step back and look into a more general-purpose design
>>>> that can
>>>>>>> be applied throughout CloudStack where we have these situations.
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> I just published my review @ https://reviews.apache.org/r/11479/.
>>>>>>>>
>>>>>>>> I apologize for the delay,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> No problem, John.
>>>>>>>>>
>>>>>>>>> I still want to re-review it by myself before coming up with a new
>>>>>> patch
>>>>>>>>> file.
>>>>>>>>>
>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked in
>>>> and
>>>>>>>>> merge those into mine before generating a new patch file?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jburwell@basho.com
>>>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> I just realized that I forgot to publish my review. I am offline
>>>> ATM,
>>>>>>>>>> but I will publish it in the next couple of hours.
>>>>>>>>>>
>>>>>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>>>>>
>>>>>>>>>> Sorry for the oversight,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>>>>>>>
>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas we
>>>> wanted
>>>>>>>>>>> addressed.
>>>>>>>>>>>
>>>>>>>>>>> I plan to review the code again tomorrow morning or afternoon,
>>>> then
>>>>>>>> send
>>>>>>>>>> in
>>>>>>>>>>> another patch.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for all the work on this everyone!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Sure, that sounds good.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>>> ustcweizhou@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java
>>>> code,
>>>>>>>>>> except
>>>>>>>>>>>>> the cloudstack UI.
>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>>>>>>>> master
>>>>>>>>>>>>> this
>>>>>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store
>>>> branch,
>>>>>> but
>>>>>>>> -
>>>>>>>>>> as
>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>>> SolidFire
>>>>>>>> branch
>>>>>>>>>>>>> into
>>>>>>>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>>>>>> strategy.
>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The
>>>>>> disk_io_throttling,
>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of
>>>>>> overlapping
>>>>>>>>>>>>>>> elements. I understand the dependencies between the patches
>>>> to
>>>>>> be
>>>>>>>> as
>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Am I correct that the device management aspects of SolidFire
>>>> are
>>>>>>>>>>>>> additive
>>>>>>>>>>>>>>> to the object_store branch or there are circular dependency
>>>>>> between
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>>>>>> determine
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>>>> branch
>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>>> merge
>>>>>> and
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>>>>>>>> storage
>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>>>> which
>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use (although
>>>> it
>>>>>> does
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>>>> use))
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>>>> field
>>>>>>>> (if
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>>>>>> question, we
>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place of
>>>> the
>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
>>>> for
>>>>>> the
>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>>>> data
>>>>>>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to
>>>> attach
>>>>>> the
>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>>>>>>>> volume,
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>>> structure
>>>>>> is
>>>>>>>>>>>>>>> removed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>>>>>> unless
>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
>>>> some
>>>>>>>>>>>>> idea,
>>>>>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>>>>>> implementation
>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>>>>>> think
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>> format
>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>>>>>>>> returned on
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>>>>>> response.
>>>>>>>>>>>>>> Why
>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>>>>>>>> clients
>>>>>>>>>>>>> to
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>>>>>>>> Offering
>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>>>> able to
>>>>>>>>>>>>>> associate
>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>>>>>> Maybe
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
>>>> the
>>>>>> GUI
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
So, here's my suggestion for 4.2:
Accept the values as they are currently required (four new fields for Wei's
feature or two new fields for mine).
The Add Disk Offering dialog needs three new radio buttons:
1) No QoS
2) Hypervisor QoS
3) Storage Qos
The admin needs to specify storage tags that only map to storage that
supports Storage QoS.
The admin needs to be aware for Hypervisor QoS that unless all hypervisors
in use support the new fields, they may not be enforced.
Post 4.3:
Come up with a way to more generally enter these parameters (probably just
key/value pairs sent to the drivers).
Have the drivers expose their feature set so the allocators can consider
them more fully and throw an exception if there is not a sufficient match.
On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> My thinking is, for 4.2, while not ideal, we will need to put some burden
> on the admin to configure a Disk Offering in a way that makes sense. For
> example, if he wants to use storage QoS with supported Min and Max values,
> he'll have to put in a storage tag that references the SolidFire primary
> storage (plug-in). If he puts in a storage tag that doesn't, then he's not
> going to get the Min and Max feature. We could add help text to the pop-up
> dialog that's displayed when you click in the Min and Max text fields.
>
> Same idea for Wei's feature.
>
> Not idea, true...perhaps we can brainstorm on a more comprehensive
> approach for post 4.2.
>
> Maybe in the future we could have the drivers advertise their capabilities
> and if the allocator feels a request is not being satisfied (say Min was
> entered, but it not's supported by any storage plug-in) it can throw an
> exception.
>
>
> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Comments below in red.
>>
>> Thanks
>>
>>
>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> Please see my comment in-line below.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>> > Hi John,
>>> >
>>> > I've put comments below in red.
>>> >
>>> > Thanks!
>>> >
>>> >
>>> > On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>> >
>>> >> Mike,
>>> >>
>>> >> First and foremost, we must ensure that these two features are
>>> mutually
>>> >> exclusive in 4.2. We don't want to find a configuration that
>>> contains both
>>> >> hypervisor and storage IOPS guarantees that leads to non-deterministic
>>> >> operations.
>>> >
>>> >
>>> > Agreed
>>> >
>>> >
>>> >> Restricting QoS expression to be either hypervisor or storage oriented
>>> >> solves the problem in short term. As I understand storage tags, we
>>> have no
>>> >> means of expressing this type of mutual exclusion. I wasn't
>>> necessarily
>>> >> intending that we implement this allocation model in 4.3, but instead,
>>> >> determine if this type model would be one we would want to support in
>>> the
>>> >> future. If so, I would encourage us to ensure that the data model and
>>> >> current implementation would not preclude evolution in that
>>> direction. My
>>> >> view is that this type of allocation model is what user's expect of
>>> "cloud"
>>> >> systems -- selecting the best available resource set to fulfill a
>>> set of
>>> >> system requirements.
>>> >>
>>> >
>>> > I believe we have meet your requirement here in that what we've
>>> implemented
>>> > should not make refinement difficult in the future. If we don't modify
>>> > allocators for 4.2, but we do for 4.3, we've made relatively simple
>>> changes
>>> > to enhance the current functioning of the system.
>>> >
>>>
>>> Looking through both patches, I have to say that the aggregated result
>>> seems a bit confusing. There are six new attributes for throttled I/O and
>>> two for provisioned IOPS with no obvious grouping. My concern is not
>>> technical, but rather, about maintainability. At minimum, Javadoc should
>>> be added explaining the two sets of attributes and their mutual exclusion.
>>>
>>
>> I agree: We need JavaDoc to explain them and their mutual exclusion.
>>
>>
>>>
>>> The other part that is interesting is that throttled I/O provides both
>>> an IOPS and byte measured QoS as a rate where provisioned IOPS uses a
>>> range. In order to select the best available resource to fulfill a QoS, we
>>> would need to have the QoS expression normalized in terms of units (IOPS or
>>> bytes) and their expression (rate vs. range). If we want to achieve a
>>> model like I described, I think we would need to resolve this issue in 4.2
>>> to ensure a viable migration path.
>>
>>
>> I think we're not likely to be able to normalize the input for 4.2. Plus
>> people probably want to input the data in terms they're familiar with for
>> the products in question.
>>
>> Ideally we would fix the way we do storage tagging and let the user send
>> key/value pairs to each vendor that could be selected due to a given
>> storage tag. I'm still not sure that would solve it because what happens if
>> you change the storage tag of a given Primary Storage after having created
>> a Disk Offering?
>>
>> Basically storage tagging is kind of a mess and we should re-think it.
>>
>> Also, we need to have a way for the drivers to expose their supported
>> feature sets so the allocators can make good choices.
>>
>>
>>>
>>> >
>>> >>
>>> >> As I think through the implications of these requirements and reflect
>>> on
>>> >> the reviews, I don't understand why they haven't already impacted the
>>> >> allocators and planners. As it stands, the current provisioned IOPS
>>> has no
>>> >> checks to ensure that the volumes are allocated to devices that have
>>> >> capacity to fulfill the requested QoS. Therefore, as I understand the
>>> >> current patch, we can overcommit storage resources -- potentially
>>> causing
>>> >> none of the QoS obligations from being fulfilled. It seems to me
>>> that a
>>> >> DataStore supporting provisioned IOPS should express the maximum IOPS
>>> which
>>> >> it can support and some type of overcommitment factor. This
>>> information
>>> >> should be used by the storage allocators to determine the device best
>>> able
>>> >> to support the resources needs of a volume. It seems that a similar
>>> set of
>>> >> considerations would need to be added to the Hypervisor layer which
>>> >> allocating a VM to a host to prevent oversubscription.
>>> >>
>>> >
>>> > Yeah, for this first release, we just followed the path that was
>>> previously
>>> > established for other properties you see on dialogs in CS: Just because
>>> > they're there doesn't mean the vendor your VM is deployed to supports
>>> them.
>>> > It is then up to the admin to make sure he inputs, say, a storage tag
>>> that
>>> > confines the deployment only to storage that supports the selected
>>> > features. This is not ideal, but it's kind of the way CloudStack works
>>> > today.
>>>
>>> I understand the tag functionality, and the need for the administrator
>>> to very carefully construct offerings. My concern is that we over
>>> guarantee a resource's available IOPS. For the purposes of illustration,
>>> let's say we have a SolidFire, and the max IOPS for that device is 100000.
>>> We also know that we can safely oversubscribe by 50%. Therefore, we
>>> need to ensure that we don't allocate more than 150,000 guaranteed IOPS
>>> from that device. Intuitively, it seems like the DataStore configuration
>>> should have a max assignable IOPS and overcommitment factor. As we
>>> allocate volumes and attach VMs, we need to ensure that guarantee more IOPS
>>> exceed the configured maximum for a DataStore. Does that make sense?
>>>
>>
>> I think that's a good idea for a future enhancement. I'm not even sure I
>> can query the SAN to find out how many IOPS safely remain. I'd have to get
>> all of the min values for all of the volumes on the SAN and total them up,
>> I suppose, and subtract it from the total (user facing) supported IOPS of
>> the system.
>>
>>
>>>
>>> >
>>> >
>>> >>
>>> >> Another question occurs to me -- should we allow non-QoS resources to
>>> be
>>> >> assigned to hosts/storage devices that ensure QoS? For provisioned
>>> IOPS, I
>>> >> think a side effect of the current implementation is SolidFire volumes
>>> >> always have a QoS. However, for hypervisor throttled I/O, it seems
>>> >> entirely possible for non-QoS VMs to allocated side-by-side with QoS
>>> VMs.
>>> >> In this scenario, a greedy, unbounded VM could potentially starve out
>>> the
>>> >> other VMs on the host -- preventing the QoSes defined the collocated
>>> VMs
>>> >> from being fulfilled.
>>> >>
>>> >
>>> > You can make SolidFire volumes (inside and outside of CS) and not
>>> specify
>>> > IOPS. You'll still get guaranteed IOPS, but it will be at the defaults
>>> we
>>> > choose. Unless you over-provision IOPS on a SolidFire SAN, you will
>>> have
>>> > your Mins met.
>>> >
>>> > It sounds like you're perhaps looking for a storage tags exclusions
>>> list,
>>> > which might be nice to have at some point (i.e. don't deploy my volume
>>> to
>>> > storage with these following tags).
>>>
>>> I don't like the idea of a storage tags exclusion list as it would
>>> complicate component assembly. It would require a storage plugin to
>>> anticipate all of the possible component assemblies and determine the
>>> invalid relationships. I prefer that drivers express their capabilities
>>> which can be matched to a set of requested requirements.
>>>
>>
>> I'm not sure why a storage plug-in would care about inclusion or
>> exclusion lists. It just needs to advertise its functionality in a way the
>> allocator understands.
>>
>>
>>>
>>> >
>>> > I agree with your assessment of Hypervisor QoS. Since it only limits
>>> IOPS,
>>> > it does not solve the Noisy Neighbor problem. Only a system with
>>> guaranteed
>>> > minimum IOPS does this.
>>>
>>> As I said, for SolidFire, it sounds like this problem does not exist.
>>> However, I am concerned with the more general case as we supported more
>>> devices with provisioned IOPS.
>>>
>>
>> Post 4.2 we need to investigate a way to pass vendor-specific values to
>> drivers. Min and Max and pretty industry standard for provisioned IOPS, but
>> what if you break them out by read and write or do something else? We need
>> a more general mechanism.
>>
>>>
>>> >
>>> >
>>> >>
>>> >> In my opinion, we need to ensure that hypervisor throttled I/O and
>>> >> storage provisioned IOPS are mutually exclusive per volume.
>>> >
>>> >
>>> > Agreed
>>> >
>>> >
>>> >> We also need to understand the implications of these QoS guarantees on
>>> >> operation of the system to ensure that the underlying hardware
>>> resources
>>> >> can fulfill them. Given the time frame, we will likely be forced to
>>> make
>>> >> compromises to achieve these goals, and refine the implementation in
>>> 4.3.
>>> >>
>>> >
>>> > I agree, John. I also think you've come up with some great ideas for
>>> 4.3. :)
>>> >
>>> >
>>> >>
>>> >> Thanks,
>>> >> -John
>>> >>
>>> >> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>> >> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> Yeah, Alex, I think that's the way we were planning (with storage
>>> tags).
>>> >> I
>>> >>> believe John was just throwing out an idea that - in addition to
>>> storage
>>> >>> tags - we could look into these allocators (storage QoS being
>>> preferred,
>>> >>> then hypervisor QoS if storage QoS is not available, but hypervisor
>>> QoS
>>> >> is).
>>> >>>
>>> >>> I think John's concern is that you can enter in values for Wei's and
>>> my
>>> >>> feature that are not honored by other vendors (at least yet), so he
>>> was
>>> >>> hoping - in addition to storage tags - for the allocators to prefer
>>> these
>>> >>> vendors when these fields are filled in. As it stands today in
>>> >> CloudStack,
>>> >>> we already have this kind of an issue with other features (fields in
>>> >>> dialogs for features that not all vendors support). Perhaps post 4.2
>>> we
>>> >>> could look into generic name/value pairs (this is how OpenStack
>>> addresses
>>> >>> the issue).
>>> >>>
>>> >>> Honestly, I think we're too late in the game (two weeks until code
>>> >> freeze)
>>> >>> to go too deeply down that path in 4.2.
>>> >>>
>>> >>> It's probably better if we - at least for 4.2 - keep Wei's fields
>>> and my
>>> >>> fields as is, make sure only one or the other feature has data
>>> entered
>>> >> for
>>> >>> it (or neither), and call it good for 4.2.
>>> >>>
>>> >>> Then let's step back and look into a more general-purpose design
>>> that can
>>> >>> be applied throughout CloudStack where we have these situations.
>>> >>>
>>> >>> What do you think?
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
>>> >> wrote:
>>> >>>
>>> >>>> Mike,
>>> >>>>
>>> >>>> I just published my review @ https://reviews.apache.org/r/11479/.
>>> >>>>
>>> >>>> I apologize for the delay,
>>> >>>> -John
>>> >>>>
>>> >>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>>> >> mike.tutkowski@solidfire.com>
>>> >>>> wrote:
>>> >>>>
>>> >>>>> No problem, John.
>>> >>>>>
>>> >>>>> I still want to re-review it by myself before coming up with a new
>>> >> patch
>>> >>>>> file.
>>> >>>>>
>>> >>>>> Also, maybe I should first wait for Wei's changes to be checked in
>>> and
>>> >>>>> merge those into mine before generating a new patch file?
>>> >>>>>
>>> >>>>>
>>> >>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jburwell@basho.com
>>> >
>>> >>>> wrote:
>>> >>>>>
>>> >>>>>> Mike,
>>> >>>>>>
>>> >>>>>> I just realized that I forgot to publish my review. I am offline
>>> ATM,
>>> >>>>>> but I will publish it in the next couple of hours.
>>> >>>>>>
>>> >>>>>> Do you plan to update your the patch in Review Board?
>>> >>>>>>
>>> >>>>>> Sorry for the oversight,
>>> >>>>>> -John
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>> >>>>>> <mi...@solidfire.com> wrote:
>>> >>>>>>
>>> >>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>> >>>>>>>
>>> >>>>>>> Just an FYI that I believe I have implemented all the areas we
>>> wanted
>>> >>>>>>> addressed.
>>> >>>>>>>
>>> >>>>>>> I plan to review the code again tomorrow morning or afternoon,
>>> then
>>> >>>> send
>>> >>>>>> in
>>> >>>>>>> another patch.
>>> >>>>>>>
>>> >>>>>>> Thanks for all the work on this everyone!
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>
>>> >>>>>>>> Sure, that sounds good.
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>>> ustcweizhou@gmail.com>
>>> >>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> Hi Mike,
>>> >>>>>>>>>
>>> >>>>>>>>> It looks the two feature do not have many conflicts in Java
>>> code,
>>> >>>>>> except
>>> >>>>>>>>> the cloudstack UI.
>>> >>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>>> >>>> master
>>> >>>>>>>>> this
>>> >>>>>>>>> week, so that you can develop based on it.
>>> >>>>>>>>>
>>> >>>>>>>>> -Wei
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>> >>>>>>>>>
>>> >>>>>>>>>> Hey John,
>>> >>>>>>>>>>
>>> >>>>>>>>>> The SolidFire patch does not depend on the object_store
>>> branch,
>>> >> but
>>> >>>> -
>>> >>>>>> as
>>> >>>>>>>>>> Edison mentioned - it might be easier if we merge the
>>> SolidFire
>>> >>>> branch
>>> >>>>>>>>> into
>>> >>>>>>>>>> the object_store branch before object_store goes into master.
>>> >>>>>>>>>>
>>> >>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>> >>>> strategy.
>>> >>>>>>>>>> Perhaps Wei can chime in on that.
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>>> >> jburwell@basho.com>
>>> >>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> Mike,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> We have a delicate merge dance to perform. The
>>> >> disk_io_throttling,
>>> >>>>>>>>>>> solidfire, and object_store appear to have a number of
>>> >> overlapping
>>> >>>>>>>>>>> elements. I understand the dependencies between the patches
>>> to
>>> >> be
>>> >>>> as
>>> >>>>>>>>>>> follows:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Am I correct that the device management aspects of SolidFire
>>> are
>>> >>>>>>>>> additive
>>> >>>>>>>>>>> to the object_store branch or there are circular dependency
>>> >> between
>>> >>>>>>>>> the
>>> >>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>> >>>> determine
>>> >>>>>>>>> the
>>> >>>>>>>>>>> best approach to land the changes in master.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks,
>>> >>>>>>>>>>> -John
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>>> branch
>>> >>>>>>>>> before
>>> >>>>>>>>>>>> going into master, I am good with that.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>>> merge
>>> >> and
>>> >>>>>>>>> we
>>> >>>>>>>>>> can
>>> >>>>>>>>>>>> deal with Burst IOPS then, as well.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>>> >>>> storage
>>> >>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>>> which
>>> >>>>>>>>>>> currently
>>> >>>>>>>>>>>>> sends several commands to the hypervisor in use (although
>>> it
>>> >> does
>>> >>>>>>>>> not
>>> >>>>>>>>>>> know
>>> >>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>>> use))
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>>> field
>>> >>>> (if
>>> >>>>>>>>>> not
>>> >>>>>>>>>>>>> present, we default to false). This info is stored in the
>>> >>>>>>>>>>>>> storage_pool_details table.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>> >>>>>>>>> question, we
>>> >>>>>>>>>>>>> pass the managed property along (this takes the place of
>>> the
>>> >>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks
>>> for
>>> >> the
>>> >>>>>>>>>> managed
>>> >>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>>> data
>>> >>>>>>>>>>> structure is
>>> >>>>>>>>>>>>> created and the rest of the attach command executes to
>>> attach
>>> >> the
>>> >>>>>>>>>>> volume.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>>> >>>> volume,
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>>> structure
>>> >> is
>>> >>>>>>>>>>> removed.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>>> >> unless
>>> >>>>>>>>> it is
>>> >>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
>>> some
>>> >>>>>>>>> idea,
>>> >>>>>>>>>>>>> that'd be cool.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>> >>>> implementation
>>> >>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>>> >> think
>>> >>>>>>>>> we
>>> >>>>>>>>>>> should
>>> >>>>>>>>>>>>>> beef up the description in the documentation regarding the
>>> >>>>>>>>> expected
>>> >>>>>>>>>>> format
>>> >>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>> >>>>>>>>> returned on
>>> >>>>>>>>>>> the
>>> >>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>> >>>> response.
>>> >>>>>>>>>> Why
>>> >>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>>> >>>> clients
>>> >>>>>>>>> to
>>> >>>>>>>>>> be
>>> >>>>>>>>>>>>>> able to inspect the contents of the details field."
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>>> >>>> Offering
>>> >>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be
>>> able to
>>> >>>>>>>>>> associate
>>> >>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>>> >> Maybe
>>> >>>>>>>>> it
>>> >>>>>>>>>>> could
>>> >>>>>>>>>>>>>> go there?
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
>>> the
>>> >> GUI
>>> >>>>>>>>> if
>>> >>>>>>>>>>> it's
>>> >>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> --
>>> >>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> --
>>> >>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>> *â„¢*
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>> *Mike Tutkowski*
>>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>> o: 303.746.7302
>>> >>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>> *â„¢*
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> *Mike Tutkowski*
>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>> o: 303.746.7302
>>> >>>>>>> Advancing the way the world uses the
>>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>> *â„¢*
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> *Mike Tutkowski*
>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>> e: mike.tutkowski@solidfire.com
>>> >>>>> o: 303.746.7302
>>> >>>>> Advancing the way the world uses the
>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>> *â„¢*
>>> >>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Mike Tutkowski*
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>> e: mike.tutkowski@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the
>>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>> *â„¢*
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
My thinking is, for 4.2, while not ideal, we will need to put some burden
on the admin to configure a Disk Offering in a way that makes sense. For
example, if he wants to use storage QoS with supported Min and Max values,
he'll have to put in a storage tag that references the SolidFire primary
storage (plug-in). If he puts in a storage tag that doesn't, then he's not
going to get the Min and Max feature. We could add help text to the pop-up
dialog that's displayed when you click in the Min and Max text fields.
Same idea for Wei's feature.
Not idea, true...perhaps we can brainstorm on a more comprehensive approach
for post 4.2.
Maybe in the future we could have the drivers advertise their capabilities
and if the allocator feels a request is not being satisfied (say Min was
entered, but it not's supported by any storage plug-in) it can throw an
exception.
On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Comments below in red.
>
> Thanks
>
>
> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Please see my comment in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > Hi John,
>> >
>> > I've put comments below in red.
>> >
>> > Thanks!
>> >
>> >
>> > On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Mike,
>> >>
>> >> First and foremost, we must ensure that these two features are mutually
>> >> exclusive in 4.2. We don't want to find a configuration that contains
>> both
>> >> hypervisor and storage IOPS guarantees that leads to non-deterministic
>> >> operations.
>> >
>> >
>> > Agreed
>> >
>> >
>> >> Restricting QoS expression to be either hypervisor or storage oriented
>> >> solves the problem in short term. As I understand storage tags, we
>> have no
>> >> means of expressing this type of mutual exclusion. I wasn't
>> necessarily
>> >> intending that we implement this allocation model in 4.3, but instead,
>> >> determine if this type model would be one we would want to support in
>> the
>> >> future. If so, I would encourage us to ensure that the data model and
>> >> current implementation would not preclude evolution in that direction.
>> My
>> >> view is that this type of allocation model is what user's expect of
>> "cloud"
>> >> systems -- selecting the best available resource set to fulfill a set
>> of
>> >> system requirements.
>> >>
>> >
>> > I believe we have meet your requirement here in that what we've
>> implemented
>> > should not make refinement difficult in the future. If we don't modify
>> > allocators for 4.2, but we do for 4.3, we've made relatively simple
>> changes
>> > to enhance the current functioning of the system.
>> >
>>
>> Looking through both patches, I have to say that the aggregated result
>> seems a bit confusing. There are six new attributes for throttled I/O and
>> two for provisioned IOPS with no obvious grouping. My concern is not
>> technical, but rather, about maintainability. At minimum, Javadoc should
>> be added explaining the two sets of attributes and their mutual exclusion.
>>
>
> I agree: We need JavaDoc to explain them and their mutual exclusion.
>
>
>>
>> The other part that is interesting is that throttled I/O provides both an
>> IOPS and byte measured QoS as a rate where provisioned IOPS uses a range.
>> In order to select the best available resource to fulfill a QoS, we would
>> need to have the QoS expression normalized in terms of units (IOPS or
>> bytes) and their expression (rate vs. range). If we want to achieve a
>> model like I described, I think we would need to resolve this issue in 4.2
>> to ensure a viable migration path.
>
>
> I think we're not likely to be able to normalize the input for 4.2. Plus
> people probably want to input the data in terms they're familiar with for
> the products in question.
>
> Ideally we would fix the way we do storage tagging and let the user send
> key/value pairs to each vendor that could be selected due to a given
> storage tag. I'm still not sure that would solve it because what happens if
> you change the storage tag of a given Primary Storage after having created
> a Disk Offering?
>
> Basically storage tagging is kind of a mess and we should re-think it.
>
> Also, we need to have a way for the drivers to expose their supported
> feature sets so the allocators can make good choices.
>
>
>>
>> >
>> >>
>> >> As I think through the implications of these requirements and reflect
>> on
>> >> the reviews, I don't understand why they haven't already impacted the
>> >> allocators and planners. As it stands, the current provisioned IOPS
>> has no
>> >> checks to ensure that the volumes are allocated to devices that have
>> >> capacity to fulfill the requested QoS. Therefore, as I understand the
>> >> current patch, we can overcommit storage resources -- potentially
>> causing
>> >> none of the QoS obligations from being fulfilled. It seems to me that
>> a
>> >> DataStore supporting provisioned IOPS should express the maximum IOPS
>> which
>> >> it can support and some type of overcommitment factor. This
>> information
>> >> should be used by the storage allocators to determine the device best
>> able
>> >> to support the resources needs of a volume. It seems that a similar
>> set of
>> >> considerations would need to be added to the Hypervisor layer which
>> >> allocating a VM to a host to prevent oversubscription.
>> >>
>> >
>> > Yeah, for this first release, we just followed the path that was
>> previously
>> > established for other properties you see on dialogs in CS: Just because
>> > they're there doesn't mean the vendor your VM is deployed to supports
>> them.
>> > It is then up to the admin to make sure he inputs, say, a storage tag
>> that
>> > confines the deployment only to storage that supports the selected
>> > features. This is not ideal, but it's kind of the way CloudStack works
>> > today.
>>
>> I understand the tag functionality, and the need for the administrator to
>> very carefully construct offerings. My concern is that we over guarantee a
>> resource's available IOPS. For the purposes of illustration, let's say we
>> have a SolidFire, and the max IOPS for that device is 100000. We also
>> know that we can safely oversubscribe by 50%. Therefore, we need to ensure
>> that we don't allocate more than 150,000 guaranteed IOPS from that device.
>> Intuitively, it seems like the DataStore configuration should have a max
>> assignable IOPS and overcommitment factor. As we allocate volumes and
>> attach VMs, we need to ensure that guarantee more IOPS exceed the
>> configured maximum for a DataStore. Does that make sense?
>>
>
> I think that's a good idea for a future enhancement. I'm not even sure I
> can query the SAN to find out how many IOPS safely remain. I'd have to get
> all of the min values for all of the volumes on the SAN and total them up,
> I suppose, and subtract it from the total (user facing) supported IOPS of
> the system.
>
>
>>
>> >
>> >
>> >>
>> >> Another question occurs to me -- should we allow non-QoS resources to
>> be
>> >> assigned to hosts/storage devices that ensure QoS? For provisioned
>> IOPS, I
>> >> think a side effect of the current implementation is SolidFire volumes
>> >> always have a QoS. However, for hypervisor throttled I/O, it seems
>> >> entirely possible for non-QoS VMs to allocated side-by-side with QoS
>> VMs.
>> >> In this scenario, a greedy, unbounded VM could potentially starve out
>> the
>> >> other VMs on the host -- preventing the QoSes defined the collocated
>> VMs
>> >> from being fulfilled.
>> >>
>> >
>> > You can make SolidFire volumes (inside and outside of CS) and not
>> specify
>> > IOPS. You'll still get guaranteed IOPS, but it will be at the defaults
>> we
>> > choose. Unless you over-provision IOPS on a SolidFire SAN, you will have
>> > your Mins met.
>> >
>> > It sounds like you're perhaps looking for a storage tags exclusions
>> list,
>> > which might be nice to have at some point (i.e. don't deploy my volume
>> to
>> > storage with these following tags).
>>
>> I don't like the idea of a storage tags exclusion list as it would
>> complicate component assembly. It would require a storage plugin to
>> anticipate all of the possible component assemblies and determine the
>> invalid relationships. I prefer that drivers express their capabilities
>> which can be matched to a set of requested requirements.
>>
>
> I'm not sure why a storage plug-in would care about inclusion or exclusion
> lists. It just needs to advertise its functionality in a way the allocator
> understands.
>
>
>>
>> >
>> > I agree with your assessment of Hypervisor QoS. Since it only limits
>> IOPS,
>> > it does not solve the Noisy Neighbor problem. Only a system with
>> guaranteed
>> > minimum IOPS does this.
>>
>> As I said, for SolidFire, it sounds like this problem does not exist.
>> However, I am concerned with the more general case as we supported more
>> devices with provisioned IOPS.
>>
>
> Post 4.2 we need to investigate a way to pass vendor-specific values to
> drivers. Min and Max and pretty industry standard for provisioned IOPS, but
> what if you break them out by read and write or do something else? We need
> a more general mechanism.
>
>>
>> >
>> >
>> >>
>> >> In my opinion, we need to ensure that hypervisor throttled I/O and
>> >> storage provisioned IOPS are mutually exclusive per volume.
>> >
>> >
>> > Agreed
>> >
>> >
>> >> We also need to understand the implications of these QoS guarantees on
>> >> operation of the system to ensure that the underlying hardware
>> resources
>> >> can fulfill them. Given the time frame, we will likely be forced to
>> make
>> >> compromises to achieve these goals, and refine the implementation in
>> 4.3.
>> >>
>> >
>> > I agree, John. I also think you've come up with some great ideas for
>> 4.3. :)
>> >
>> >
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> Yeah, Alex, I think that's the way we were planning (with storage
>> tags).
>> >> I
>> >>> believe John was just throwing out an idea that - in addition to
>> storage
>> >>> tags - we could look into these allocators (storage QoS being
>> preferred,
>> >>> then hypervisor QoS if storage QoS is not available, but hypervisor
>> QoS
>> >> is).
>> >>>
>> >>> I think John's concern is that you can enter in values for Wei's and
>> my
>> >>> feature that are not honored by other vendors (at least yet), so he
>> was
>> >>> hoping - in addition to storage tags - for the allocators to prefer
>> these
>> >>> vendors when these fields are filled in. As it stands today in
>> >> CloudStack,
>> >>> we already have this kind of an issue with other features (fields in
>> >>> dialogs for features that not all vendors support). Perhaps post 4.2
>> we
>> >>> could look into generic name/value pairs (this is how OpenStack
>> addresses
>> >>> the issue).
>> >>>
>> >>> Honestly, I think we're too late in the game (two weeks until code
>> >> freeze)
>> >>> to go too deeply down that path in 4.2.
>> >>>
>> >>> It's probably better if we - at least for 4.2 - keep Wei's fields and
>> my
>> >>> fields as is, make sure only one or the other feature has data entered
>> >> for
>> >>> it (or neither), and call it good for 4.2.
>> >>>
>> >>> Then let's step back and look into a more general-purpose design that
>> can
>> >>> be applied throughout CloudStack where we have these situations.
>> >>>
>> >>> What do you think?
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> I just published my review @ https://reviews.apache.org/r/11479/.
>> >>>>
>> >>>> I apologize for the delay,
>> >>>> -John
>> >>>>
>> >>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com>
>> >>>> wrote:
>> >>>>
>> >>>>> No problem, John.
>> >>>>>
>> >>>>> I still want to re-review it by myself before coming up with a new
>> >> patch
>> >>>>> file.
>> >>>>>
>> >>>>> Also, maybe I should first wait for Wei's changes to be checked in
>> and
>> >>>>> merge those into mine before generating a new patch file?
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> I just realized that I forgot to publish my review. I am offline
>> ATM,
>> >>>>>> but I will publish it in the next couple of hours.
>> >>>>>>
>> >>>>>> Do you plan to update your the patch in Review Board?
>> >>>>>>
>> >>>>>> Sorry for the oversight,
>> >>>>>> -John
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> >>>>>> <mi...@solidfire.com> wrote:
>> >>>>>>
>> >>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>> >>>>>>>
>> >>>>>>> Just an FYI that I believe I have implemented all the areas we
>> wanted
>> >>>>>>> addressed.
>> >>>>>>>
>> >>>>>>> I plan to review the code again tomorrow morning or afternoon,
>> then
>> >>>> send
>> >>>>>> in
>> >>>>>>> another patch.
>> >>>>>>>
>> >>>>>>> Thanks for all the work on this everyone!
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>
>> >>>>>>>> Sure, that sounds good.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
>> ustcweizhou@gmail.com>
>> >>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi Mike,
>> >>>>>>>>>
>> >>>>>>>>> It looks the two feature do not have many conflicts in Java
>> code,
>> >>>>>> except
>> >>>>>>>>> the cloudstack UI.
>> >>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>> >>>> master
>> >>>>>>>>> this
>> >>>>>>>>> week, so that you can develop based on it.
>> >>>>>>>>>
>> >>>>>>>>> -Wei
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>> >>>>>>>>>
>> >>>>>>>>>> Hey John,
>> >>>>>>>>>>
>> >>>>>>>>>> The SolidFire patch does not depend on the object_store branch,
>> >> but
>> >>>> -
>> >>>>>> as
>> >>>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
>> >>>> branch
>> >>>>>>>>> into
>> >>>>>>>>>> the object_store branch before object_store goes into master.
>> >>>>>>>>>>
>> >>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>> >>>> strategy.
>> >>>>>>>>>> Perhaps Wei can chime in on that.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> >> jburwell@basho.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Mike,
>> >>>>>>>>>>>
>> >>>>>>>>>>> We have a delicate merge dance to perform. The
>> >> disk_io_throttling,
>> >>>>>>>>>>> solidfire, and object_store appear to have a number of
>> >> overlapping
>> >>>>>>>>>>> elements. I understand the dependencies between the patches
>> to
>> >> be
>> >>>> as
>> >>>>>>>>>>> follows:
>> >>>>>>>>>>>
>> >>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>> >>>>>>>>>>>
>> >>>>>>>>>>> Am I correct that the device management aspects of SolidFire
>> are
>> >>>>>>>>> additive
>> >>>>>>>>>>> to the object_store branch or there are circular dependency
>> >> between
>> >>>>>>>>> the
>> >>>>>>>>>>> branches? Once we understand the dependency graph, we can
>> >>>> determine
>> >>>>>>>>> the
>> >>>>>>>>>>> best approach to land the changes in master.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> -John
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Also, if we are good with Edison merging my code into his
>> branch
>> >>>>>>>>> before
>> >>>>>>>>>>>> going into master, I am good with that.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his
>> merge
>> >> and
>> >>>>>>>>> we
>> >>>>>>>>>> can
>> >>>>>>>>>>>> deal with Burst IOPS then, as well.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Let me make sure I follow where we're going here:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>> >>>> storage
>> >>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
>> which
>> >>>>>>>>>>> currently
>> >>>>>>>>>>>>> sends several commands to the hypervisor in use (although it
>> >> does
>> >>>>>>>>> not
>> >>>>>>>>>>> know
>> >>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
>> use))
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
>> field
>> >>>> (if
>> >>>>>>>>>> not
>> >>>>>>>>>>>>> present, we default to false). This info is stored in the
>> >>>>>>>>>>>>> storage_pool_details table.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>> >>>>>>>>> question, we
>> >>>>>>>>>>>>> pass the managed property along (this takes the place of the
>> >>>>>>>>>>>>> StoragePoolType.Dynamic check).
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
>> >> the
>> >>>>>>>>>> managed
>> >>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
>> data
>> >>>>>>>>>>> structure is
>> >>>>>>>>>>>>> created and the rest of the attach command executes to
>> attach
>> >> the
>> >>>>>>>>>>> volume.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>> >>>> volume,
>> >>>>>>>>>> the
>> >>>>>>>>>>>>> same check is made. If managed, the hypervisor data
>> structure
>> >> is
>> >>>>>>>>>>> removed.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>> >> unless
>> >>>>>>>>> it is
>> >>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
>> some
>> >>>>>>>>> idea,
>> >>>>>>>>>>>>> that'd be cool.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>> >>>> implementation
>> >>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>> >> think
>> >>>>>>>>> we
>> >>>>>>>>>>> should
>> >>>>>>>>>>>>>> beef up the description in the documentation regarding the
>> >>>>>>>>> expected
>> >>>>>>>>>>> format
>> >>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>> >>>>>>>>> returned on
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>> >>>> response.
>> >>>>>>>>>> Why
>> >>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>> >>>> clients
>> >>>>>>>>> to
>> >>>>>>>>>> be
>> >>>>>>>>>>>>>> able to inspect the contents of the details field."
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>> >>>> Offering
>> >>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able
>> to
>> >>>>>>>>>> associate
>> >>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>> >> Maybe
>> >>>>>>>>> it
>> >>>>>>>>>>> could
>> >>>>>>>>>>>>>> go there?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in
>> the
>> >> GUI
>> >>>>>>>>> if
>> >>>>>>>>>>> it's
>> >>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Comments below in red.
Thanks
On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my comment in-line below.
>
> Thanks,
> -John
>
> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hi John,
> >
> > I've put comments below in red.
> >
> > Thanks!
> >
> >
> > On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> First and foremost, we must ensure that these two features are mutually
> >> exclusive in 4.2. We don't want to find a configuration that contains
> both
> >> hypervisor and storage IOPS guarantees that leads to non-deterministic
> >> operations.
> >
> >
> > Agreed
> >
> >
> >> Restricting QoS expression to be either hypervisor or storage oriented
> >> solves the problem in short term. As I understand storage tags, we
> have no
> >> means of expressing this type of mutual exclusion. I wasn't necessarily
> >> intending that we implement this allocation model in 4.3, but instead,
> >> determine if this type model would be one we would want to support in
> the
> >> future. If so, I would encourage us to ensure that the data model and
> >> current implementation would not preclude evolution in that direction.
> My
> >> view is that this type of allocation model is what user's expect of
> "cloud"
> >> systems -- selecting the best available resource set to fulfill a set
> of
> >> system requirements.
> >>
> >
> > I believe we have meet your requirement here in that what we've
> implemented
> > should not make refinement difficult in the future. If we don't modify
> > allocators for 4.2, but we do for 4.3, we've made relatively simple
> changes
> > to enhance the current functioning of the system.
> >
>
> Looking through both patches, I have to say that the aggregated result
> seems a bit confusing. There are six new attributes for throttled I/O and
> two for provisioned IOPS with no obvious grouping. My concern is not
> technical, but rather, about maintainability. At minimum, Javadoc should
> be added explaining the two sets of attributes and their mutual exclusion.
>
I agree: We need JavaDoc to explain them and their mutual exclusion.
>
> The other part that is interesting is that throttled I/O provides both an
> IOPS and byte measured QoS as a rate where provisioned IOPS uses a range.
> In order to select the best available resource to fulfill a QoS, we would
> need to have the QoS expression normalized in terms of units (IOPS or
> bytes) and their expression (rate vs. range). If we want to achieve a
> model like I described, I think we would need to resolve this issue in 4.2
> to ensure a viable migration path.
I think we're not likely to be able to normalize the input for 4.2. Plus
people probably want to input the data in terms they're familiar with for
the products in question.
Ideally we would fix the way we do storage tagging and let the user send
key/value pairs to each vendor that could be selected due to a given
storage tag. I'm still not sure that would solve it because what happens if
you change the storage tag of a given Primary Storage after having created
a Disk Offering?
Basically storage tagging is kind of a mess and we should re-think it.
Also, we need to have a way for the drivers to expose their supported
feature sets so the allocators can make good choices.
>
> >
> >>
> >> As I think through the implications of these requirements and reflect on
> >> the reviews, I don't understand why they haven't already impacted the
> >> allocators and planners. As it stands, the current provisioned IOPS
> has no
> >> checks to ensure that the volumes are allocated to devices that have
> >> capacity to fulfill the requested QoS. Therefore, as I understand the
> >> current patch, we can overcommit storage resources -- potentially
> causing
> >> none of the QoS obligations from being fulfilled. It seems to me that a
> >> DataStore supporting provisioned IOPS should express the maximum IOPS
> which
> >> it can support and some type of overcommitment factor. This information
> >> should be used by the storage allocators to determine the device best
> able
> >> to support the resources needs of a volume. It seems that a similar
> set of
> >> considerations would need to be added to the Hypervisor layer which
> >> allocating a VM to a host to prevent oversubscription.
> >>
> >
> > Yeah, for this first release, we just followed the path that was
> previously
> > established for other properties you see on dialogs in CS: Just because
> > they're there doesn't mean the vendor your VM is deployed to supports
> them.
> > It is then up to the admin to make sure he inputs, say, a storage tag
> that
> > confines the deployment only to storage that supports the selected
> > features. This is not ideal, but it's kind of the way CloudStack works
> > today.
>
> I understand the tag functionality, and the need for the administrator to
> very carefully construct offerings. My concern is that we over guarantee a
> resource's available IOPS. For the purposes of illustration, let's say we
> have a SolidFire, and the max IOPS for that device is 100000. We also
> know that we can safely oversubscribe by 50%. Therefore, we need to ensure
> that we don't allocate more than 150,000 guaranteed IOPS from that device.
> Intuitively, it seems like the DataStore configuration should have a max
> assignable IOPS and overcommitment factor. As we allocate volumes and
> attach VMs, we need to ensure that guarantee more IOPS exceed the
> configured maximum for a DataStore. Does that make sense?
>
I think that's a good idea for a future enhancement. I'm not even sure I
can query the SAN to find out how many IOPS safely remain. I'd have to get
all of the min values for all of the volumes on the SAN and total them up,
I suppose, and subtract it from the total (user facing) supported IOPS of
the system.
>
> >
> >
> >>
> >> Another question occurs to me -- should we allow non-QoS resources to be
> >> assigned to hosts/storage devices that ensure QoS? For provisioned
> IOPS, I
> >> think a side effect of the current implementation is SolidFire volumes
> >> always have a QoS. However, for hypervisor throttled I/O, it seems
> >> entirely possible for non-QoS VMs to allocated side-by-side with QoS
> VMs.
> >> In this scenario, a greedy, unbounded VM could potentially starve out
> the
> >> other VMs on the host -- preventing the QoSes defined the collocated VMs
> >> from being fulfilled.
> >>
> >
> > You can make SolidFire volumes (inside and outside of CS) and not specify
> > IOPS. You'll still get guaranteed IOPS, but it will be at the defaults we
> > choose. Unless you over-provision IOPS on a SolidFire SAN, you will have
> > your Mins met.
> >
> > It sounds like you're perhaps looking for a storage tags exclusions list,
> > which might be nice to have at some point (i.e. don't deploy my volume to
> > storage with these following tags).
>
> I don't like the idea of a storage tags exclusion list as it would
> complicate component assembly. It would require a storage plugin to
> anticipate all of the possible component assemblies and determine the
> invalid relationships. I prefer that drivers express their capabilities
> which can be matched to a set of requested requirements.
>
I'm not sure why a storage plug-in would care about inclusion or exclusion
lists. It just needs to advertise its functionality in a way the allocator
understands.
>
> >
> > I agree with your assessment of Hypervisor QoS. Since it only limits
> IOPS,
> > it does not solve the Noisy Neighbor problem. Only a system with
> guaranteed
> > minimum IOPS does this.
>
> As I said, for SolidFire, it sounds like this problem does not exist.
> However, I am concerned with the more general case as we supported more
> devices with provisioned IOPS.
>
Post 4.2 we need to investigate a way to pass vendor-specific values to
drivers. Min and Max and pretty industry standard for provisioned IOPS, but
what if you break them out by read and write or do something else? We need
a more general mechanism.
>
> >
> >
> >>
> >> In my opinion, we need to ensure that hypervisor throttled I/O and
> >> storage provisioned IOPS are mutually exclusive per volume.
> >
> >
> > Agreed
> >
> >
> >> We also need to understand the implications of these QoS guarantees on
> >> operation of the system to ensure that the underlying hardware resources
> >> can fulfill them. Given the time frame, we will likely be forced to
> make
> >> compromises to achieve these goals, and refine the implementation in
> 4.3.
> >>
> >
> > I agree, John. I also think you've come up with some great ideas for
> 4.3. :)
> >
> >
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Yeah, Alex, I think that's the way we were planning (with storage
> tags).
> >> I
> >>> believe John was just throwing out an idea that - in addition to
> storage
> >>> tags - we could look into these allocators (storage QoS being
> preferred,
> >>> then hypervisor QoS if storage QoS is not available, but hypervisor QoS
> >> is).
> >>>
> >>> I think John's concern is that you can enter in values for Wei's and my
> >>> feature that are not honored by other vendors (at least yet), so he was
> >>> hoping - in addition to storage tags - for the allocators to prefer
> these
> >>> vendors when these fields are filled in. As it stands today in
> >> CloudStack,
> >>> we already have this kind of an issue with other features (fields in
> >>> dialogs for features that not all vendors support). Perhaps post 4.2 we
> >>> could look into generic name/value pairs (this is how OpenStack
> addresses
> >>> the issue).
> >>>
> >>> Honestly, I think we're too late in the game (two weeks until code
> >> freeze)
> >>> to go too deeply down that path in 4.2.
> >>>
> >>> It's probably better if we - at least for 4.2 - keep Wei's fields and
> my
> >>> fields as is, make sure only one or the other feature has data entered
> >> for
> >>> it (or neither), and call it good for 4.2.
> >>>
> >>> Then let's step back and look into a more general-purpose design that
> can
> >>> be applied throughout CloudStack where we have these situations.
> >>>
> >>> What do you think?
> >>>
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> I just published my review @ https://reviews.apache.org/r/11479/.
> >>>>
> >>>> I apologize for the delay,
> >>>> -John
> >>>>
> >>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> No problem, John.
> >>>>>
> >>>>> I still want to re-review it by myself before coming up with a new
> >> patch
> >>>>> file.
> >>>>>
> >>>>> Also, maybe I should first wait for Wei's changes to be checked in
> and
> >>>>> merge those into mine before generating a new patch file?
> >>>>>
> >>>>>
> >>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> I just realized that I forgot to publish my review. I am offline
> ATM,
> >>>>>> but I will publish it in the next couple of hours.
> >>>>>>
> >>>>>> Do you plan to update your the patch in Review Board?
> >>>>>>
> >>>>>> Sorry for the oversight,
> >>>>>> -John
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>> <mi...@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>>>>>
> >>>>>>> Just an FYI that I believe I have implemented all the areas we
> wanted
> >>>>>>> addressed.
> >>>>>>>
> >>>>>>> I plan to review the code again tomorrow morning or afternoon, then
> >>>> send
> >>>>>> in
> >>>>>>> another patch.
> >>>>>>>
> >>>>>>> Thanks for all the work on this everyone!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> Sure, that sounds good.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <ustcweizhou@gmail.com
> >
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Mike,
> >>>>>>>>>
> >>>>>>>>> It looks the two feature do not have many conflicts in Java code,
> >>>>>> except
> >>>>>>>>> the cloudstack UI.
> >>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
> >>>> master
> >>>>>>>>> this
> >>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>
> >>>>>>>>> -Wei
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>
> >>>>>>>>>> Hey John,
> >>>>>>>>>>
> >>>>>>>>>> The SolidFire patch does not depend on the object_store branch,
> >> but
> >>>> -
> >>>>>> as
> >>>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
> >>>> branch
> >>>>>>>>> into
> >>>>>>>>>> the object_store branch before object_store goes into master.
> >>>>>>>>>>
> >>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
> >>>> strategy.
> >>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> >> jburwell@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> We have a delicate merge dance to perform. The
> >> disk_io_throttling,
> >>>>>>>>>>> solidfire, and object_store appear to have a number of
> >> overlapping
> >>>>>>>>>>> elements. I understand the dependencies between the patches to
> >> be
> >>>> as
> >>>>>>>>>>> follows:
> >>>>>>>>>>>
> >>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>>>
> >>>>>>>>>>> Am I correct that the device management aspects of SolidFire
> are
> >>>>>>>>> additive
> >>>>>>>>>>> to the object_store branch or there are circular dependency
> >> between
> >>>>>>>>> the
> >>>>>>>>>>> branches? Once we understand the dependency graph, we can
> >>>> determine
> >>>>>>>>> the
> >>>>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Also, if we are good with Edison merging my code into his
> branch
> >>>>>>>>> before
> >>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>
> >>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge
> >> and
> >>>>>>>>> we
> >>>>>>>>>> can
> >>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
> >>>> storage
> >>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in,
> which
> >>>>>>>>>>> currently
> >>>>>>>>>>>>> sends several commands to the hypervisor in use (although it
> >> does
> >>>>>>>>> not
> >>>>>>>>>>> know
> >>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in
> use))
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url
> field
> >>>> (if
> >>>>>>>>>> not
> >>>>>>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>>>>>> question, we
> >>>>>>>>>>>>> pass the managed property along (this takes the place of the
> >>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
> >> the
> >>>>>>>>>> managed
> >>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor
> data
> >>>>>>>>>>> structure is
> >>>>>>>>>>>>> created and the rest of the attach command executes to attach
> >> the
> >>>>>>>>>>> volume.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> >>>> volume,
> >>>>>>>>>> the
> >>>>>>>>>>>>> same check is made. If managed, the hypervisor data structure
> >> is
> >>>>>>>>>>> removed.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
> >> unless
> >>>>>>>>> it is
> >>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have
> some
> >>>>>>>>> idea,
> >>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >>>> implementation
> >>>>>>>>>>>>>> attributes. I always wondered about the details field. I
> >> think
> >>>>>>>>> we
> >>>>>>>>>>> should
> >>>>>>>>>>>>>> beef up the description in the documentation regarding the
> >>>>>>>>> expected
> >>>>>>>>>>> format
> >>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>>>>>> returned on
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> >>>> response.
> >>>>>>>>>> Why
> >>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
> >>>> clients
> >>>>>>>>> to
> >>>>>>>>>> be
> >>>>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> >>>> Offering
> >>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able
> to
> >>>>>>>>>> associate
> >>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
> >> Maybe
> >>>>>>>>> it
> >>>>>>>>>>> could
> >>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the
> >> GUI
> >>>>>>>>> if
> >>>>>>>>>>> it's
> >>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Please see my comment in-line below.
Thanks,
-John
On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi John,
>
> I've put comments below in red.
>
> Thanks!
>
>
> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> First and foremost, we must ensure that these two features are mutually
>> exclusive in 4.2. We don't want to find a configuration that contains both
>> hypervisor and storage IOPS guarantees that leads to non-deterministic
>> operations.
>
>
> Agreed
>
>
>> Restricting QoS expression to be either hypervisor or storage oriented
>> solves the problem in short term. As I understand storage tags, we have no
>> means of expressing this type of mutual exclusion. I wasn't necessarily
>> intending that we implement this allocation model in 4.3, but instead,
>> determine if this type model would be one we would want to support in the
>> future. If so, I would encourage us to ensure that the data model and
>> current implementation would not preclude evolution in that direction. My
>> view is that this type of allocation model is what user's expect of "cloud"
>> systems -- selecting the best available resource set to fulfill a set of
>> system requirements.
>>
>
> I believe we have meet your requirement here in that what we've implemented
> should not make refinement difficult in the future. If we don't modify
> allocators for 4.2, but we do for 4.3, we've made relatively simple changes
> to enhance the current functioning of the system.
>
Looking through both patches, I have to say that the aggregated result seems a bit confusing. There are six new attributes for throttled I/O and two for provisioned IOPS with no obvious grouping. My concern is not technical, but rather, about maintainability. At minimum, Javadoc should be added explaining the two sets of attributes and their mutual exclusion.
The other part that is interesting is that throttled I/O provides both an IOPS and byte measured QoS as a rate where provisioned IOPS uses a range. In order to select the best available resource to fulfill a QoS, we would need to have the QoS expression normalized in terms of units (IOPS or bytes) and their expression (rate vs. range). If we want to achieve a model like I described, I think we would need to resolve this issue in 4.2 to ensure a viable migration path.
>
>>
>> As I think through the implications of these requirements and reflect on
>> the reviews, I don't understand why they haven't already impacted the
>> allocators and planners. As it stands, the current provisioned IOPS has no
>> checks to ensure that the volumes are allocated to devices that have
>> capacity to fulfill the requested QoS. Therefore, as I understand the
>> current patch, we can overcommit storage resources -- potentially causing
>> none of the QoS obligations from being fulfilled. It seems to me that a
>> DataStore supporting provisioned IOPS should express the maximum IOPS which
>> it can support and some type of overcommitment factor. This information
>> should be used by the storage allocators to determine the device best able
>> to support the resources needs of a volume. It seems that a similar set of
>> considerations would need to be added to the Hypervisor layer which
>> allocating a VM to a host to prevent oversubscription.
>>
>
> Yeah, for this first release, we just followed the path that was previously
> established for other properties you see on dialogs in CS: Just because
> they're there doesn't mean the vendor your VM is deployed to supports them.
> It is then up to the admin to make sure he inputs, say, a storage tag that
> confines the deployment only to storage that supports the selected
> features. This is not ideal, but it's kind of the way CloudStack works
> today.
I understand the tag functionality, and the need for the administrator to very carefully construct offerings. My concern is that we over guarantee a resource's available IOPS. For the purposes of illustration, let's say we have a SolidFire, and the max IOPS for that device is 100000. We also know that we can safely oversubscribe by 50%. Therefore, we need to ensure that we don't allocate more than 150,000 guaranteed IOPS from that device. Intuitively, it seems like the DataStore configuration should have a max assignable IOPS and overcommitment factor. As we allocate volumes and attach VMs, we need to ensure that guarantee more IOPS exceed the configured maximum for a DataStore. Does that make sense?
>
>
>>
>> Another question occurs to me -- should we allow non-QoS resources to be
>> assigned to hosts/storage devices that ensure QoS? For provisioned IOPS, I
>> think a side effect of the current implementation is SolidFire volumes
>> always have a QoS. However, for hypervisor throttled I/O, it seems
>> entirely possible for non-QoS VMs to allocated side-by-side with QoS VMs.
>> In this scenario, a greedy, unbounded VM could potentially starve out the
>> other VMs on the host -- preventing the QoSes defined the collocated VMs
>> from being fulfilled.
>>
>
> You can make SolidFire volumes (inside and outside of CS) and not specify
> IOPS. You'll still get guaranteed IOPS, but it will be at the defaults we
> choose. Unless you over-provision IOPS on a SolidFire SAN, you will have
> your Mins met.
>
> It sounds like you're perhaps looking for a storage tags exclusions list,
> which might be nice to have at some point (i.e. don't deploy my volume to
> storage with these following tags).
I don't like the idea of a storage tags exclusion list as it would complicate component assembly. It would require a storage plugin to anticipate all of the possible component assemblies and determine the invalid relationships. I prefer that drivers express their capabilities which can be matched to a set of requested requirements.
>
> I agree with your assessment of Hypervisor QoS. Since it only limits IOPS,
> it does not solve the Noisy Neighbor problem. Only a system with guaranteed
> minimum IOPS does this.
As I said, for SolidFire, it sounds like this problem does not exist. However, I am concerned with the more general case as we supported more devices with provisioned IOPS.
>
>
>>
>> In my opinion, we need to ensure that hypervisor throttled I/O and
>> storage provisioned IOPS are mutually exclusive per volume.
>
>
> Agreed
>
>
>> We also need to understand the implications of these QoS guarantees on
>> operation of the system to ensure that the underlying hardware resources
>> can fulfill them. Given the time frame, we will likely be forced to make
>> compromises to achieve these goals, and refine the implementation in 4.3.
>>
>
> I agree, John. I also think you've come up with some great ideas for 4.3. :)
>
>
>>
>> Thanks,
>> -John
>>
>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Yeah, Alex, I think that's the way we were planning (with storage tags).
>> I
>>> believe John was just throwing out an idea that - in addition to storage
>>> tags - we could look into these allocators (storage QoS being preferred,
>>> then hypervisor QoS if storage QoS is not available, but hypervisor QoS
>> is).
>>>
>>> I think John's concern is that you can enter in values for Wei's and my
>>> feature that are not honored by other vendors (at least yet), so he was
>>> hoping - in addition to storage tags - for the allocators to prefer these
>>> vendors when these fields are filled in. As it stands today in
>> CloudStack,
>>> we already have this kind of an issue with other features (fields in
>>> dialogs for features that not all vendors support). Perhaps post 4.2 we
>>> could look into generic name/value pairs (this is how OpenStack addresses
>>> the issue).
>>>
>>> Honestly, I think we're too late in the game (two weeks until code
>> freeze)
>>> to go too deeply down that path in 4.2.
>>>
>>> It's probably better if we - at least for 4.2 - keep Wei's fields and my
>>> fields as is, make sure only one or the other feature has data entered
>> for
>>> it (or neither), and call it good for 4.2.
>>>
>>> Then let's step back and look into a more general-purpose design that can
>>> be applied throughout CloudStack where we have these situations.
>>>
>>> What do you think?
>>>
>>>
>>>
>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> I just published my review @ https://reviews.apache.org/r/11479/.
>>>>
>>>> I apologize for the delay,
>>>> -John
>>>>
>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> No problem, John.
>>>>>
>>>>> I still want to re-review it by myself before coming up with a new
>> patch
>>>>> file.
>>>>>
>>>>> Also, maybe I should first wait for Wei's changes to be checked in and
>>>>> merge those into mine before generating a new patch file?
>>>>>
>>>>>
>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> I just realized that I forgot to publish my review. I am offline ATM,
>>>>>> but I will publish it in the next couple of hours.
>>>>>>
>>>>>> Do you plan to update your the patch in Review Board?
>>>>>>
>>>>>> Sorry for the oversight,
>>>>>> -John
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>>>> <mi...@solidfire.com> wrote:
>>>>>>
>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>>>
>>>>>>> Just an FYI that I believe I have implemented all the areas we wanted
>>>>>>> addressed.
>>>>>>>
>>>>>>> I plan to review the code again tomorrow morning or afternoon, then
>>>> send
>>>>>> in
>>>>>>> another patch.
>>>>>>>
>>>>>>> Thanks for all the work on this everyone!
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> Sure, that sounds good.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Mike,
>>>>>>>>>
>>>>>>>>> It looks the two feature do not have many conflicts in Java code,
>>>>>> except
>>>>>>>>> the cloudstack UI.
>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>>>> master
>>>>>>>>> this
>>>>>>>>> week, so that you can develop based on it.
>>>>>>>>>
>>>>>>>>> -Wei
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>
>>>>>>>>>> Hey John,
>>>>>>>>>>
>>>>>>>>>> The SolidFire patch does not depend on the object_store branch,
>> but
>>>> -
>>>>>> as
>>>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
>>>> branch
>>>>>>>>> into
>>>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>>>
>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>>>> strategy.
>>>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
>> jburwell@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> We have a delicate merge dance to perform. The
>> disk_io_throttling,
>>>>>>>>>>> solidfire, and object_store appear to have a number of
>> overlapping
>>>>>>>>>>> elements. I understand the dependencies between the patches to
>> be
>>>> as
>>>>>>>>>>> follows:
>>>>>>>>>>>
>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>>>
>>>>>>>>>>> Am I correct that the device management aspects of SolidFire are
>>>>>>>>> additive
>>>>>>>>>>> to the object_store branch or there are circular dependency
>> between
>>>>>>>>> the
>>>>>>>>>>> branches? Once we understand the dependency graph, we can
>>>> determine
>>>>>>>>> the
>>>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Also, if we are good with Edison merging my code into his branch
>>>>>>>>> before
>>>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>>>
>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge
>> and
>>>>>>>>> we
>>>>>>>>>> can
>>>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>>>> storage
>>>>>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
>>>>>>>>>>> currently
>>>>>>>>>>>>> sends several commands to the hypervisor in use (although it
>> does
>>>>>>>>> not
>>>>>>>>>>> know
>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url field
>>>> (if
>>>>>>>>>> not
>>>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>>>> question, we
>>>>>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>>>
>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
>> the
>>>>>>>>>> managed
>>>>>>>>>>>>> property. If true for an attach, the necessary hypervisor data
>>>>>>>>>>> structure is
>>>>>>>>>>>>> created and the rest of the attach command executes to attach
>> the
>>>>>>>>>>> volume.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>>>> volume,
>>>>>>>>>> the
>>>>>>>>>>>>> same check is made. If managed, the hypervisor data structure
>> is
>>>>>>>>>>> removed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
>> unless
>>>>>>>>> it is
>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
>>>>>>>>> idea,
>>>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>>>> implementation
>>>>>>>>>>>>>> attributes. I always wondered about the details field. I
>> think
>>>>>>>>> we
>>>>>>>>>>> should
>>>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>>>> expected
>>>>>>>>>>> format
>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>>>> returned on
>>>>>>>>>>> the
>>>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>>>> response.
>>>>>>>>>> Why
>>>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>>>> clients
>>>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>>>> Offering
>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
>>>>>>>>>> associate
>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
>> Maybe
>>>>>>>>> it
>>>>>>>>>>> could
>>>>>>>>>>>>>> go there?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the
>> GUI
>>>>>>>>> if
>>>>>>>>>>> it's
>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
I've put comments below in red.
Thanks!
On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> First and foremost, we must ensure that these two features are mutually
> exclusive in 4.2. We don't want to find a configuration that contains both
> hypervisor and storage IOPS guarantees that leads to non-deterministic
> operations.
Agreed
> Restricting QoS expression to be either hypervisor or storage oriented
> solves the problem in short term. As I understand storage tags, we have no
> means of expressing this type of mutual exclusion. I wasn't necessarily
> intending that we implement this allocation model in 4.3, but instead,
> determine if this type model would be one we would want to support in the
> future. If so, I would encourage us to ensure that the data model and
> current implementation would not preclude evolution in that direction. My
> view is that this type of allocation model is what user's expect of "cloud"
> systems -- selecting the best available resource set to fulfill a set of
> system requirements.
>
I believe we have meet your requirement here in that what we've implemented
should not make refinement difficult in the future. If we don't modify
allocators for 4.2, but we do for 4.3, we've made relatively simple changes
to enhance the current functioning of the system.
>
> As I think through the implications of these requirements and reflect on
> the reviews, I don't understand why they haven't already impacted the
> allocators and planners. As it stands, the current provisioned IOPS has no
> checks to ensure that the volumes are allocated to devices that have
> capacity to fulfill the requested QoS. Therefore, as I understand the
> current patch, we can overcommit storage resources -- potentially causing
> none of the QoS obligations from being fulfilled. It seems to me that a
> DataStore supporting provisioned IOPS should express the maximum IOPS which
> it can support and some type of overcommitment factor. This information
> should be used by the storage allocators to determine the device best able
> to support the resources needs of a volume. It seems that a similar set of
> considerations would need to be added to the Hypervisor layer which
> allocating a VM to a host to prevent oversubscription.
>
Yeah, for this first release, we just followed the path that was previously
established for other properties you see on dialogs in CS: Just because
they're there doesn't mean the vendor your VM is deployed to supports them.
It is then up to the admin to make sure he inputs, say, a storage tag that
confines the deployment only to storage that supports the selected
features. This is not ideal, but it's kind of the way CloudStack works
today.
>
> Another question occurs to me -- should we allow non-QoS resources to be
> assigned to hosts/storage devices that ensure QoS? For provisioned IOPS, I
> think a side effect of the current implementation is SolidFire volumes
> always have a QoS. However, for hypervisor throttled I/O, it seems
> entirely possible for non-QoS VMs to allocated side-by-side with QoS VMs.
> In this scenario, a greedy, unbounded VM could potentially starve out the
> other VMs on the host -- preventing the QoSes defined the collocated VMs
> from being fulfilled.
>
You can make SolidFire volumes (inside and outside of CS) and not specify
IOPS. You'll still get guaranteed IOPS, but it will be at the defaults we
choose. Unless you over-provision IOPS on a SolidFire SAN, you will have
your Mins met.
It sounds like you're perhaps looking for a storage tags exclusions list,
which might be nice to have at some point (i.e. don't deploy my volume to
storage with these following tags).
I agree with your assessment of Hypervisor QoS. Since it only limits IOPS,
it does not solve the Noisy Neighbor problem. Only a system with guaranteed
minimum IOPS does this.
>
> In my opinion, we need to ensure that hypervisor throttled I/O and
> storage provisioned IOPS are mutually exclusive per volume.
Agreed
> We also need to understand the implications of these QoS guarantees on
> operation of the system to ensure that the underlying hardware resources
> can fulfill them. Given the time frame, we will likely be forced to make
> compromises to achieve these goals, and refine the implementation in 4.3.
>
I agree, John. I also think you've come up with some great ideas for 4.3. :)
>
> Thanks,
> -John
>
> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hi,
> >
> > Yeah, Alex, I think that's the way we were planning (with storage tags).
> I
> > believe John was just throwing out an idea that - in addition to storage
> > tags - we could look into these allocators (storage QoS being preferred,
> > then hypervisor QoS if storage QoS is not available, but hypervisor QoS
> is).
> >
> > I think John's concern is that you can enter in values for Wei's and my
> > feature that are not honored by other vendors (at least yet), so he was
> > hoping - in addition to storage tags - for the allocators to prefer these
> > vendors when these fields are filled in. As it stands today in
> CloudStack,
> > we already have this kind of an issue with other features (fields in
> > dialogs for features that not all vendors support). Perhaps post 4.2 we
> > could look into generic name/value pairs (this is how OpenStack addresses
> > the issue).
> >
> > Honestly, I think we're too late in the game (two weeks until code
> freeze)
> > to go too deeply down that path in 4.2.
> >
> > It's probably better if we - at least for 4.2 - keep Wei's fields and my
> > fields as is, make sure only one or the other feature has data entered
> for
> > it (or neither), and call it good for 4.2.
> >
> > Then let's step back and look into a more general-purpose design that can
> > be applied throughout CloudStack where we have these situations.
> >
> > What do you think?
> >
> >
> >
> > On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> I just published my review @ https://reviews.apache.org/r/11479/.
> >>
> >> I apologize for the delay,
> >> -John
> >>
> >> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> No problem, John.
> >>>
> >>> I still want to re-review it by myself before coming up with a new
> patch
> >>> file.
> >>>
> >>> Also, maybe I should first wait for Wei's changes to be checked in and
> >>> merge those into mine before generating a new patch file?
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> I just realized that I forgot to publish my review. I am offline ATM,
> >>>> but I will publish it in the next couple of hours.
> >>>>
> >>>> Do you plan to update your the patch in Review Board?
> >>>>
> >>>> Sorry for the oversight,
> >>>> -John
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>> <mi...@solidfire.com> wrote:
> >>>>
> >>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>>>
> >>>>> Just an FYI that I believe I have implemented all the areas we wanted
> >>>>> addressed.
> >>>>>
> >>>>> I plan to review the code again tomorrow morning or afternoon, then
> >> send
> >>>> in
> >>>>> another patch.
> >>>>>
> >>>>> Thanks for all the work on this everyone!
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>>> Sure, that sounds good.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Hi Mike,
> >>>>>>>
> >>>>>>> It looks the two feature do not have many conflicts in Java code,
> >>>> except
> >>>>>>> the cloudstack UI.
> >>>>>>> If you do not mind, I will merge disk_io_throttling branch into
> >> master
> >>>>>>> this
> >>>>>>> week, so that you can develop based on it.
> >>>>>>>
> >>>>>>> -Wei
> >>>>>>>
> >>>>>>>
> >>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>
> >>>>>>>> Hey John,
> >>>>>>>>
> >>>>>>>> The SolidFire patch does not depend on the object_store branch,
> but
> >> -
> >>>> as
> >>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
> >> branch
> >>>>>>> into
> >>>>>>>> the object_store branch before object_store goes into master.
> >>>>>>>>
> >>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
> >> strategy.
> >>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <
> jburwell@basho.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> We have a delicate merge dance to perform. The
> disk_io_throttling,
> >>>>>>>>> solidfire, and object_store appear to have a number of
> overlapping
> >>>>>>>>> elements. I understand the dependencies between the patches to
> be
> >> as
> >>>>>>>>> follows:
> >>>>>>>>>
> >>>>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>>>
> >>>>>>>>> Am I correct that the device management aspects of SolidFire are
> >>>>>>> additive
> >>>>>>>>> to the object_store branch or there are circular dependency
> between
> >>>>>>> the
> >>>>>>>>> branches? Once we understand the dependency graph, we can
> >> determine
> >>>>>>> the
> >>>>>>>>> best approach to land the changes in master.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Also, if we are good with Edison merging my code into his branch
> >>>>>>> before
> >>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>
> >>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge
> and
> >>>>>>> we
> >>>>>>>> can
> >>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>>>
> >>>>>>>>>>> 1) There should be NO references to hypervisor code in the
> >> storage
> >>>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
> >>>>>>>>> currently
> >>>>>>>>>>> sends several commands to the hypervisor in use (although it
> does
> >>>>>>> not
> >>>>>>>>> know
> >>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>>>>>>>>>>
> >>>>>>>>>>> 2) managed=true or managed=false can be placed in the url field
> >> (if
> >>>>>>>> not
> >>>>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>
> >>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>>>> question, we
> >>>>>>>>>>> pass the managed property along (this takes the place of the
> >>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>
> >>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for
> the
> >>>>>>>> managed
> >>>>>>>>>>> property. If true for an attach, the necessary hypervisor data
> >>>>>>>>> structure is
> >>>>>>>>>>> created and the rest of the attach command executes to attach
> the
> >>>>>>>>> volume.
> >>>>>>>>>>>
> >>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> >> volume,
> >>>>>>>> the
> >>>>>>>>>>> same check is made. If managed, the hypervisor data structure
> is
> >>>>>>>>> removed.
> >>>>>>>>>>>
> >>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2
> unless
> >>>>>>> it is
> >>>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
> >>>>>>> idea,
> >>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> >> implementation
> >>>>>>>>>>>> attributes. I always wondered about the details field. I
> think
> >>>>>>> we
> >>>>>>>>> should
> >>>>>>>>>>>> beef up the description in the documentation regarding the
> >>>>>>> expected
> >>>>>>>>> format
> >>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>>>> returned on
> >>>>>>>>> the
> >>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> >> response.
> >>>>>>>> Why
> >>>>>>>>>>>> don't we return it? It seems like it would be useful for
> >> clients
> >>>>>>> to
> >>>>>>>> be
> >>>>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>>>
> >>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> >> Offering
> >>>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
> >>>>>>>> associate
> >>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table.
> Maybe
> >>>>>>> it
> >>>>>>>>> could
> >>>>>>>>>>>> go there?
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the
> GUI
> >>>>>>> if
> >>>>>>>>> it's
> >>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
First and foremost, we must ensure that these two features are mutually exclusive in 4.2. We don't want to find a configuration that contains both hypervisor and storage IOPS guarantees that leads to non-deterministic operations. Restricting QoS expression to be either hypervisor or storage oriented solves the problem in short term. As I understand storage tags, we have no means of expressing this type of mutual exclusion. I wasn't necessarily intending that we implement this allocation model in 4.3, but instead, determine if this type model would be one we would want to support in the future. If so, I would encourage us to ensure that the data model and current implementation would not preclude evolution in that direction. My view is that this type of allocation model is what user's expect of "cloud" systems -- selecting the best available resource set to fulfill a set of system requirements.
As I think through the implications of these requirements and reflect on the reviews, I don't understand why they haven't already impacted the allocators and planners. As it stands, the current provisioned IOPS has no checks to ensure that the volumes are allocated to devices that have capacity to fulfill the requested QoS. Therefore, as I understand the current patch, we can overcommit storage resources -- potentially causing none of the QoS obligations from being fulfilled. It seems to me that a DataStore supporting provisioned IOPS should express the maximum IOPS which it can support and some type of overcommitment factor. This information should be used by the storage allocators to determine the device best able to support the resources needs of a volume. It seems that a similar set of considerations would need to be added to the Hypervisor layer which allocating a VM to a host to prevent oversubscription.
Another question occurs to me -- should we allow non-QoS resources to be assigned to hosts/storage devices that ensure QoS? For provisioned IOPS, I think a side effect of the current implementation is SolidFire volumes always have a QoS. However, for hypervisor throttled I/O, it seems entirely possible for non-QoS VMs to allocated side-by-side with QoS VMs. In this scenario, a greedy, unbounded VM could potentially starve out the other VMs on the host -- preventing the QoSes defined the collocated VMs from being fulfilled.
In my opinion, we need to ensure that hypervisor throttled I/O and storage provisioned IOPS are mutually exclusive per volume. We also need to understand the implications of these QoS guarantees on operation of the system to ensure that the underlying hardware resources can fulfill them. Given the time frame, we will likely be forced to make compromises to achieve these goals, and refine the implementation in 4.3.
Thanks,
-John
On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi,
>
> Yeah, Alex, I think that's the way we were planning (with storage tags). I
> believe John was just throwing out an idea that - in addition to storage
> tags - we could look into these allocators (storage QoS being preferred,
> then hypervisor QoS if storage QoS is not available, but hypervisor QoS is).
>
> I think John's concern is that you can enter in values for Wei's and my
> feature that are not honored by other vendors (at least yet), so he was
> hoping - in addition to storage tags - for the allocators to prefer these
> vendors when these fields are filled in. As it stands today in CloudStack,
> we already have this kind of an issue with other features (fields in
> dialogs for features that not all vendors support). Perhaps post 4.2 we
> could look into generic name/value pairs (this is how OpenStack addresses
> the issue).
>
> Honestly, I think we're too late in the game (two weeks until code freeze)
> to go too deeply down that path in 4.2.
>
> It's probably better if we - at least for 4.2 - keep Wei's fields and my
> fields as is, make sure only one or the other feature has data entered for
> it (or neither), and call it good for 4.2.
>
> Then let's step back and look into a more general-purpose design that can
> be applied throughout CloudStack where we have these situations.
>
> What do you think?
>
>
>
> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I just published my review @ https://reviews.apache.org/r/11479/.
>>
>> I apologize for the delay,
>> -John
>>
>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> No problem, John.
>>>
>>> I still want to re-review it by myself before coming up with a new patch
>>> file.
>>>
>>> Also, maybe I should first wait for Wei's changes to be checked in and
>>> merge those into mine before generating a new patch file?
>>>
>>>
>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> I just realized that I forgot to publish my review. I am offline ATM,
>>>> but I will publish it in the next couple of hours.
>>>>
>>>> Do you plan to update your the patch in Review Board?
>>>>
>>>> Sorry for the oversight,
>>>> -John
>>>>
>>>>
>>>>
>>>>
>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>>>> <mi...@solidfire.com> wrote:
>>>>
>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>>>
>>>>> Just an FYI that I believe I have implemented all the areas we wanted
>>>>> addressed.
>>>>>
>>>>> I plan to review the code again tomorrow morning or afternoon, then
>> send
>>>> in
>>>>> another patch.
>>>>>
>>>>> Thanks for all the work on this everyone!
>>>>>
>>>>>
>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> Sure, that sounds good.
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>>> Hi Mike,
>>>>>>>
>>>>>>> It looks the two feature do not have many conflicts in Java code,
>>>> except
>>>>>>> the cloudstack UI.
>>>>>>> If you do not mind, I will merge disk_io_throttling branch into
>> master
>>>>>>> this
>>>>>>> week, so that you can develop based on it.
>>>>>>>
>>>>>>> -Wei
>>>>>>>
>>>>>>>
>>>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>
>>>>>>>> Hey John,
>>>>>>>>
>>>>>>>> The SolidFire patch does not depend on the object_store branch, but
>> -
>>>> as
>>>>>>>> Edison mentioned - it might be easier if we merge the SolidFire
>> branch
>>>>>>> into
>>>>>>>> the object_store branch before object_store goes into master.
>>>>>>>>
>>>>>>>> I'm not sure how the disk_io_throttling fits into this merge
>> strategy.
>>>>>>>> Perhaps Wei can chime in on that.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
>>>>>>>>> solidfire, and object_store appear to have a number of overlapping
>>>>>>>>> elements. I understand the dependencies between the patches to be
>> as
>>>>>>>>> follows:
>>>>>>>>>
>>>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>>>
>>>>>>>>> Am I correct that the device management aspects of SolidFire are
>>>>>>> additive
>>>>>>>>> to the object_store branch or there are circular dependency between
>>>>>>> the
>>>>>>>>> branches? Once we understand the dependency graph, we can
>> determine
>>>>>>> the
>>>>>>>>> best approach to land the changes in master.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Also, if we are good with Edison merging my code into his branch
>>>>>>> before
>>>>>>>>>> going into master, I am good with that.
>>>>>>>>>>
>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
>>>>>>> we
>>>>>>>> can
>>>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>>>
>>>>>>>>>>> 1) There should be NO references to hypervisor code in the
>> storage
>>>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
>>>>>>>>> currently
>>>>>>>>>>> sends several commands to the hypervisor in use (although it does
>>>>>>> not
>>>>>>>>> know
>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>>>>>>>>
>>>>>>>>>>> 2) managed=true or managed=false can be placed in the url field
>> (if
>>>>>>>> not
>>>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>>>> storage_pool_details table.
>>>>>>>>>>>
>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>>>> question, we
>>>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>>>
>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
>>>>>>>> managed
>>>>>>>>>>> property. If true for an attach, the necessary hypervisor data
>>>>>>>>> structure is
>>>>>>>>>>> created and the rest of the attach command executes to attach the
>>>>>>>>> volume.
>>>>>>>>>>>
>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
>> volume,
>>>>>>>> the
>>>>>>>>>>> same check is made. If managed, the hypervisor data structure is
>>>>>>>>> removed.
>>>>>>>>>>>
>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
>>>>>>> it is
>>>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
>>>>>>> idea,
>>>>>>>>>>> that'd be cool.
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
>> implementation
>>>>>>>>>>>> attributes. I always wondered about the details field. I think
>>>>>>> we
>>>>>>>>> should
>>>>>>>>>>>> beef up the description in the documentation regarding the
>>>>>>> expected
>>>>>>>>> format
>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>>>> returned on
>>>>>>>>> the
>>>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
>> response.
>>>>>>>> Why
>>>>>>>>>>>> don't we return it? It seems like it would be useful for
>> clients
>>>>>>> to
>>>>>>>> be
>>>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>>>
>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>>>
>>>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
>> Offering
>>>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
>>>>>>>> associate
>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
>>>>>>> it
>>>>>>>>> could
>>>>>>>>>>>> go there?
>>>>>>>>>>>>
>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
>>>>>>> if
>>>>>>>>> it's
>>>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi,
Yeah, Alex, I think that's the way we were planning (with storage tags). I
believe John was just throwing out an idea that - in addition to storage
tags - we could look into these allocators (storage QoS being preferred,
then hypervisor QoS if storage QoS is not available, but hypervisor QoS is).
I think John's concern is that you can enter in values for Wei's and my
feature that are not honored by other vendors (at least yet), so he was
hoping - in addition to storage tags - for the allocators to prefer these
vendors when these fields are filled in. As it stands today in CloudStack,
we already have this kind of an issue with other features (fields in
dialogs for features that not all vendors support). Perhaps post 4.2 we
could look into generic name/value pairs (this is how OpenStack addresses
the issue).
Honestly, I think we're too late in the game (two weeks until code freeze)
to go too deeply down that path in 4.2.
It's probably better if we - at least for 4.2 - keep Wei's fields and my
fields as is, make sure only one or the other feature has data entered for
it (or neither), and call it good for 4.2.
Then let's step back and look into a more general-purpose design that can
be applied throughout CloudStack where we have these situations.
What do you think?
On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I just published my review @ https://reviews.apache.org/r/11479/.
>
> I apologize for the delay,
> -John
>
> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > No problem, John.
> >
> > I still want to re-review it by myself before coming up with a new patch
> > file.
> >
> > Also, maybe I should first wait for Wei's changes to be checked in and
> > merge those into mine before generating a new patch file?
> >
> >
> > On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> I just realized that I forgot to publish my review. I am offline ATM,
> >> but I will publish it in the next couple of hours.
> >>
> >> Do you plan to update your the patch in Review Board?
> >>
> >> Sorry for the oversight,
> >> -John
> >>
> >>
> >>
> >>
> >> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >> <mi...@solidfire.com> wrote:
> >>
> >>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >>>
> >>> Just an FYI that I believe I have implemented all the areas we wanted
> >>> addressed.
> >>>
> >>> I plan to review the code again tomorrow morning or afternoon, then
> send
> >> in
> >>> another patch.
> >>>
> >>> Thanks for all the work on this everyone!
> >>>
> >>>
> >>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> Sure, that sounds good.
> >>>>
> >>>>
> >>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
> >> wrote:
> >>>>
> >>>>> Hi Mike,
> >>>>>
> >>>>> It looks the two feature do not have many conflicts in Java code,
> >> except
> >>>>> the cloudstack UI.
> >>>>> If you do not mind, I will merge disk_io_throttling branch into
> master
> >>>>> this
> >>>>> week, so that you can develop based on it.
> >>>>>
> >>>>> -Wei
> >>>>>
> >>>>>
> >>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>>>
> >>>>>> Hey John,
> >>>>>>
> >>>>>> The SolidFire patch does not depend on the object_store branch, but
> -
> >> as
> >>>>>> Edison mentioned - it might be easier if we merge the SolidFire
> branch
> >>>>> into
> >>>>>> the object_store branch before object_store goes into master.
> >>>>>>
> >>>>>> I'm not sure how the disk_io_throttling fits into this merge
> strategy.
> >>>>>> Perhaps Wei can chime in on that.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
> >>>>>>> solidfire, and object_store appear to have a number of overlapping
> >>>>>>> elements. I understand the dependencies between the patches to be
> as
> >>>>>>> follows:
> >>>>>>>
> >>>>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>>>
> >>>>>>> Am I correct that the device management aspects of SolidFire are
> >>>>> additive
> >>>>>>> to the object_store branch or there are circular dependency between
> >>>>> the
> >>>>>>> branches? Once we understand the dependency graph, we can
> determine
> >>>>> the
> >>>>>>> best approach to land the changes in master.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>>
> >>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Also, if we are good with Edison merging my code into his branch
> >>>>> before
> >>>>>>>> going into master, I am good with that.
> >>>>>>>>
> >>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
> >>>>> we
> >>>>>> can
> >>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>>>
> >>>>>>>>> 1) There should be NO references to hypervisor code in the
> storage
> >>>>>>>>> plug-ins code (this includes the default storage plug-in, which
> >>>>>>> currently
> >>>>>>>>> sends several commands to the hypervisor in use (although it does
> >>>>> not
> >>>>>>> know
> >>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>>>>>>>>
> >>>>>>>>> 2) managed=true or managed=false can be placed in the url field
> (if
> >>>>>> not
> >>>>>>>>> present, we default to false). This info is stored in the
> >>>>>>>>> storage_pool_details table.
> >>>>>>>>>
> >>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>>>> question, we
> >>>>>>>>> pass the managed property along (this takes the place of the
> >>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>
> >>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> >>>>>> managed
> >>>>>>>>> property. If true for an attach, the necessary hypervisor data
> >>>>>>> structure is
> >>>>>>>>> created and the rest of the attach command executes to attach the
> >>>>>>> volume.
> >>>>>>>>>
> >>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a
> volume,
> >>>>>> the
> >>>>>>>>> same check is made. If managed, the hypervisor data structure is
> >>>>>>> removed.
> >>>>>>>>>
> >>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
> >>>>> it is
> >>>>>>>>> stored in the volumes and disk_offerings table. If we have some
> >>>>> idea,
> >>>>>>>>> that'd be cool.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding
> implementation
> >>>>>>>>>> attributes. I always wondered about the details field. I think
> >>>>> we
> >>>>>>> should
> >>>>>>>>>> beef up the description in the documentation regarding the
> >>>>> expected
> >>>>>>> format
> >>>>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>>>> returned on
> >>>>>>> the
> >>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool
> response.
> >>>>>> Why
> >>>>>>>>>> don't we return it? It seems like it would be useful for
> clients
> >>>>> to
> >>>>>> be
> >>>>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>>>
> >>>>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>>>
> >>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk
> Offering
> >>>>>>>>>> basis. For each Disk Offering created, you have to be able to
> >>>>>> associate
> >>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
> >>>>> it
> >>>>>>> could
> >>>>>>>>>> go there?
> >>>>>>>>>>
> >>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
> >>>>> if
> >>>>>>> it's
> >>>>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
I just published my review @ https://reviews.apache.org/r/11479/.
I apologize for the delay,
-John
On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> No problem, John.
>
> I still want to re-review it by myself before coming up with a new patch
> file.
>
> Also, maybe I should first wait for Wei's changes to be checked in and
> merge those into mine before generating a new patch file?
>
>
> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I just realized that I forgot to publish my review. I am offline ATM,
>> but I will publish it in the next couple of hours.
>>
>> Do you plan to update your the patch in Review Board?
>>
>> Sorry for the oversight,
>> -John
>>
>>
>>
>>
>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
>> <mi...@solidfire.com> wrote:
>>
>>> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>>>
>>> Just an FYI that I believe I have implemented all the areas we wanted
>>> addressed.
>>>
>>> I plan to review the code again tomorrow morning or afternoon, then send
>> in
>>> another patch.
>>>
>>> Thanks for all the work on this everyone!
>>>
>>>
>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> Sure, that sounds good.
>>>>
>>>>
>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
>> wrote:
>>>>
>>>>> Hi Mike,
>>>>>
>>>>> It looks the two feature do not have many conflicts in Java code,
>> except
>>>>> the cloudstack UI.
>>>>> If you do not mind, I will merge disk_io_throttling branch into master
>>>>> this
>>>>> week, so that you can develop based on it.
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>>>
>>>>>> Hey John,
>>>>>>
>>>>>> The SolidFire patch does not depend on the object_store branch, but -
>> as
>>>>>> Edison mentioned - it might be easier if we merge the SolidFire branch
>>>>> into
>>>>>> the object_store branch before object_store goes into master.
>>>>>>
>>>>>> I'm not sure how the disk_io_throttling fits into this merge strategy.
>>>>>> Perhaps Wei can chime in on that.
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
>>>>>>> solidfire, and object_store appear to have a number of overlapping
>>>>>>> elements. I understand the dependencies between the patches to be as
>>>>>>> follows:
>>>>>>>
>>>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>>>
>>>>>>> Am I correct that the device management aspects of SolidFire are
>>>>> additive
>>>>>>> to the object_store branch or there are circular dependency between
>>>>> the
>>>>>>> branches? Once we understand the dependency graph, we can determine
>>>>> the
>>>>>>> best approach to land the changes in master.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>>
>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Also, if we are good with Edison merging my code into his branch
>>>>> before
>>>>>>>> going into master, I am good with that.
>>>>>>>>
>>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
>>>>> we
>>>>>> can
>>>>>>>> deal with Burst IOPS then, as well.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>>>
>>>>>>>>> 1) There should be NO references to hypervisor code in the storage
>>>>>>>>> plug-ins code (this includes the default storage plug-in, which
>>>>>>> currently
>>>>>>>>> sends several commands to the hypervisor in use (although it does
>>>>> not
>>>>>>> know
>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>>>>>>
>>>>>>>>> 2) managed=true or managed=false can be placed in the url field (if
>>>>>> not
>>>>>>>>> present, we default to false). This info is stored in the
>>>>>>>>> storage_pool_details table.
>>>>>>>>>
>>>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>>>> question, we
>>>>>>>>> pass the managed property along (this takes the place of the
>>>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>>>
>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
>>>>>> managed
>>>>>>>>> property. If true for an attach, the necessary hypervisor data
>>>>>>> structure is
>>>>>>>>> created and the rest of the attach command executes to attach the
>>>>>>> volume.
>>>>>>>>>
>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
>>>>>> the
>>>>>>>>> same check is made. If managed, the hypervisor data structure is
>>>>>>> removed.
>>>>>>>>>
>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
>>>>> it is
>>>>>>>>> stored in the volumes and disk_offerings table. If we have some
>>>>> idea,
>>>>>>>>> that'd be cool.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>>>>>>>>>> attributes. I always wondered about the details field. I think
>>>>> we
>>>>>>> should
>>>>>>>>>> beef up the description in the documentation regarding the
>>>>> expected
>>>>>>> format
>>>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>>>> returned on
>>>>>>> the
>>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool response.
>>>>>> Why
>>>>>>>>>> don't we return it? It seems like it would be useful for clients
>>>>> to
>>>>>> be
>>>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>>>
>>>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>>>
>>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
>>>>>>>>>> basis. For each Disk Offering created, you have to be able to
>>>>>> associate
>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
>>>>> it
>>>>>>> could
>>>>>>>>>> go there?
>>>>>>>>>>
>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
>>>>> if
>>>>>>> it's
>>>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
No problem, John.
I still want to re-review it by myself before coming up with a new patch
file.
Also, maybe I should first wait for Wei's changes to be checked in and
merge those into mine before generating a new patch file?
On Wed, Jun 12, 2013 at 10:40 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I just realized that I forgot to publish my review. I am offline ATM,
> but I will publish it in the next couple of hours.
>
> Do you plan to update your the patch in Review Board?
>
> Sorry for the oversight,
> -John
>
>
>
>
> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> <mi...@solidfire.com> wrote:
>
> > Hi Edison, John, and Wei (and whoever else is reading this :) ),
> >
> > Just an FYI that I believe I have implemented all the areas we wanted
> > addressed.
> >
> > I plan to review the code again tomorrow morning or afternoon, then send
> in
> > another patch.
> >
> > Thanks for all the work on this everyone!
> >
> >
> > On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> Sure, that sounds good.
> >>
> >>
> >> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com>
> wrote:
> >>
> >>> Hi Mike,
> >>>
> >>> It looks the two feature do not have many conflicts in Java code,
> except
> >>> the cloudstack UI.
> >>> If you do not mind, I will merge disk_io_throttling branch into master
> >>> this
> >>> week, so that you can develop based on it.
> >>>
> >>> -Wei
> >>>
> >>>
> >>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> >>>
> >>>> Hey John,
> >>>>
> >>>> The SolidFire patch does not depend on the object_store branch, but -
> as
> >>>> Edison mentioned - it might be easier if we merge the SolidFire branch
> >>> into
> >>>> the object_store branch before object_store goes into master.
> >>>>
> >>>> I'm not sure how the disk_io_throttling fits into this merge strategy.
> >>>> Perhaps Wei can chime in on that.
> >>>>
> >>>>
> >>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
> >>> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> We have a delicate merge dance to perform. The disk_io_throttling,
> >>>>> solidfire, and object_store appear to have a number of overlapping
> >>>>> elements. I understand the dependencies between the patches to be as
> >>>>> follows:
> >>>>>
> >>>>> object_store <- solidfire -> disk_io_throttling
> >>>>>
> >>>>> Am I correct that the device management aspects of SolidFire are
> >>> additive
> >>>>> to the object_store branch or there are circular dependency between
> >>> the
> >>>>> branches? Once we understand the dependency graph, we can determine
> >>> the
> >>>>> best approach to land the changes in master.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>>
> >>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Also, if we are good with Edison merging my code into his branch
> >>> before
> >>>>>> going into master, I am good with that.
> >>>>>>
> >>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
> >>> we
> >>>> can
> >>>>>> deal with Burst IOPS then, as well.
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Let me make sure I follow where we're going here:
> >>>>>>>
> >>>>>>> 1) There should be NO references to hypervisor code in the storage
> >>>>>>> plug-ins code (this includes the default storage plug-in, which
> >>>>> currently
> >>>>>>> sends several commands to the hypervisor in use (although it does
> >>> not
> >>>>> know
> >>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>>>>>>
> >>>>>>> 2) managed=true or managed=false can be placed in the url field (if
> >>>> not
> >>>>>>> present, we default to false). This info is stored in the
> >>>>>>> storage_pool_details table.
> >>>>>>>
> >>>>>>> 3) When the "attach" command is sent to the hypervisor in
> >>> question, we
> >>>>>>> pass the managed property along (this takes the place of the
> >>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>
> >>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> >>>> managed
> >>>>>>> property. If true for an attach, the necessary hypervisor data
> >>>>> structure is
> >>>>>>> created and the rest of the attach command executes to attach the
> >>>>> volume.
> >>>>>>>
> >>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
> >>>> the
> >>>>>>> same check is made. If managed, the hypervisor data structure is
> >>>>> removed.
> >>>>>>>
> >>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
> >>> it is
> >>>>>>> stored in the volumes and disk_offerings table. If we have some
> >>> idea,
> >>>>>>> that'd be cool.
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding implementation
> >>>>>>>> attributes. I always wondered about the details field. I think
> >>> we
> >>>>> should
> >>>>>>>> beef up the description in the documentation regarding the
> >>> expected
> >>>>> format
> >>>>>>>> of the field. In 4.1, I noticed that the details are not
> >>> returned on
> >>>>> the
> >>>>>>>> createStoratePool updateStoragePool, or listStoragePool response.
> >>>> Why
> >>>>>>>> don't we return it? It seems like it would be useful for clients
> >>> to
> >>>> be
> >>>>>>>> able to inspect the contents of the details field."
> >>>>>>>>
> >>>>>>>> Not sure how this would work storing Burst IOPS here.
> >>>>>>>>
> >>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
> >>>>>>>> basis. For each Disk Offering created, you have to be able to
> >>>> associate
> >>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
> >>> it
> >>>>> could
> >>>>>>>> go there?
> >>>>>>>>
> >>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
> >>> if
> >>>>> it's
> >>>>>>>> not stored like the Min and Max fields are in the DB.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
I just realized that I forgot to publish my review. I am offline ATM,
but I will publish it in the next couple of hours.
Do you plan to update your the patch in Review Board?
Sorry for the oversight,
-John
On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
<mi...@solidfire.com> wrote:
> Hi Edison, John, and Wei (and whoever else is reading this :) ),
>
> Just an FYI that I believe I have implemented all the areas we wanted
> addressed.
>
> I plan to review the code again tomorrow morning or afternoon, then send in
> another patch.
>
> Thanks for all the work on this everyone!
>
>
> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Sure, that sounds good.
>>
>>
>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> Hi Mike,
>>>
>>> It looks the two feature do not have many conflicts in Java code, except
>>> the cloudstack UI.
>>> If you do not mind, I will merge disk_io_throttling branch into master
>>> this
>>> week, so that you can develop based on it.
>>>
>>> -Wei
>>>
>>>
>>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>>
>>>> Hey John,
>>>>
>>>> The SolidFire patch does not depend on the object_store branch, but - as
>>>> Edison mentioned - it might be easier if we merge the SolidFire branch
>>> into
>>>> the object_store branch before object_store goes into master.
>>>>
>>>> I'm not sure how the disk_io_throttling fits into this merge strategy.
>>>> Perhaps Wei can chime in on that.
>>>>
>>>>
>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
>>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> We have a delicate merge dance to perform. The disk_io_throttling,
>>>>> solidfire, and object_store appear to have a number of overlapping
>>>>> elements. I understand the dependencies between the patches to be as
>>>>> follows:
>>>>>
>>>>> object_store <- solidfire -> disk_io_throttling
>>>>>
>>>>> Am I correct that the device management aspects of SolidFire are
>>> additive
>>>>> to the object_store branch or there are circular dependency between
>>> the
>>>>> branches? Once we understand the dependency graph, we can determine
>>> the
>>>>> best approach to land the changes in master.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>>
>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>
>>>>>> Also, if we are good with Edison merging my code into his branch
>>> before
>>>>>> going into master, I am good with that.
>>>>>>
>>>>>> We can remove the StoragePoolType.Dynamic code after his merge and
>>> we
>>>> can
>>>>>> deal with Burst IOPS then, as well.
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> Let me make sure I follow where we're going here:
>>>>>>>
>>>>>>> 1) There should be NO references to hypervisor code in the storage
>>>>>>> plug-ins code (this includes the default storage plug-in, which
>>>>> currently
>>>>>>> sends several commands to the hypervisor in use (although it does
>>> not
>>>>> know
>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>>>>>>
>>>>>>> 2) managed=true or managed=false can be placed in the url field (if
>>>> not
>>>>>>> present, we default to false). This info is stored in the
>>>>>>> storage_pool_details table.
>>>>>>>
>>>>>>> 3) When the "attach" command is sent to the hypervisor in
>>> question, we
>>>>>>> pass the managed property along (this takes the place of the
>>>>>>> StoragePoolType.Dynamic check).
>>>>>>>
>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
>>>> managed
>>>>>>> property. If true for an attach, the necessary hypervisor data
>>>>> structure is
>>>>>>> created and the rest of the attach command executes to attach the
>>>>> volume.
>>>>>>>
>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
>>>> the
>>>>>>> same check is made. If managed, the hypervisor data structure is
>>>>> removed.
>>>>>>>
>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
>>> it is
>>>>>>> stored in the volumes and disk_offerings table. If we have some
>>> idea,
>>>>>>> that'd be cool.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>>>>>>>> attributes. I always wondered about the details field. I think
>>> we
>>>>> should
>>>>>>>> beef up the description in the documentation regarding the
>>> expected
>>>>> format
>>>>>>>> of the field. In 4.1, I noticed that the details are not
>>> returned on
>>>>> the
>>>>>>>> createStoratePool updateStoragePool, or listStoragePool response.
>>>> Why
>>>>>>>> don't we return it? It seems like it would be useful for clients
>>> to
>>>> be
>>>>>>>> able to inspect the contents of the details field."
>>>>>>>>
>>>>>>>> Not sure how this would work storing Burst IOPS here.
>>>>>>>>
>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
>>>>>>>> basis. For each Disk Offering created, you have to be able to
>>>> associate
>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. Maybe
>>> it
>>>>> could
>>>>>>>> go there?
>>>>>>>>
>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the GUI
>>> if
>>>>> it's
>>>>>>>> not stored like the Min and Max fields are in the DB.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi Edison, John, and Wei (and whoever else is reading this :) ),
Just an FYI that I believe I have implemented all the areas we wanted
addressed.
I plan to review the code again tomorrow morning or afternoon, then send in
another patch.
Thanks for all the work on this everyone!
On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Sure, that sounds good.
>
>
> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com> wrote:
>
>> Hi Mike,
>>
>> It looks the two feature do not have many conflicts in Java code, except
>> the cloudstack UI.
>> If you do not mind, I will merge disk_io_throttling branch into master
>> this
>> week, so that you can develop based on it.
>>
>> -Wei
>>
>>
>> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>>
>> > Hey John,
>> >
>> > The SolidFire patch does not depend on the object_store branch, but - as
>> > Edison mentioned - it might be easier if we merge the SolidFire branch
>> into
>> > the object_store branch before object_store goes into master.
>> >
>> > I'm not sure how the disk_io_throttling fits into this merge strategy.
>> > Perhaps Wei can chime in on that.
>> >
>> >
>> > On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> > > Mike,
>> > >
>> > > We have a delicate merge dance to perform. The disk_io_throttling,
>> > > solidfire, and object_store appear to have a number of overlapping
>> > > elements. I understand the dependencies between the patches to be as
>> > > follows:
>> > >
>> > > object_store <- solidfire -> disk_io_throttling
>> > >
>> > > Am I correct that the device management aspects of SolidFire are
>> additive
>> > > to the object_store branch or there are circular dependency between
>> the
>> > > branches? Once we understand the dependency graph, we can determine
>> the
>> > > best approach to land the changes in master.
>> > >
>> > > Thanks,
>> > > -John
>> > >
>> > >
>> > > On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com>
>> > > wrote:
>> > >
>> > > > Also, if we are good with Edison merging my code into his branch
>> before
>> > > > going into master, I am good with that.
>> > > >
>> > > > We can remove the StoragePoolType.Dynamic code after his merge and
>> we
>> > can
>> > > > deal with Burst IOPS then, as well.
>> > > >
>> > > >
>> > > > On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
>> > > > mike.tutkowski@solidfire.com> wrote:
>> > > >
>> > > >> Let me make sure I follow where we're going here:
>> > > >>
>> > > >> 1) There should be NO references to hypervisor code in the storage
>> > > >> plug-ins code (this includes the default storage plug-in, which
>> > > currently
>> > > >> sends several commands to the hypervisor in use (although it does
>> not
>> > > know
>> > > >> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>> > > >>
>> > > >> 2) managed=true or managed=false can be placed in the url field (if
>> > not
>> > > >> present, we default to false). This info is stored in the
>> > > >> storage_pool_details table.
>> > > >>
>> > > >> 3) When the "attach" command is sent to the hypervisor in
>> question, we
>> > > >> pass the managed property along (this takes the place of the
>> > > >> StoragePoolType.Dynamic check).
>> > > >>
>> > > >> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
>> > managed
>> > > >> property. If true for an attach, the necessary hypervisor data
>> > > structure is
>> > > >> created and the rest of the attach command executes to attach the
>> > > volume.
>> > > >>
>> > > >> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
>> > the
>> > > >> same check is made. If managed, the hypervisor data structure is
>> > > removed.
>> > > >>
>> > > >> 6) I do not see an clear way to support Burst IOPS in 4.2 unless
>> it is
>> > > >> stored in the volumes and disk_offerings table. If we have some
>> idea,
>> > > >> that'd be cool.
>> > > >>
>> > > >> Thanks!
>> > > >>
>> > > >>
>> > > >> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>> > > >> mike.tutkowski@solidfire.com> wrote:
>> > > >>
>> > > >>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>> > > >>> attributes. I always wondered about the details field. I think
>> we
>> > > should
>> > > >>> beef up the description in the documentation regarding the
>> expected
>> > > format
>> > > >>> of the field. In 4.1, I noticed that the details are not
>> returned on
>> > > the
>> > > >>> createStoratePool updateStoragePool, or listStoragePool response.
>> > Why
>> > > >>> don't we return it? It seems like it would be useful for clients
>> to
>> > be
>> > > >>> able to inspect the contents of the details field."
>> > > >>>
>> > > >>> Not sure how this would work storing Burst IOPS here.
>> > > >>>
>> > > >>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
>> > > >>> basis. For each Disk Offering created, you have to be able to
>> > associate
>> > > >>> unique Burst IOPS. There is a disk_offering_details table. Maybe
>> it
>> > > could
>> > > >>> go there?
>> > > >>>
>> > > >>> I'm also not sure how you would accept the Burst IOPS in the GUI
>> if
>> > > it's
>> > > >>> not stored like the Min and Max fields are in the DB.
>> > > >>>
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> *Mike Tutkowski*
>> > > >> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >> e: mike.tutkowski@solidfire.com
>> > > >> o: 303.746.7302
>> > > >> Advancing the way the world uses the cloud<
>> > > http://solidfire.com/solution/overview/?video=play>
>> > > >> *â„¢*
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Mike Tutkowski*
>> > > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > > e: mike.tutkowski@solidfire.com
>> > > > o: 303.746.7302
>> > > > Advancing the way the world uses the
>> > > > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > > *â„¢*
>> > >
>> > >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>> >
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Sure, that sounds good.
On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <us...@gmail.com> wrote:
> Hi Mike,
>
> It looks the two feature do not have many conflicts in Java code, except
> the cloudstack UI.
> If you do not mind, I will merge disk_io_throttling branch into master this
> week, so that you can develop based on it.
>
> -Wei
>
>
> 2013/6/11 Mike Tutkowski <mi...@solidfire.com>
>
> > Hey John,
> >
> > The SolidFire patch does not depend on the object_store branch, but - as
> > Edison mentioned - it might be easier if we merge the SolidFire branch
> into
> > the object_store branch before object_store goes into master.
> >
> > I'm not sure how the disk_io_throttling fits into this merge strategy.
> > Perhaps Wei can chime in on that.
> >
> >
> > On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com>
> wrote:
> >
> > > Mike,
> > >
> > > We have a delicate merge dance to perform. The disk_io_throttling,
> > > solidfire, and object_store appear to have a number of overlapping
> > > elements. I understand the dependencies between the patches to be as
> > > follows:
> > >
> > > object_store <- solidfire -> disk_io_throttling
> > >
> > > Am I correct that the device management aspects of SolidFire are
> additive
> > > to the object_store branch or there are circular dependency between the
> > > branches? Once we understand the dependency graph, we can determine
> the
> > > best approach to land the changes in master.
> > >
> > > Thanks,
> > > -John
> > >
> > >
> > > On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > > wrote:
> > >
> > > > Also, if we are good with Edison merging my code into his branch
> before
> > > > going into master, I am good with that.
> > > >
> > > > We can remove the StoragePoolType.Dynamic code after his merge and we
> > can
> > > > deal with Burst IOPS then, as well.
> > > >
> > > >
> > > > On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> > > > mike.tutkowski@solidfire.com> wrote:
> > > >
> > > >> Let me make sure I follow where we're going here:
> > > >>
> > > >> 1) There should be NO references to hypervisor code in the storage
> > > >> plug-ins code (this includes the default storage plug-in, which
> > > currently
> > > >> sends several commands to the hypervisor in use (although it does
> not
> > > know
> > > >> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> > > >>
> > > >> 2) managed=true or managed=false can be placed in the url field (if
> > not
> > > >> present, we default to false). This info is stored in the
> > > >> storage_pool_details table.
> > > >>
> > > >> 3) When the "attach" command is sent to the hypervisor in question,
> we
> > > >> pass the managed property along (this takes the place of the
> > > >> StoragePoolType.Dynamic check).
> > > >>
> > > >> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> > managed
> > > >> property. If true for an attach, the necessary hypervisor data
> > > structure is
> > > >> created and the rest of the attach command executes to attach the
> > > volume.
> > > >>
> > > >> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
> > the
> > > >> same check is made. If managed, the hypervisor data structure is
> > > removed.
> > > >>
> > > >> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it
> is
> > > >> stored in the volumes and disk_offerings table. If we have some
> idea,
> > > >> that'd be cool.
> > > >>
> > > >> Thanks!
> > > >>
> > > >>
> > > >> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> > > >> mike.tutkowski@solidfire.com> wrote:
> > > >>
> > > >>> "+1 -- Burst IOPS can be implemented while avoiding implementation
> > > >>> attributes. I always wondered about the details field. I think we
> > > should
> > > >>> beef up the description in the documentation regarding the expected
> > > format
> > > >>> of the field. In 4.1, I noticed that the details are not returned
> on
> > > the
> > > >>> createStoratePool updateStoragePool, or listStoragePool response.
> > Why
> > > >>> don't we return it? It seems like it would be useful for clients
> to
> > be
> > > >>> able to inspect the contents of the details field."
> > > >>>
> > > >>> Not sure how this would work storing Burst IOPS here.
> > > >>>
> > > >>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
> > > >>> basis. For each Disk Offering created, you have to be able to
> > associate
> > > >>> unique Burst IOPS. There is a disk_offering_details table. Maybe it
> > > could
> > > >>> go there?
> > > >>>
> > > >>> I'm also not sure how you would accept the Burst IOPS in the GUI if
> > > it's
> > > >>> not stored like the Min and Max fields are in the DB.
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> *Mike Tutkowski*
> > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >> e: mike.tutkowski@solidfire.com
> > > >> o: 303.746.7302
> > > >> Advancing the way the world uses the cloud<
> > > http://solidfire.com/solution/overview/?video=play>
> > > >> *â„¢*
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkowski@solidfire.com
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > *â„¢*
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
Hi Mike,
It looks the two feature do not have many conflicts in Java code, except
the cloudstack UI.
If you do not mind, I will merge disk_io_throttling branch into master this
week, so that you can develop based on it.
-Wei
2013/6/11 Mike Tutkowski <mi...@solidfire.com>
> Hey John,
>
> The SolidFire patch does not depend on the object_store branch, but - as
> Edison mentioned - it might be easier if we merge the SolidFire branch into
> the object_store branch before object_store goes into master.
>
> I'm not sure how the disk_io_throttling fits into this merge strategy.
> Perhaps Wei can chime in on that.
>
>
> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com> wrote:
>
> > Mike,
> >
> > We have a delicate merge dance to perform. The disk_io_throttling,
> > solidfire, and object_store appear to have a number of overlapping
> > elements. I understand the dependencies between the patches to be as
> > follows:
> >
> > object_store <- solidfire -> disk_io_throttling
> >
> > Am I correct that the device management aspects of SolidFire are additive
> > to the object_store branch or there are circular dependency between the
> > branches? Once we understand the dependency graph, we can determine the
> > best approach to land the changes in master.
> >
> > Thanks,
> > -John
> >
> >
> > On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> > wrote:
> >
> > > Also, if we are good with Edison merging my code into his branch before
> > > going into master, I am good with that.
> > >
> > > We can remove the StoragePoolType.Dynamic code after his merge and we
> can
> > > deal with Burst IOPS then, as well.
> > >
> > >
> > > On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> > > mike.tutkowski@solidfire.com> wrote:
> > >
> > >> Let me make sure I follow where we're going here:
> > >>
> > >> 1) There should be NO references to hypervisor code in the storage
> > >> plug-ins code (this includes the default storage plug-in, which
> > currently
> > >> sends several commands to the hypervisor in use (although it does not
> > know
> > >> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> > >>
> > >> 2) managed=true or managed=false can be placed in the url field (if
> not
> > >> present, we default to false). This info is stored in the
> > >> storage_pool_details table.
> > >>
> > >> 3) When the "attach" command is sent to the hypervisor in question, we
> > >> pass the managed property along (this takes the place of the
> > >> StoragePoolType.Dynamic check).
> > >>
> > >> 4) execute(AttachVolumeCommand) in the hypervisor checks for the
> managed
> > >> property. If true for an attach, the necessary hypervisor data
> > structure is
> > >> created and the rest of the attach command executes to attach the
> > volume.
> > >>
> > >> 5) When execute(AttachVolumeCommand) is invoked to detach a volume,
> the
> > >> same check is made. If managed, the hypervisor data structure is
> > removed.
> > >>
> > >> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
> > >> stored in the volumes and disk_offerings table. If we have some idea,
> > >> that'd be cool.
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> > >> mike.tutkowski@solidfire.com> wrote:
> > >>
> > >>> "+1 -- Burst IOPS can be implemented while avoiding implementation
> > >>> attributes. I always wondered about the details field. I think we
> > should
> > >>> beef up the description in the documentation regarding the expected
> > format
> > >>> of the field. In 4.1, I noticed that the details are not returned on
> > the
> > >>> createStoratePool updateStoragePool, or listStoragePool response.
> Why
> > >>> don't we return it? It seems like it would be useful for clients to
> be
> > >>> able to inspect the contents of the details field."
> > >>>
> > >>> Not sure how this would work storing Burst IOPS here.
> > >>>
> > >>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
> > >>> basis. For each Disk Offering created, you have to be able to
> associate
> > >>> unique Burst IOPS. There is a disk_offering_details table. Maybe it
> > could
> > >>> go there?
> > >>>
> > >>> I'm also not sure how you would accept the Burst IOPS in the GUI if
> > it's
> > >>> not stored like the Min and Max fields are in the DB.
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> *Mike Tutkowski*
> > >> *Senior CloudStack Developer, SolidFire Inc.*
> > >> e: mike.tutkowski@solidfire.com
> > >> o: 303.746.7302
> > >> Advancing the way the world uses the cloud<
> > http://solidfire.com/solution/overview/?video=play>
> > >> *â„¢*
> > >>
> > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *â„¢*
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hey John,
The SolidFire patch does not depend on the object_store branch, but - as
Edison mentioned - it might be easier if we merge the SolidFire branch into
the object_store branch before object_store goes into master.
I'm not sure how the disk_io_throttling fits into this merge strategy.
Perhaps Wei can chime in on that.
On Tue, Jun 11, 2013 at 11:07 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> We have a delicate merge dance to perform. The disk_io_throttling,
> solidfire, and object_store appear to have a number of overlapping
> elements. I understand the dependencies between the patches to be as
> follows:
>
> object_store <- solidfire -> disk_io_throttling
>
> Am I correct that the device management aspects of SolidFire are additive
> to the object_store branch or there are circular dependency between the
> branches? Once we understand the dependency graph, we can determine the
> best approach to land the changes in master.
>
> Thanks,
> -John
>
>
> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Also, if we are good with Edison merging my code into his branch before
> > going into master, I am good with that.
> >
> > We can remove the StoragePoolType.Dynamic code after his merge and we can
> > deal with Burst IOPS then, as well.
> >
> >
> > On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> Let me make sure I follow where we're going here:
> >>
> >> 1) There should be NO references to hypervisor code in the storage
> >> plug-ins code (this includes the default storage plug-in, which
> currently
> >> sends several commands to the hypervisor in use (although it does not
> know
> >> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
> >>
> >> 2) managed=true or managed=false can be placed in the url field (if not
> >> present, we default to false). This info is stored in the
> >> storage_pool_details table.
> >>
> >> 3) When the "attach" command is sent to the hypervisor in question, we
> >> pass the managed property along (this takes the place of the
> >> StoragePoolType.Dynamic check).
> >>
> >> 4) execute(AttachVolumeCommand) in the hypervisor checks for the managed
> >> property. If true for an attach, the necessary hypervisor data
> structure is
> >> created and the rest of the attach command executes to attach the
> volume.
> >>
> >> 5) When execute(AttachVolumeCommand) is invoked to detach a volume, the
> >> same check is made. If managed, the hypervisor data structure is
> removed.
> >>
> >> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
> >> stored in the volumes and disk_offerings table. If we have some idea,
> >> that'd be cool.
> >>
> >> Thanks!
> >>
> >>
> >> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> "+1 -- Burst IOPS can be implemented while avoiding implementation
> >>> attributes. I always wondered about the details field. I think we
> should
> >>> beef up the description in the documentation regarding the expected
> format
> >>> of the field. In 4.1, I noticed that the details are not returned on
> the
> >>> createStoratePool updateStoragePool, or listStoragePool response. Why
> >>> don't we return it? It seems like it would be useful for clients to be
> >>> able to inspect the contents of the details field."
> >>>
> >>> Not sure how this would work storing Burst IOPS here.
> >>>
> >>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
> >>> basis. For each Disk Offering created, you have to be able to associate
> >>> unique Burst IOPS. There is a disk_offering_details table. Maybe it
> could
> >>> go there?
> >>>
> >>> I'm also not sure how you would accept the Burst IOPS in the GUI if
> it's
> >>> not stored like the Min and Max fields are in the DB.
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
We have a delicate merge dance to perform. The disk_io_throttling, solidfire, and object_store appear to have a number of overlapping elements. I understand the dependencies between the patches to be as follows:
object_store <- solidfire -> disk_io_throttling
Am I correct that the device management aspects of SolidFire are additive to the object_store branch or there are circular dependency between the branches? Once we understand the dependency graph, we can determine the best approach to land the changes in master.
Thanks,
-John
On Jun 10, 2013, at 11:10 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Also, if we are good with Edison merging my code into his branch before
> going into master, I am good with that.
>
> We can remove the StoragePoolType.Dynamic code after his merge and we can
> deal with Burst IOPS then, as well.
>
>
> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Let me make sure I follow where we're going here:
>>
>> 1) There should be NO references to hypervisor code in the storage
>> plug-ins code (this includes the default storage plug-in, which currently
>> sends several commands to the hypervisor in use (although it does not know
>> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>>
>> 2) managed=true or managed=false can be placed in the url field (if not
>> present, we default to false). This info is stored in the
>> storage_pool_details table.
>>
>> 3) When the "attach" command is sent to the hypervisor in question, we
>> pass the managed property along (this takes the place of the
>> StoragePoolType.Dynamic check).
>>
>> 4) execute(AttachVolumeCommand) in the hypervisor checks for the managed
>> property. If true for an attach, the necessary hypervisor data structure is
>> created and the rest of the attach command executes to attach the volume.
>>
>> 5) When execute(AttachVolumeCommand) is invoked to detach a volume, the
>> same check is made. If managed, the hypervisor data structure is removed.
>>
>> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
>> stored in the volumes and disk_offerings table. If we have some idea,
>> that'd be cool.
>>
>> Thanks!
>>
>>
>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>>> attributes. I always wondered about the details field. I think we should
>>> beef up the description in the documentation regarding the expected format
>>> of the field. In 4.1, I noticed that the details are not returned on the
>>> createStoratePool updateStoragePool, or listStoragePool response. Why
>>> don't we return it? It seems like it would be useful for clients to be
>>> able to inspect the contents of the details field."
>>>
>>> Not sure how this would work storing Burst IOPS here.
>>>
>>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
>>> basis. For each Disk Offering created, you have to be able to associate
>>> unique Burst IOPS. There is a disk_offering_details table. Maybe it could
>>> go there?
>>>
>>> I'm also not sure how you would accept the Burst IOPS in the GUI if it's
>>> not stored like the Min and Max fields are in the DB.
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Also, if we are good with Edison merging my code into his branch before
going into master, I am good with that.
We can remove the StoragePoolType.Dynamic code after his merge and we can
deal with Burst IOPS then, as well.
On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Let me make sure I follow where we're going here:
>
> 1) There should be NO references to hypervisor code in the storage
> plug-ins code (this includes the default storage plug-in, which currently
> sends several commands to the hypervisor in use (although it does not know
> which hypervisor (XenServer, ESX(i), etc.) is actually in use))
>
> 2) managed=true or managed=false can be placed in the url field (if not
> present, we default to false). This info is stored in the
> storage_pool_details table.
>
> 3) When the "attach" command is sent to the hypervisor in question, we
> pass the managed property along (this takes the place of the
> StoragePoolType.Dynamic check).
>
> 4) execute(AttachVolumeCommand) in the hypervisor checks for the managed
> property. If true for an attach, the necessary hypervisor data structure is
> created and the rest of the attach command executes to attach the volume.
>
> 5) When execute(AttachVolumeCommand) is invoked to detach a volume, the
> same check is made. If managed, the hypervisor data structure is removed.
>
> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
> stored in the volumes and disk_offerings table. If we have some idea,
> that'd be cool.
>
> Thanks!
>
>
> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>> attributes. I always wondered about the details field. I think we should
>> beef up the description in the documentation regarding the expected format
>> of the field. In 4.1, I noticed that the details are not returned on the
>> createStoratePool updateStoragePool, or listStoragePool response. Why
>> don't we return it? It seems like it would be useful for clients to be
>> able to inspect the contents of the details field."
>>
>> Not sure how this would work storing Burst IOPS here.
>>
>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
>> basis. For each Disk Offering created, you have to be able to associate
>> unique Burst IOPS. There is a disk_offering_details table. Maybe it could
>> go there?
>>
>> I'm also not sure how you would accept the Burst IOPS in the GUI if it's
>> not stored like the Min and Max fields are in the DB.
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
My comments are below in *red*.
Thanks!
On Tue, Jun 11, 2013 at 11:01 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my responses in-line below.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 11:08 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Let me make sure I follow where we're going here:
> >
> > 1) There should be NO references to hypervisor code in the storage
> plug-ins
> > code (this includes the default storage plug-in, which currently sends
> > several commands to the hypervisor in use (although it does not know
> which
> > hypervisor (XenServer, ESX(i), etc.) is actually in use))
>
*Agreed. The default storage plug-in (which has references to hypervisor
code) will be left as is for 4.2. Perhaps this can be addressed in 4.3.*
>
> The Storage->Hypervisor dependencies have been in CloudStack for some
> time. My goal is eventually eliminate these, and as part of that
> evolution, I don't want to see any more such dependencies added.
> Additionally, as we invert the dependency in new code, it will lay the
> foundation for remove the existing Storage->Hypervisor dependencies.
>
> >
> > 2) managed=true or managed=false can be placed in the url field (if not
> > present, we default to false). This info is stored in the
> > storage_pool_details table.
>
> As I understand the data model, storage_pool_details implementation
> specific properties. I see the managed flag as a common value for all
> storage pools calculated as follows:
>
> - If the associated driver does not support device management, the
> value is always set to false.
> - If the associated driver supports device management, the value
> defaults to true, but can be overridden when the device definition is
> created
>
> As such, it seems to be that it should be a new column on the storage_pool
> table.
>
*Let's see here...when you add a new Primary Storage into CloudStack, one
of the new parameters is called "Provider". If you do not set this
parameter, it defaults to Edison's default storage plug-in. If you specify,
say, "SolidFire", then it will be associated with my plug-in. Either way, a
new row is added to the storage_pool table. This table could have a new
column, called "managed", that contains the data we later send to the
hypervisor to let it know if it needs to create, say on Xen, an SR.*
*
*
*Is this what you're thinking, John?*
>
> >
> > 3) When the "attach" command is sent to the hypervisor in question, we
> pass
> > the managed property along (this takes the place of the
> > StoragePoolType.Dynamic check).
> >
> > 4) execute(AttachVolumeCommand) in the hypervisor checks for the managed
> > property. If true for an attach, the necessary hypervisor data structure
> is
> > created and the rest of the attach command executes to attach the volume.
>
> > 5) When execute(AttachVolumeCommand) is invoked to detach a volume, the
> > same check is made. If managed, the hypervisor data structure is removed.
>
> Sounds reasonable to me. Will there is be a new method added to the
> Hypervisor plugin to create this data structure (e.g.
> createVMStorage(VolumeTO))?
*Yeah, I already have the code, I just need to factor it into a new method
that can be called from the "attach/detach" method.*
>
> >
> > 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
> > stored in the volumes and disk_offerings table. If we have some idea,
> > that'd be cool.
>
> Shucks. It sounds like the StoragePool#details won't sufficient. I think
> we need to address extended data in a number of places (e.g. hypervisor,
> storage, and network drivers, compute and disk offerings, etc). I propose
> that we address it broadly in 4.3 in a manner that provides a mechanism to
> store, describe, validate, and render them.
>
*Waiting until 4.3 is fine. I can set the Burst above the Max by a certain
percentage automatically.*
>
> >
> > Thanks!
> >
> >
> > On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> "+1 -- Burst IOPS can be implemented while avoiding implementation
> >> attributes. I always wondered about the details field. I think we
> should
> >> beef up the description in the documentation regarding the expected
> format
> >> of the field. In 4.1, I noticed that the details are not returned on
> the
> >> createStoratePool updateStoragePool, or listStoragePool response. Why
> >> don't we return it? It seems like it would be useful for clients to be
> >> able to inspect the contents of the details field."
> >>
> >> Not sure how this would work storing Burst IOPS here.
> >>
> >> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering
> basis.
> >> For each Disk Offering created, you have to be able to associate unique
> >> Burst IOPS. There is a disk_offering_details table. Maybe it could go
> there?
> >>
> >> I'm also not sure how you would accept the Burst IOPS in the GUI if it's
> >> not stored like the Min and Max fields are in the DB.
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Please see my responses in-line below.
Thanks,
-John
On Jun 10, 2013, at 11:08 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Let me make sure I follow where we're going here:
>
> 1) There should be NO references to hypervisor code in the storage plug-ins
> code (this includes the default storage plug-in, which currently sends
> several commands to the hypervisor in use (although it does not know which
> hypervisor (XenServer, ESX(i), etc.) is actually in use))
The Storage->Hypervisor dependencies have been in CloudStack for some time. My goal is eventually eliminate these, and as part of that evolution, I don't want to see any more such dependencies added. Additionally, as we invert the dependency in new code, it will lay the foundation for remove the existing Storage->Hypervisor dependencies.
>
> 2) managed=true or managed=false can be placed in the url field (if not
> present, we default to false). This info is stored in the
> storage_pool_details table.
As I understand the data model, storage_pool_details implementation specific properties. I see the managed flag as a common value for all storage pools calculated as follows:
- If the associated driver does not support device management, the value is always set to false.
- If the associated driver supports device management, the value defaults to true, but can be overridden when the device definition is created
As such, it seems to be that it should be a new column on the storage_pool table.
>
> 3) When the "attach" command is sent to the hypervisor in question, we pass
> the managed property along (this takes the place of the
> StoragePoolType.Dynamic check).
>
> 4) execute(AttachVolumeCommand) in the hypervisor checks for the managed
> property. If true for an attach, the necessary hypervisor data structure is
> created and the rest of the attach command executes to attach the volume.
> 5) When execute(AttachVolumeCommand) is invoked to detach a volume, the
> same check is made. If managed, the hypervisor data structure is removed.
Sounds reasonable to me. Will there is be a new method added to the Hypervisor plugin to create this data structure (e.g. createVMStorage(VolumeTO))?
>
> 6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
> stored in the volumes and disk_offerings table. If we have some idea,
> that'd be cool.
Shucks. It sounds like the StoragePool#details won't sufficient. I think we need to address extended data in a number of places (e.g. hypervisor, storage, and network drivers, compute and disk offerings, etc). I propose that we address it broadly in 4.3 in a manner that provides a mechanism to store, describe, validate, and render them.
>
> Thanks!
>
>
> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> "+1 -- Burst IOPS can be implemented while avoiding implementation
>> attributes. I always wondered about the details field. I think we should
>> beef up the description in the documentation regarding the expected format
>> of the field. In 4.1, I noticed that the details are not returned on the
>> createStoratePool updateStoragePool, or listStoragePool response. Why
>> don't we return it? It seems like it would be useful for clients to be
>> able to inspect the contents of the details field."
>>
>> Not sure how this would work storing Burst IOPS here.
>>
>> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering basis.
>> For each Disk Offering created, you have to be able to associate unique
>> Burst IOPS. There is a disk_offering_details table. Maybe it could go there?
>>
>> I'm also not sure how you would accept the Burst IOPS in the GUI if it's
>> not stored like the Min and Max fields are in the DB.
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Let me make sure I follow where we're going here:
1) There should be NO references to hypervisor code in the storage plug-ins
code (this includes the default storage plug-in, which currently sends
several commands to the hypervisor in use (although it does not know which
hypervisor (XenServer, ESX(i), etc.) is actually in use))
2) managed=true or managed=false can be placed in the url field (if not
present, we default to false). This info is stored in the
storage_pool_details table.
3) When the "attach" command is sent to the hypervisor in question, we pass
the managed property along (this takes the place of the
StoragePoolType.Dynamic check).
4) execute(AttachVolumeCommand) in the hypervisor checks for the managed
property. If true for an attach, the necessary hypervisor data structure is
created and the rest of the attach command executes to attach the volume.
5) When execute(AttachVolumeCommand) is invoked to detach a volume, the
same check is made. If managed, the hypervisor data structure is removed.
6) I do not see an clear way to support Burst IOPS in 4.2 unless it is
stored in the volumes and disk_offerings table. If we have some idea,
that'd be cool.
Thanks!
On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> "+1 -- Burst IOPS can be implemented while avoiding implementation
> attributes. I always wondered about the details field. I think we should
> beef up the description in the documentation regarding the expected format
> of the field. In 4.1, I noticed that the details are not returned on the
> createStoratePool updateStoragePool, or listStoragePool response. Why
> don't we return it? It seems like it would be useful for clients to be
> able to inspect the contents of the details field."
>
> Not sure how this would work storing Burst IOPS here.
>
> Burst IOPS need to be variable on a Disk Offering-by-Disk Offering basis.
> For each Disk Offering created, you have to be able to associate unique
> Burst IOPS. There is a disk_offering_details table. Maybe it could go there?
>
> I'm also not sure how you would accept the Burst IOPS in the GUI if it's
> not stored like the Min and Max fields are in the DB.
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"+1 -- Burst IOPS can be implemented while avoiding implementation
attributes. I always wondered about the details field. I think we should
beef up the description in the documentation regarding the expected format
of the field. In 4.1, I noticed that the details are not returned on the
createStoratePool updateStoragePool, or listStoragePool response. Why
don't we return it? It seems like it would be useful for clients to be
able to inspect the contents of the details field."
Not sure how this would work storing Burst IOPS here.
Burst IOPS need to be variable on a Disk Offering-by-Disk Offering basis.
For each Disk Offering created, you have to be able to associate unique
Burst IOPS. There is a disk_offering_details table. Maybe it could go there?
I'm also not sure how you would accept the Burst IOPS in the GUI if it's
not stored like the Min and Max fields are in the DB.
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Edison,
Please see comment in-line below.
Thanks,
-John
On Jun 10, 2013, at 7:47 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: John Burwell [mailto:jburwell@basho.com]
>> Sent: Monday, June 10, 2013 4:27 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>> Edison,
>>
>> Please see my responses in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 6:52 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>> Sent: Monday, June 10, 2013 2:44 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>
>>>> Edison,
>>>>
>>>> Please see my response in-line below.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 10, 2013, at 4:45 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>>>> Sent: Monday, June 10, 2013 1:25 PM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Yes, a vendor may hand a non-managed plugin. For example, on the
>>>>>> secondary storage side, S3 and Swift don't support the concept of
>> being
>>>>>> managed -- an operator is required to create a bucket and point
>>>> CloudStack
>>>>>> at it. Another example would be a generic iSCSI device (e.g. OpenFiler,
>>>>>> FreeNAS) which CloudStack can not directly manage, but can consume
>> a
>>>> LUN
>>>>>> that an operator has created for it.
>>>>>
>>>>> Manage or non-managed storage is handled by DataStoreLifeCyclce.
>>>>> It depends on storage vendor to implement DataStoreLifeCyclce, which
>>>> basically says how to initialize a storage, how to add a storage into a
>> cluster
>>>> etc.
>>>>
>>>> So, implementation of the DataStoreLifeCycle interface indicates that
>> driver
>>>> is capable of managing the device. It does not answer the question of
>>>
>>> No, the interface only means there is a way to add a storage into cloudstack,
>> no matter
>>> It's managed or un-managed.
>>>
>>>> whether or not the administrator/operator wants CloudStack to manage
>> the
>>>> device. Hence, the need for a flag on the StoragePool to indicate we
>> should
>>> Cloudstack don't need to care about the storage is managed or un-
>> managed, what it cares about
>>> Is how to create/delete something on it, it will call storage plugins to do all
>> the storage related operations.
>>>
>>>
>>>> be managing it. I am cool with the flag defaulting to true when the driver
>>>> supports it, but we must provide the administrator/operator to ability to
>> opt-
>>>
>>> If the storage provider wants to opt-out or wants to do things differently
>> based on input from administrator/operator,
>>> The storage provider can add parameters in createstoragepoolcmd->details,
>> details is a map, which can contain whatever information needed by the
>> storage provider.
>>>
>>> Let's say a storage provider which can support both managed/un-managed
>> storage pool, when admin wants a storage pool managed by the storage pool,
>> admin can set "managed" = "true" in the details, then
>>> The storage provider will know, oh, admin wants a managed storage pool,
>> so the DataStoreLifeCyclce->init will create a storage pool dynamically on the
>> storage box. While if the "managed" = "false", then DataStoreLifeCyclce->init
>> will not create a storage pool, instead, it will check the existence of the
>> storage pool, if it's there, then return OK.
>>>
>>> I don't think cloudstack mgt server needs to know, the storage pool is
>> managed/unmanaged, it's up to the implementation of storage provider.
>>
>> I think we are speaking past each other. I am concerned with the policy the
>> operator/administrator intends the implement not the overall capabilities of
>> the underlying driver. A driver may be capable of features that an
>
> The admin/operator needs to know what he/she is doing, when adding a storage pool into cloudstack.
> Each storage provider has its own way of configure a storage pool, cloudstack doesn't need to know and care about how to do it.
> It's a contract between storage provider and admin/operator, cloudstack just helps storage provider and admin to establish this contract.
I agree that the administrator/operator needs to know what they are doing. Hence, they should have the option to use a storage driver, and opt-out of having CloudStack attempt to automatically manage a resource. Without this option, a user has to choose between using a particular driver and accepting turning over full control of the device to CloudStack or not using it all. I am simply saying we should provide a middle group that allows the expert operator to use the device and manage it themselves. Functionally, it is one additional attribute on the StoragePool, and querying isManaged on the StoragePool rather than the driver. From a code perspective, the implementation doesn't seem much more difficult (one more attribute on VolumeTO), and we provide our users with more flexibility.
>
>> operator/administrator has elected not to use. The only place to capture
>> that policy is in the management server configuration of the DataStore.
>
> If admin is doing whatever the underlying storage provider doesn't allowed to do, then the storage provider itself will report error, during the createStoragePoolcmd.
>
>>
>>>
>>>> out. Does this interface provide for a set of administrative credentials
>>>> separate from the credentials used to access storage?
>>>
>>> Yes, createstoragepoolcmd->details, is the place to add whatever
>> information when creating a storage pool.
>>
>> What is the format of the details field? The API does just list it as a non-
>> required field in 4.1.
>
> It's a map, it's the place for each storage provider to extend createstoreagepoolcmd api, if the storage provider wants to do that.
> CloudStack mgt server just passes the url/details in createstoragepoolcmd to driver's *lifecycle interface, it's driver's responsibility to how to decode the information passed from UI/admin.
+1 -- Burst IOPS can be implemented while avoiding implementation attributes. I always wondered about the details field. I think we should beef up the description in the documentation regarding the expected format of the field. In 4.1, I noticed that the details are not returned on the createStoratePool updateStoragePool, or listStoragePool response. Why don't we return it? It seems like it would be useful for clients to be able to inspect the contents of the details field.
>
>>
>>>
>>>>
>>>> Another question comes to mind -- why do we have a Dynamic
>>>> StoragePoolType when this interface exists?
>>>
>>> It shouldn't.
>>> Storage provider can return a VolumeTO which can contain whatever
>> information about the volume.
>>> At the resource side, it may check VolumeTO, if the volume is not a VDI,
>> VolumeTO can contain a field called "isManaged", which can be set to true by
>> Solidfire's driver, while by default it's false.
>>> If isManaged is true, then the resource code can create a SR and create a
>> VDI on the fly.
>>
>> My point is that nothing in the Storage layer should know out an SR or VDI.
> Yes, it shouldn't know.
>
>> These are hypervisor concepts -- specifically Xen. The Storage layer should
>> only be exposing the discrete operations necessary for the hypervisor to
>> determine if an SR and/or VDI are present, as well as, the operations to
>> create the structures for them. Fundamentally, it's this circular dependency
>> between the Hypervisor and Storage layer that I believe we need to
>> eliminate. It is one of the sources of the feature inconsistency across
>> hypervisors. For SolidFire, the "managed" flag indicates that the driver
>> should programmatically allocate a LUN with a particular IQN -- nothing else.
>> The Xen hypervisor plugin should have the knowledge that iSCSI volumes
>> need to have an SR created in a particular form on an iSCSI device.
> Yes, I think we agree on it.
> In CreateAsync, solidfire just creates a LUN.
> In VolumeManagerImpl->AttachVolume:
> Mgt server will ask solidfire;s driver to return a VolumeTO about the volume,
> VolumeTO has a field, called, "isManged", Solidfire's driver needs to set it as true.
>
> On the hypervisor resource code, execute(AttachVolumeCmd) has the following code:
>
> If (volumeTO.isManaged()) {
> Get IQN, create SR, create VDI
> } {
> vid = vid.finduuid()
> }
>
> Above code is generic for all the storage vendor, as long as they use the VolumeTO.
+10. This code is exactly what I have in mind. The Xen plugin understands that it needs an SR and a VDI. Though, as I understand it, the SR and VDI are created regardless of whether or not the Volume is managed. As I understand it from Mike's patch, the logic for Xen is roughly:
if (aVolume.isManaged()) {
// Allocate the underlying storage for the volume ...
}
if (aVolume.getType() == ISCSI && /* the volume if does not have an SR */) {
// Create SR and VDI
}
Mike's thought was that we could push the allocate step down into the attach method within the Storage layer. The part that we need to flesh out are the method(s) on the volume that allow a client (in this case the Xen plugin) to query the contents of the volume to determine whether or the SR/VDI are present. Here's my straw man -- DataStore#hassDataObject(URI) : boolean.
>
>>
>>>
>>>>
>>>>>
>>>>> Take S3 as an example, if the S3 storage provider provides a
>>>> DataStoreLifeCyclce which can create a bucket, then when admin adding a
>> S3
>>>> storage through API, mgt server will first call DataStoreLifeCyclce, which
>> will
>>>> create a bucket, then stored in the image_store_details table, then later
>> on,
>>>> all the referenced to this image store, will contain this bucket information.
>>>>
>>>> From an S3 perspective, creating a bucket from CloudStack would not a be
>>>> security best practice. The general guidance would be for the bucket to
>>>> created and associated to a dedicated CloudStack access/secret key pair
>> to
>>>> properly isolate data. Given this best practice and that only one bucket
>> per
>>>> S3 instance would be necessary for CloudStack operation, I think most
>>>> administrators/operators would want to opt-out CloudStack managing S3
>>>> buckets.
>>>>
>>>>>
>>>>> Take an storage box from some vendor as an example, admin wants to
>>>> create an ISCSI server on a storage box, which can be used by a
>>>> cluster(multiple volumes on a giant LUN ), as long as the storage provider
>> can
>>>> provide a DataStoreLifeCyclce which can program the storage box, and
>> create
>>>> an ISCSI server, then stored the information in storage_pool table, then
>> all
>>>> the referenced to the storage pool will use the dynamically created ISCSI
>> LUN.
>>>>>
>>>>> Take another storage box from some vendor as an example, admin
>> wants
>>>> to add the whole storage box into cloudstack, which can be used as per
>> LUN
>>>> per ISCSI, so in this case, DataStoreLifeCyclce won't create an ISCSI server
>>>> during the initialize method call. Instead, for every createAsync, it will
>> create
>>>> an ISCSI LUN, stored the LUN information in volume table and
>> volume_detail
>>>> table. Whenever the LUN is used by hypervisor, the hypervisor needs to
>> take
>>>> extra care about the volume.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> From an operational perspective, an administrator may elect to say,
>> "Hey,
>>>> I
>>>>>> know you can manage that device for me, but it is has a wider use than
>>>>>> CloudStack can manage. Let me handle it, and just provide the
>>>> information
>>>>>> CloudStack needs." In this scenario, the operator is accepting a greater
>>>>>> management burden in exchange for control. I don't think many will
>> elect
>>>> to
>>>>>> manage these devices themselves, but this ability will be extremely
>>>>>> important for the edge cases that require it. Speaking as someone who
>>>> has
>>>>>> run cloud operations, I can testify to this need -- nothing aggravates
>> you
>>>>>> more than a system insisting on managing a device or service without
>> the
>>>>>> opportunity to opt-out
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 10, 2013, at 3:19 PM, Mike Tutkowski
>>>> <mi...@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> So, would a storage vendor write a non-CloudStack-managed plug-in
>> and
>>>>>> maybe
>>>>>>> just implement the snapshot feature or something?
>>>>>>>
>>>>>>> In this case, they'd have to set the hypervisor up ahead of time with a
>>>>>>> large portion of their storage, but they'd still get the vendor-snapshot
>>>>>>> ability.
>>>>>>>
>>>>>>> Just trying to undertand this more.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Edison,
>>>>>>>>
>>>>>>>> TL;DR Always give operators/system administrators the ability to
>>>> manage
>>>>>>>> things themselves, if they like.
>>>>>>>>
>>>>>>>> As more manageable drivers emerge, I think it will be import to allow
>>>>>>>> operators to opt-out of CloudStack device management. Therefore,
>> a
>>>>>> driver
>>>>>>>> declares whether or not it can support device management (many,
>> in
>>>> fact
>>>>>>>> most, won't). For those drivers that do support management, allow
>>>>>>>> operators to opt-in/out of CloudStack management on a per device
>>>> basis.
>>>>>> To
>>>>>>>> support this notion, we need a flag on the storage_pool table and
>>>>>>>> associated classes to indicate whether or not the operator wants
>>>>>> CloudStack
>>>>>>>> to manage the device.
>>>>>>>>
>>>>>>>> Does this explanation make more sense?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>>>>>
>>>>>>>>> I don't understand why we need to add a "managed" in the UI and
>> in
>>>>>>>> "storage_pool" table?
>>>>>>>>> AFAIK, what you guys trying to do is to enable
>>>>>>>> attachVolume/DettachVolume if the volume is created on Solidfire,
>>>> right?
>>>>>>>>> If the volume is created on Solidfire already, by createAsync, then
>>>> why
>>>>>>>> we can't just send out all the information about the volume(it's
>>>> volumeTO
>>>>>>>> which will be returned by each storage driver) to hypervisor, then at
>>>> the
>>>>>>>> hypervisor side, check the volumeTO, if the volumeTO is managed,
>>>> then
>>>>>> doing
>>>>>>>> whatever necessary operations(create SR, create VDI etc), then
>> attach
>>>>>> the
>>>>>>>> VDI to volume.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>> Sent: Monday, June 10, 2013 11:10 AM
>>>>>>>>>> To: John Burwell
>>>>>>>>>> Cc: Edison Su; dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>
>>>>>>>>>> I believe we're on the same page, John.
>>>>>>>>>>
>>>>>>>>>> I would add a new parameter to the createStoragePool API
>>>> command
>>>>>> (let's
>>>>>>>>>> call it "managed" and it can be true or false). If you pass in "true",
>>>>>>>> but
>>>>>>>>>> the driver does not support being managed by CloudStack, an
>>>>>> exception is
>>>>>>>>>> thrown by the driver. If you pass in "false", but the driver does
>> not
>>>>>>>>>> support being managed outside of CloudStack, an exception is
>>>> thrown
>>>>>> by
>>>>>>>> the
>>>>>>>>>> driver. It is possible one could write a driver that can be managed
>>>> both
>>>>>>>>>> inside or outside of CloudStack (the SolidFire driver requires being
>>>>>>>>>> managed only by CloudStack).
>>>>>>>>>>
>>>>>>>>>> This managed value would end up in the storage_pool table in a
>> new
>>>>>>>> column
>>>>>>>>>> ("managed").
>>>>>>>>>>
>>>>>>>>>> An example usage: After the storage framework calls on the
>> driver's
>>>>>>>>>> createAsync method, it will check to see if the driver is managed.
>> If
>>>>>>>> the
>>>>>>>>>> driver is managed, the storage framework will invoke the proper
>>>> (new)
>>>>>>>>>> command of the hypervisor in use to set up, say, the SR for the
>> SAN
>>>>>>>> volume.
>>>>>>>>>>
>>>>>>>>>> There may be other cases when the isManaged() method is
>> invoked
>>>> on
>>>>>> the
>>>>>>>>>> driver and - if true is returned - the storage framework sends a
>>>>>>>> particular
>>>>>>>>>> command to the hypervisor in use.
>>>>>>>>>>
>>>>>>>>>> ********************
>>>>>>>>>>
>>>>>>>>>> As far as reworking the default storage plug-in (which sends
>>>> commands
>>>>>> to
>>>>>>>>>> hypervisors) for 4.2, I would check with Edison on his schedule.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell
>>>> <jb...@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> I want to make sure we are on the same page regarding
>> isManaged.
>>>> I
>>>>>> see
>>>>>>>>>>> the addition of the following operations:
>>>>>>>>>>>
>>>>>>>>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates
>>>>>> whether
>>>>>>>>>> or
>>>>>>>>>>> not the driver support/implements the management functions
>>>>>>>>>>> * DataStore#isManaged() : boolean -- indicates whether or not a
>>>>>> device
>>>>>>>> is
>>>>>>>>>>> managed by CloudStack
>>>>>>>>>>>
>>>>>>>>>>> Only DataStores associated with DataStoreDrivers that support
>>>>>>>>>> management
>>>>>>>>>>> would be able to have the management flag enabled. This
>>>> behavior
>>>>>>>> allows
>>>>>>>>>>> operators to declare their intention to allow CloudStack to
>> manage
>>>> a
>>>>>>>>>>> device. This behavior would need to be exposed as flag on the
>>>> HTTP
>>>>>>>> API.
>>>>>>>>>>> Additionally, this flag would only be settable when a device is
>>>>>>>>>>> defined/created in the system (i.e. once a device is managed by
>>>>>>>> CloudStack,
>>>>>>>>>>> you can't unmanaged it without dissociating it from CloudStack).
>> In
>>>>>>>> the
>>>>>>>>>>> future, we may consider relaxing that rule, but, in the short term,
>> it
>>>>>>>>>>> greatly decreases the alternative flows.
>>>>>>>>>>>
>>>>>>>>>>> In terms of removing the dependencies from Storage to
>> Hypervisor,
>>>> I
>>>>>>>>>> would
>>>>>>>>>>> really like to see this accomplished in 4.2. It will be much more
>>>>>>>>>>> difficult to address this issue post 4.2 because we will have more
>>>>>>>> plugins
>>>>>>>>>>> to address. I also see this issue as a critical architectural issues
>>>>>>>> that
>>>>>>>>>>> for the management server. I recommend we identify the
>>>>>> dependencies
>>>>>>>>>> that
>>>>>>>>>>> need to be inverted, and see if we can divide the work across a
>> few
>>>>>>>> more
>>>>>>>>>>> resources.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
>>>>>>>>>> <mi...@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Sounds good, Edison :)
>>>>>>>>>>>
>>>>>>>>>>> I can implement John's isManaged() logic and resubmit my code
>> for
>>>>>>>> review.
>>>>>>>>>>>
>>>>>>>>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
>>>>>>>> requires
>>>>>>>>>>> some refactoring of the storage framework, then I could do that.
>>>>>>>>>>>
>>>>>>>>>>> Also, the default storage plug-in does have hypervisor logic in it
>> in
>>>>>>>> 4.2.
>>>>>>>>>>> I recommend we leave this as is for 4.2 and it can be refactored
>> in
>>>> 4.3
>>>>>>>>>>> with the above work I mentioned.
>>>>>>>>>>>
>>>>>>>>>>> Are you OK with that, guys?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su
>> <Ed...@citrix.com>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
>>>>>>>>>>>> *To:* dev@cloudstack.apache.org
>>>>>>>>>>>> *Cc:* Edison Su; John Burwell
>>>>>>>>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> Another thing we might want to consider for 4.2 is that the way
>>>> the
>>>>>>>> code
>>>>>>>>>>>> is currently implemented in master, I don't think it's even
>> possible
>>>>>>>> to
>>>>>>>>>>>> write a plug-in that connects storage directly to a VM because
>> the
>>>>>>>>>>>> CloudStack storage framework will call the "attach" method of
>> the
>>>>>>>>>>>> hypervisor in use and this method will fail because it assumes
>>>> there
>>>>>>>> is
>>>>>>>>>>>> (talking Xen here) an SR existent and in this situation there isn't
>>>>>>>> one.*
>>>>>>>>>>>> ***
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> My thinking is that we should ignore the direct-attach-of-
>> volume-
>>>> to-
>>>>>> VM
>>>>>>>>>>>> use case for 4.2. I think it will require too much of a rewrite at
>>>>>>>> this
>>>>>>>>>>>> point in development (fewer than three weeks left until code
>>>>>>>>>> freeze).****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> If the plug-in implements an isManaged() method (as John
>>>>>> suggested),
>>>>>>>>>> the
>>>>>>>>>>>> storage framework can invoke the necessary commands on the
>>>>>>>>>> hypervisor in
>>>>>>>>>>>> use.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> For example, the createAsync method is called. The storage
>> plug-
>>>> in
>>>>>>>>>>>> creates a SAN volume and updates the CS DB. The storage
>>>>>> framework
>>>>>>>>>> then asks
>>>>>>>>>>>> the driver if it's managed. If it is, the storage framework
>> executes
>>>> a
>>>>>>>>>>>> command against the hypervisor in use to have it create, say,
>> an
>>>> SR
>>>>>>>> (with
>>>>>>>>>> a
>>>>>>>>>>>> single VDI that takes up the entire SR). Now, when the storage
>>>>>>>>>> framework
>>>>>>>>>>>> executes the "attach" command against the hypervisor, this
>>>>>> command
>>>>>>>> will
>>>>>>>>>>>> work, as well (because that logic assumes the prior existence of
>>>> an
>>>>>>>> SR and
>>>>>>>>>>>> it will, in fact, be present).****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:****
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Edison and John,****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> I wanted to point out something I thought was relevant to this
>>>>>>>>>>>> conversation.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> My plug-in supports both XenServer and ESX(i).****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> These two hypervisors handle their respective hypervisor data
>>>>>>>> structures
>>>>>>>>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> In XenServer, the SR that resides on the SAN LUN is simply
>>>>>> "forgotten"
>>>>>>>>>>>> when you issue a remove-SR command to XenServer.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> When you later want to access the contents of that data, it is
>> still
>>>>>>>>>>>> safely stored on the LUN and can be accessed from XenServer
>>>> (from
>>>>>> the
>>>>>>>>>> same
>>>>>>>>>>>> or different cluster (what XenServer calls a resource pool)) by
>>>>>>>> creating a
>>>>>>>>>>>> new SR that is based on the same IQN and LUN.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> XenServer SRs exist at the cluster level.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
>>>>>>>>>>>> destroyed when you issue a remove-datastore command to
>>>> ESX(i).
>>>>>> In
>>>>>>>>>> other
>>>>>>>>>>>> words, the contents of the SAN LUN are destroyed.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> ESX(i) datastores exist at the datacenter level.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> The XenServer behavior (not destroying the contents of the
>> LUN
>>>>>> when
>>>>>>>>>> the
>>>>>>>>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
>>>>>>>> contents
>>>>>>>>>>>> of the LUN when the datastore is removed) is not
>> desirable.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> I can get around the ESX(i) behavior by removing the
>> referenced
>>>> IQN
>>>>>>>>>> from
>>>>>>>>>>>> each host in the cluster when the CS storage framework
>> notifies
>>>> my
>>>>>>>> plug-
>>>>>>>>>> in
>>>>>>>>>>>> of a detach-volume event. When the CS volume is later
>> attached
>>>>>> again
>>>>>>>>>>>> (either to the same or a different ESX(i) cluster), I can have
>> each
>>>>>>>> host in
>>>>>>>>>>>> the relevant ESX(i) cluster reference the IQN and the datastore
>>>> will
>>>>>>>> be
>>>>>>>>>>>> back again.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> This sequence of events is critical for my plug-in to support
>> both
>>>>>>>>>>>> environments:****
>>>>>>>>>>>>
>>>>>>>>>>>> *XenServer flow (events for Driver class):*****
>>>>>>>>>>>>
>>>>>>>>>>>> createAsync (called right before first attach) = creates SAN
>>>>>>>> volume****
>>>>>>>>>>>>
>>>>>>>>>>>> preAttachVolume = creates SR and creates VDI or just
>> introduces
>>>>>> SR****
>>>>>>>>>>>>
>>>>>>>>>>>> (CS volume gets attached and later detached)****
>>>>>>>>>>>>
>>>>>>>>>>>> postDetachVolume = removes SR (this does not delete data on
>>>> SAN
>>>>>>>>>> volume)**
>>>>>>>>>>>> **
>>>>>>>>>>>>
>>>>>>>>>>>> deleteAsync (can be called when volume is not attached) =
>>>> deletes
>>>>>> the
>>>>>>>>>> SAN
>>>>>>>>>>>> volume****
>>>>>>>>>>>>
>>>>>>>>>>>> *VMware flow (events for Driver class):*****
>>>>>>>>>>>>
>>>>>>>>>>>> createAsync (called right before first attach) = creates SAN
>>>> volume,
>>>>>>>>>>>> creates datastore, and creates VMDK file****
>>>>>>>>>>>>
>>>>>>>>>>>> preAttachVolume = adds iSCSI target to each host in
>> cluster****
>>>>>>>>>>>>
>>>>>>>>>>>> (CS volume gets attached and later detached)****
>>>>>>>>>>>>
>>>>>>>>>>>> postDetachVolume = removes iSCSI target from each host in
>>>>>> cluster****
>>>>>>>>>>>>
>>>>>>>>>>>> deleteAsync (can be called when volume is not attached) =
>>>> deletes
>>>>>>>>>>>> datastore and deletes the SAN volume****
>>>>>>>>>>>>
>>>>>>>>>>>> ************************
>>>>>>>>>>>>
>>>>>>>>>>>> So, the way I have it written up here, the driver is talking to the
>>>>>>>>>>>> hypervisor. As John has pointed out, this may not be ideal.****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> Whatever we decide on, it needs to be able to support this kind
>> of
>>>>>>>> flow.
>>>>>>>>>>>> :)****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> We could go John's route, where the driver says if it's managed
>> or
>>>>>>>> not.**
>>>>>>>>>>>> **
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> If it is, then the storage framework could send the necessary
>>>>>>>> messages to
>>>>>>>>>>>> the hypervisor in question at the right times (just migrate the
>>>>>>>> hypervisor
>>>>>>>>>>>> logic above from the driver to the right parts of the storage
>>>>>>>> framework).
>>>>>>>>>>>> ****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> This is fine, except that it does not resolve Edison's use case of
>>>>>>>>>>>> desiring to have the storage LUN potentially connected directly
>> to
>>>> a
>>>>>>>> VM.*
>>>>>>>>>>>> ***
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not sure we should design around that use case, though. Is
>> it
>>>>>>>> common
>>>>>>>>>>>> enough? Has anyone asked for it? Is it even desirable? How are
>>>>>> usage
>>>>>>>>>>>> statistics gathered for such volumes? It's nice to offer the
>>>>>>>> flexibility,
>>>>>>>>>>>> but is it a good idea in this case?****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
>>>>>>>> wrote:**
>>>>>>>>>>>> **
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>> Sent: Friday, June 07, 2013 4:08 PM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, you are correct that Xen must layer an SR on top of the
>> iSCSI
>>>>>>>>>>>> volume
>>>>>>>>>>>>> to use it. Same story for VMware: ESX(i) must layer a
>> datastore
>>>> on
>>>>>>>> top
>>>>>>>>>>>> of
>>>>>>>>>>>>> the iSCSI volume to use it. I look at it like they are layering a
>>>>>>>>>>>> clustered
>>>>>>>>>>>>> file system on the SAN volume so the hypervisors can share
>>>> access
>>>>>> to
>>>>>>>>>> the
>>>>>>>>>>>>> contents of the volume.
>>>>>>>>>>>>>
>>>>>>>>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called,
>>>> the
>>>>>>>>>>>> SR or
>>>>>>>>>>>>> datastore has already been created (usually manually by an
>>>> admin).
>>>>>>>>>>>>>
>>>>>>>>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in
>> an
>>>>>>>>>>>>> environment where each CloudStack volume that a user
>> creates
>>>> is
>>>>>>>>>> mapped
>>>>>>>>>>>>> to a
>>>>>>>>>>>>> single SAN volume (via the SR).
>>>>>>>>>>>>>
>>>>>>>>>>>>> The question comes down to who should allocate the SR
>>>>>> dynamically.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We could have the storage framework ask the storage plug-in
>> if
>>>> it is
>>>>>>>>>>>>> managed. If it is, then the storage framework could send a
>>>>>> message to
>>>>>>>>>>>> the
>>>>>>>>>>>>> hypervisor in question to create (let's talk Xen here) the SR
>>>> ahead
>>>>>>>> of
>>>>>>>>>>>>> time. Then, when the storage framework next sends the
>> attach-
>>>>>> volume
>>>>>>>>>>>>> command
>>>>>>>>>>>>> to the hypervisor, it should work without changes to that
>>>>>>>> attach-volume
>>>>>>>>>>>>> logic (because - from the point of view of the attach logic - the
>> SR
>>>>>>>> is
>>>>>>>>>>>>> already existent, as expected).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Now, as Edison has pointed out, this limits the power of the
>>>> storage
>>>>>>>>>>>>> plug-in. A storage plug-in in this model cannot directly attach
>>>>>>>> storage
>>>>>>>>>>>> to
>>>>>>>>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK.
>> We
>>>>>> need
>>>>>>>>>> to
>>>>>>>>>>>>> make a call on that.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, I think we need to address this usage case, and won't limit
>>>> the
>>>>>>>>>>>> power of the storage plug-in.
>>>>>>>>>>>> John, what's your idea to fit this usage case into your
>> abstraction?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> If we want to give more power to the storage vendor, we
>> would
>>>>>> have to
>>>>>>>>>>>>> have
>>>>>>>>>>>>> the storage framework call into the storage plug-in to send
>> the
>>>>>>>>>>>> appropriate
>>>>>>>>>>>>> attach commands to the hypervisor.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's discuss and come to a consensus (or at least agree on a
>>>> path)
>>>>>>>> as
>>>>>>>>>>>> soon
>>>>>>>>>>>>> as we can. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell
>>>> <jb...@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My understanding is the Xen expects an SR structure on all
>>>> ISCSI
>>>>>>>>>>>> devices
>>>>>>>>>>>>>> -- at least that is how I read the code in your patch. Is my
>>>>>>>>>>>> understanding
>>>>>>>>>>>>>> correct? If so, the Xen plugin should be able to query the
>>>> storage
>>>>>>>>>>>> device
>>>>>>>>>>>>>> to determine the presence of the SR structure and create it
>> if it
>>>>>>>>>>>> does not
>>>>>>>>>>>>>> exist. Am I missing something in the implementation that
>>>> makes
>>>>>> that
>>>>>>>>>>>> type
>>>>>>>>>>>>>> of implementation impossible?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, if a storage vendor wanted to attach a volume
>> directly
>>>> to a
>>>>>>>>>>>> VM in
>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>> today, it would probably fail because the attach-volume
>> logic
>>>>>>>>>>>> assumes
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> existence of the necessary hypervisor data structure (ex.
>> SR
>>>> on
>>>>>>>>>>>> Xen).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If we wanted to enable such an attach, we could do it the
>> way
>>>>>>>>>> Edison
>>>>>>>>>>>>>>> suggests.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su
>>>> <Ed...@citrix.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>> From: Mike Tutkowski
>>>> [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As we only have three weeks until feature freeze, we
>>>> should
>>>>>>>>>> come
>>>>>>>>>>>> to
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> consensus on this design point as soon as possible.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Right now, if the storage framework asks my driver it is is
>>>>>>>>>>>> managed, it
>>>>>>>>>>>>>>>>> will say 'yes.' This means the framework will tell the
>> driver
>>>> to
>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>> its management activities. This then means the driver
>> will
>>>> call
>>>>>>>>>>>> into
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to
>>>>>> perform
>>>>>>>>>> the
>>>>>>>>>>>>>>>> activity
>>>>>>>>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on
>> ESX.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The driver doesn't know which hypervisor it's talking to,
>> it
>>>> just
>>>>>>>>>>>>>> sends a
>>>>>>>>>>>>>>>>> message to the host to perform the necessary pre-
>> attach
>>>>>> work.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Could we just expose a method like
>>>>>>>>>> "attachVolume/dettachVolume" on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> PrimaryDataStoreDriver driver?
>>>>>>>>>>>>>>>> For most of cases, the implementation of each driver
>> would
>>>>>> just
>>>>>>>>>>>> send
>>>>>>>>>>>>>>>> attachvolumecommand/dettachvolumecmd to
>>>> hypervisor(we
>>>>>> can
>>>>>>>>>> put
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> implementation in a base class, so that can be shared by
>> all
>>>> of
>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> drivers), and in each hypervisor resource code, which may
>>>> just
>>>>>>>> call
>>>>>>>>>>>>>>>> hypervisor's api to attach the volume to VM, while for
>>>> certain
>>>>>>>>>>>> storage,
>>>>>>>>>>>>>>>> like SolidFire, may need to create a SR first, and create
>>>> volume
>>>>>>>>>>>> on it,
>>>>>>>>>>>>>>>> then call hypervisor's API to attach volume to VM.
>>>>>>>>>>>>>>>> While for some other storage vendor, may want to bypass
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>> during
>>>>>>>>>>>>>>>> attaching volume, so inside driver's attachvolume
>>>>>> implementation,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> driver can do some magic, such as, directly talk to an agent
>>>>>>>>>>>> inside VM
>>>>>>>>>>>>>>>> instance, then create a disk inside VM.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How do you guys think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su
>>>>>> <Ed...@citrix.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>>>> From: Mike Tutkowski
>>>>>> [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> How's about this:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The driver can implement an isManaged() method.
>> The
>>>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
>>>>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>>>> (which is responsible for calling into the hypervisor to
>>>>>> attach
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> disk)
>>>>>>>>>>>>>>>>>>> can call into the hypervisor to create the necessary
>>>>>> hypervisor
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The problem here is that storage vendor may work
>>>>>> differently
>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN,
>>>> while
>>>>>>>>>>>> maybe
>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign
>> the
>>>>>> LUN
>>>>>>>>>>>>> directly
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> VM instance, see the discuss(
>>>>>>>>>>>>>>>>>> http://mail-
>> archives.apache.org/mod_mbox/cloudstack-
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>>>>>>>>>>>>>>>>> %3E
>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>> So I would let storage provider to implement attach
>> disk
>>>> to
>>>>>> VM,
>>>>>>>>>>>>>>>> instead of
>>>>>>>>>>>>>>>>>> implemented by cloudstack itself.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If that's what you're going for, that works for me. By
>> the
>>>>>> way,
>>>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>>>> default storage plug-in (which handles the default
>>>> storage
>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include
>> code
>>>> that
>>>>>>>>>>>> talks to
>>>>>>>>>>>>>>>>>>> hypervisors. You might want to contact him and
>> inform
>>>> him
>>>>>> of
>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>> concerns
>>>>>>>>>>>>>>>>>>> or that logic (as is) will make it to production.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Please let me know if what I wrote in above (for my
>>>>>> solution)
>>>>>>>>>>>> is OK
>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please see my responses in-line below.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I
>>>> build
>>>>>>>>>>>> up
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>> file, but this is the gist of what I've done:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * During a volume-attach operation, after
>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a
>>>> volume, I
>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask
>>>> the
>>>>>>>>>> driver if
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver
>>>>>> perform
>>>>>>>>>>>>>>>> whatever
>>>>>>>>>>>>>>>>>>>>> activity is required. In my case, this includes sending
>> a
>>>>>>>>>>>>>>>> message to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> host where the VM is running to have, say
>> XenServer,
>>>>>> add a
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the
>>>> IQN
>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the
>>>> space
>>>>>> it
>>>>>>>>>>>> can
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> storage repository). After this, the normal attach-
>>>> volume
>>>>>>>>>>>>>>>> message is
>>>>>>>>>>>>>>>>>> sent
>>>>>>>>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> There should be **no** code from a storage driver
>> to
>>>> a
>>>>>>>>>>>> hypervisor.
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>> apologize for the repetition, but we simply can not
>>>> have
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>> specific code in the storage layer. The circular
>>>>>> dependencies
>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> two layers are not sustainable in the long term.
>> Either
>>>> the
>>>>>>>>>>>>>>>>>> VirtualManager
>>>>>>>>>>>>>>>>>>>> or Xen hypervisor plugin needs to be
>>>> refactored/modified
>>>>>> to
>>>>>>>>>>>>>>>>> coordinate
>>>>>>>>>>>>>>>>>>>> volume creation and then populating the SR. Ideally,
>>>> we
>>>>>> can
>>>>>>>>>>>>>>>>>> generalize the
>>>>>>>>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>>>>>> would only implement callbacks to perform the
>> attach
>>>>>> action
>>>>>>>>>> and
>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver
>>>> should
>>>>>>>>>>>> only be
>>>>>>>>>>>>>>>>>>>> allocating space and providing information about
>>>> contents
>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>> space
>>>>>>>>>>>>>>>>>>>> available, space consumed, streams to a URI, file
>>>> handle
>>>>>> for a
>>>>>>>>>>>> URI,
>>>>>>>>>>>>>>>>>> etc)
>>>>>>>>>>>>>>>>>>>> and capabilities.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * The reverse is performed for a detach-volume
>>>>>> command.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged()
>> in
>>>> my
>>>>>>>>>>>> driver.
>>>>>>>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>>>>>> default driver simply does a "return false;". We
>> could
>>>>>> add a
>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>> parameter
>>>>>>>>>>>>>>>>>>>>> to the createStoragePool API command, if we want,
>>>> to
>>>>>>>>>> remove
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>>>>>>>> return values in the drivers (although my driver will
>>>>>>>> probably
>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>> this parameter and always return true since it
>>>> wouldn't
>>>>>> make
>>>>>>>>>>>>>>>> sense
>>>>>>>>>>>>>>>>>> for it
>>>>>>>>>>>>>>>>>>>>> to ever return false). We'd need another column in
>>>> the
>>>>>>>>>>>>>>>> storage_pool
>>>>>>>>>>>>>>>>>>> table
>>>>>>>>>>>>>>>>>>>>> to store this value.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yes, I think we should have a parameter added to
>> the
>>>>>>>>>>>>>>>>> createStoragePool
>>>>>>>>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to
>> be
>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> management when their underlying drivers support
>> it.
>>>> To
>>>>>>>>>>>> simplify
>>>>>>>>>>>>>>>>>> things,
>>>>>>>>>>>>>>>>>>>> this flag should only be mutable when the DataStore
>> is
>>>>>>>>>>>> created. It
>>>>>>>>>>>>>>>>>> would be
>>>>>>>>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to
>>>>>> unmanaged
>>>>>>>>>> after
>>>>>>>>>>>>>>>>>> creation.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have
>> a
>>>>>>>>>> working
>>>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>> to review (in that document), if you'd like to check
>> it
>>>>>> out.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed
>>>>>> together to
>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor
>>>>>> throttled
>>>>>>>>>> IOPs
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the
>>>> potential
>>>>>> for
>>>>>>>>>>>>>>>>>> confusion,
>>>>>>>>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these
>>>> features
>>>>>> will
>>>>>>>>>>>>>>>> help
>>>>>>>>>>>>>>>>>>>>>>> minimize any potential confusion.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will
>>>> work
>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the
>> Google
>>>>>> doc I
>>>>>>>>>> sent
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> other day to see how the GUI looks for
>>>> provisioned
>>>>>>>>>> storage
>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the
>> e-
>>>> mail
>>>>>>>> list
>>>>>>>>>>>>>>>> and we
>>>>>>>>>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the
>> Add
>>>>>> Disk
>>>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>>> dialog.
>>>>>>>>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS,
>>>> so
>>>>>> they
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they,
>> say,
>>>>>> only
>>>>>>>>>>>> use
>>>>>>>>>>>>>>>> Max
>>>>>>>>>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for
>>>>>> storage
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size
>>>> field,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> admin can
>>>>>>>>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max,
>>>> and
>>>>>>>>>> Burst
>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either
>> Wei's
>>>>>> feature
>>>>>>>>>> or
>>>>>>>>>>>>>>>> mine,
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> decided by the admin).
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two
>>>>>> features
>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> speed
>>>>>>>>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN
>> express
>>>> the
>>>>>>>>>> IOPS
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> QoS in
>>>>>>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in
>> into
>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>>>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on
>> the
>>>>>>>>>>>>>>>>> Storage/Hypervisor
>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the
>>>>>> behavior
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I
>> see
>>>>>> the
>>>>>>>>>>>>>>>> following
>>>>>>>>>>>>>>>>>>>> issues
>>>>>>>>>>>>>>>>>>>>>>>>> outstanding:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as
>>>>>> discrete
>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are
>>>> expressed
>>>>>> as
>>>>>>>>>> total
>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS
>>>>>> attached
>>>>>>>>>> to
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for
>>>> throttled
>>>>>> and
>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>> that will permit providers to discriminate
>> these
>>>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled
>> and
>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would
>>>>>> neatly
>>>>>>>>>>>> solve
>>>>>>>>>>>>>>>>>> items 1
>>>>>>>>>>>>>>>>>>>>>>> and 2.
>>>>>>>>>>>>>>>>>>>>>>>>> Since the two facilities would not be
>> permitted
>>>> to
>>>>>>>>>> operate
>>>>>>>>>>>>>>>>>> together,
>>>>>>>>>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I
>>>> think
>>>>>> item
>>>>>>>>>> 3
>>>>>>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>>>>>> resolved
>>>>>>>>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage
>>>> records.
>>>>>> As
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> Item 4,
>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>>>>>>>>>> not familiar with how these two
>> enhancements
>>>> are
>>>>>>>>>>>>> depicted in
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
>>>>>>>>>> enhancements
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>> enhancements and ensure they are
>> consistent.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>>>>>>>>>>>>> <us...@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> John and Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I was busy working on other issues
>>>> (CLOUDSTACK-
>>>>>>>>>>>>> 2780/2729,
>>>>>>>>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865,
>> CLOUDSTACK-
>>>>>> 2823 ,
>>>>>>>>>>>>>>>>> CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 2875 ) this
>>>>>>>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes
>>>>>> tomorrow,
>>>>>>>>>> and
>>>>>>>>>>>>>>>> ask
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> second
>>>>>>>>>>>>>>>>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
>>>>>>>>>> <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going
>>>> with
>>>>>> this,
>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John
>> Burwell
>>>>>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike
>> Tutkowski
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager
>>>>>> determining
>>>>>>>>>>>>> whether or
>>>>>>>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
>>>>>> StoragePoolType,
>>>>>>>>>> but an
>>>>>>>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
>>>>>> allocate
>>>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an
>>>>>> isManaged()
>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>> added to
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it
>>>> returns
>>>>>> true,
>>>>>>>>>>>>>>>> then the
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or
>>>>>> whatever
>>>>>>>>>> name)
>>>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver
>> interface,
>>>>>> as
>>>>>>>>>> well)
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code
>> to
>>>>>> create,
>>>>>>>>>> say
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>> SR?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() :
>>>> boolean
>>>>>> on
>>>>>>>>>>>>>>>>>>> DataStoreDriver.
>>>>>>>>>>>>>>>>>>>>>>> Since
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to
>>>> primary
>>>>>>>>>> storage,
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage
>>>> (Storage
>>>>>> is a
>>>>>>>>>>>>> straw
>>>>>>>>>>>>>>>>> man
>>>>>>>>>>>>>>>>>>> term
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> something other than volume) methods
>> to
>>>>>>>>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed
>> is
>>>> a
>>>>>>>>>> mutable
>>>>>>>>>>>>>>>>> property
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> DataStore
>>>>>>>>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the
>>>> underlying
>>>>>>>>>>>>>>>>> DataStoreDriver
>>>>>>>>>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
>>>>>>>>>>>>> manageability
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> devices.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for
>> SR,
>>>>>> the
>>>>>>>>>> Xen
>>>>>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> responsible for composing
>>>> DataStore/Volume
>>>>>>>>>> methods
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> create any
>>>>>>>>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR.
>>>> There
>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> be no
>>>>>>>>>>>>>>>>>>>>>>>>>>> dependencies
>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer.
>> As
>>>> I
>>>>>> said
>>>>>>>>>>>>>>>> earlier,
>>>>>>>>>>>>>>>>>> such
>>>>>>>>>>>>>>>>>>>>>>>>>>> circular
>>>>>>>>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
>>>>>>>>>> unmaintainable
>>>>>>>>>>>>> mess.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same
>> page
>>>>>> with
>>>>>>>>>> you.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John
>> Burwell
>>>>>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a
>>>>>> Storage
>>>>>>>>>> layer
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> supports n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> devices types with each specific
>> behaviors
>>>> of
>>>>>> m
>>>>>>>>>>>>>>>> hypervisors.
>>>>>>>>>>>>>>>>>>>> Such
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable
>> and
>>>>>>>>>> untestable
>>>>>>>>>>>>>>>>> beast.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are
>> driven
>>>> to
>>>>>>>>>> evolve
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager
>>>>>> determining
>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
>>>>>> StoragePoolType,
>>>>>>>>>> but
>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
>>>>>> allocate
>>>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device
>>>>>> driver
>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> indicate
>>>>>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or not it supports management
>> operations.
>>>>>>>>>> Finally, I
>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to
>>>> elect
>>>>>> to
>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the
>> driver
>>>> is
>>>>>>>>>> capable
>>>>>>>>>>>>>>>>> managing
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to
>> leave it
>>>>>>>>>>>>> unmanaged).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume
>>>> should
>>>>>> be
>>>>>>>>>> done
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods
>> exposed
>>>> by
>>>>>> the
>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>> layer to
>>>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make
>> directories,
>>>>>>>>>> create a
>>>>>>>>>>>>>>>> file,
>>>>>>>>>>>>>>>>>> etc).
>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of
>> the
>>>>>> Xen
>>>>>>>>>>>>>>>>>> hypervisor, as
>>>>>>>>>>>>>>>>>>>>>>> such,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation.
>>>>>> From
>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>>>>>> nothing
>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned
>> with
>>>>>>>>>> content.
>>>>>>>>>>>>> From
>>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It
>> provides
>>>>>> the
>>>>>>>>>> means
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> query
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the
>> content
>>>> by
>>>>>>>>>> higher-
>>>>>>>>>>>>> level
>>>>>>>>>>>>>>>>>>> layers
>>>>>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should
>> be
>>>> a
>>>>>>>>>>>>> composition
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based
>> on
>>>>>> the
>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> protocol
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic
>> is
>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>> patch,
>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code
>>>> alone.
>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and
>>>> send it
>>>>>> to
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor.
>>>>>>>>>>>>>>>>>>>> Before
>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see
>> if
>>>>>> we're
>>>>>>>>>>>>> dealing
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>> (or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and
>> - if
>>>> so
>>>>>> -
>>>>>>>>>>>> send a
>>>>>>>>>>>>>>>>>> "Create
>>>>>>>>>>>>>>>>>>>>>>> SR"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this
>> returns
>>>> OK,
>>>>>> we
>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>> proceed
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the
>> same
>>>>>> and
>>>>>>>>>> we just
>>>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called
>>>> for
>>>>>> this
>>>>>>>>>>>>>>>> particular
>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer
>>>> today
>>>>>> that
>>>>>>>>>>>>> outlines
>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will
>>>>>> probably
>>>>>>>>>> be
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> you, as
>>>>>>>>>>>>>>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you
>>>> (you
>>>>>>>>>> should
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John
>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I
>>>> some
>>>>>>>>>>>>> thoughts,
>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas
>> against
>>>> the
>>>>>>>>>>>> solidfire,
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to
>>>> collect
>>>>>> them
>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>> coherent
>>>>>>>>>>>>>>>>>>>>>>> form
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
>>>>>>>>>> introduces
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in
>> just
>>>>>>>>>> introduces
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that
>> leverages
>>>>>> the
>>>>>>>>>> plug-in
>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>> incomplete,
>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a
>>>> managed
>>>>>> iSCSI
>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've
>> been
>>>>>>>>>> spending on
>>>>>>>>>>>>>>>> this.
>>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>> Do
>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we
>> should
>>>>>>>>>> accomplish
>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>> you're
>>>>>>>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM,
>> John
>>>>>> Burwell
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
>>>>>>>>>> introduces
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
>>>>>>>>>> management
>>>>>>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>>>>>>> (i.e.
>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an
>>>> operator
>>>>>> want
>>>>>>>>>> it
>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we
>> should
>>>>>> end up
>>>>>>>>>> with
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> semantic
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device
>> to
>>>> be
>>>>>>>>>> managed.
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the
>> storage
>>>>>> types
>>>>>>>>>> from
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike
>>>>>> Tutkowski
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below
>> in
>>>> red.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM,
>>>> John
>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the
>>>>>> following
>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
>>>> iSCSI
>>>>>>>>>> LUNs
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>> volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to
>> a
>>>>>> single
>>>>>>>>>> LUN
>>>>>>>>>>>>> (LUN
>>>>>>>>>>>>>>>>> 0).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
>>>> SAN
>>>>>>>>>> Volumes
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack
>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
>>>>>>>>>> management
>>>>>>>>>>>>>>>>> server or
>>>>>>>>>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire
>> plug-
>>>> in,
>>>>>> a
>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> (same
>>>>>>>>>>>>>>>>>>>> as a
>>>>>>>>>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management
>> server
>>>>>> (via
>>>>>>>>>> the
>>>>>>>>>>>>> plug-in)
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> first
>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to
>> a
>>>>>>>>>> hypervisor.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the
>>>> SolidFire
>>>>>> plug-
>>>>>>>>>> in,
>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> still
>>>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can
>> do
>>>>>> this
>>>>>>>>>> already
>>>>>>>>>>>>>>>>> today
>>>>>>>>>>>>>>>>>>>> (prior
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN
>>>>>> volume
>>>>>>>>>> and -
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> case -
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN
>>>> volume.
>>>>>>>>>> While
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> can do
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce
>>>>>> storage
>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
>>>>>>>>>> management
>>>>>>>>>>>>> server
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is
>> used,
>>>> the
>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via
>> the
>>>>>> plug-
>>>>>>>>>> in).
>>>>>>>>>>>>> There
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping
>>>> between
>>>>>> a
>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> and a
>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any
>> storage
>>>>>> vendor
>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>> true
>>>>>>>>>>>>>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these
>>>>>> pieces
>>>>>>>>>> are
>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM,
>> Mike
>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we
>> ensure
>>>>>> that
>>>>>>>>>>>>> multiple
>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the
>>>> underlying
>>>>>>>>>> storage."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we
>>>> need
>>>>>> to
>>>>>>>>>> map a
>>>>>>>>>>>>>>>>> single
>>>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We
>>>>>> cannot
>>>>>>>>>> have
>>>>>>>>>>>>>>>>> multiple
>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume
>> and
>>>> still
>>>>>>>>>>>>> guarantee
>>>>>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a
>> single
>>>>>> SAN
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> house
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can
>> do
>>>>>> that
>>>>>>>>>> today
>>>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>>>> any of
>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM,
>>>>>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will
>> allocate
>>>> a
>>>>>> SAN
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes
>> will
>>>> be
>>>>>>>>>> created."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail
>> on
>>>> this.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41
>> PM,
>>>>>> John
>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part
>>>> which
>>>>>>>>>>>>> concerns me
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into
>>>>>> storage
>>>>>>>>>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM,
>>>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the
>>>> iSCSI
>>>>>> type
>>>>>>>>>> is
>>>>>>>>>>>>> today
>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>> (as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer
>>>> (when
>>>>>> you
>>>>>>>>>> have
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> PreSetup an
>>>>>>>>>>>>>>>>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your
>> iSCSI
>>>>>>>>>> volume
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> VMware, it
>>>>>>>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your
>> iSCSI
>>>>>>>>>> volume
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> KVM,
>>>>>>>>>>>>>>>>>>> it uses
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and
>>>> Edison's
>>>>>>>>>> thinking
>>>>>>>>>>>>>>>>> here
>>>>>>>>>>>>>>>>>>> was to
>>>>>>>>>>>>>>>>>>>>>>> make a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this
>>>>>> dynamic
>>>>>>>>>> ability
>>>>>>>>>>>>>>>>> Edison
>>>>>>>>>>>>>>>>>>> added
>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it
>> should
>>>> be
>>>>>>>>>> more
>>>>>>>>>>>>>>>>> specificy,
>>>>>>>>>>>>>>>>>>> though:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27
>>>> PM,
>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
>>>>>>>>>> shouldn't
>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> care
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be
>>>> able
>>>>>> to
>>>>>>>>>> answer
>>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern
>> here
>>>> is
>>>>>> that
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage
>> so
>>>>>> that
>>>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>> knows
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to
>>>> removing
>>>>>> that
>>>>>>>>>>>>> constraint
>>>>>>>>>>>>>>>>>>> and just
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just
>>>>>> need a
>>>>>>>>>> way
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new
>>>> requirement
>>>>>> and
>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>> 1:24
>>>> PM,
>>>>>>>>>> John
>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-
>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10
>> PM,
>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture
>> this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while
>> CloudStack
>>>>>> may
>>>>>>>>>> be
>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it
>>>>>> unmanaged by
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> (e.g. the
>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and
>> the
>>>>>>>>>> operator has
>>>>>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does
>> my
>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure
>> how
>>>>>>>>>> creating a
>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>> volume via
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
>>>>>>>>>> hypervisor)
>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> work unless
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the
>>>> form
>>>>>> of,
>>>>>>>>>> say,
>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that,
>>>>>> independent
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> CloudStack, an
>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to
>>>>>> CloudStack,
>>>>>>>>>> and
>>>>>>>>>>>>>>>>> exposes it
>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of
>>>> that
>>>>>> LUN
>>>>>>>>>> over
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate
>>>>>> anymore
>>>>>>>>>> LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic
>> stands
>>>>>> prior
>>>>>>>>>> to my
>>>>>>>>>>>>>>>>>>> changes, we
>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does
>>>> not
>>>>>> have
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we
>>>> should
>>>>>>>>>> maybe
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN
>> volume
>>>>>> needs
>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent
>> code
>>>>>>>>>> specifically
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent
>>>> code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
>>>>>>>>>> management
>>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
>>>>>>>>>> determine
>>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would
>> allow
>>>>>> the
>>>>>>>>>>>>>>>>> interrogation of
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The
>> storage
>>>>>>>>>> device
>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>> shouldn't
>>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen
>> SR -
>>>> -
>>>>>>>>>> simply be
>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> answer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>>>> 1:01
>>>>>> PM,
>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as
>>>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers
>>>> the
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage).
>> Once
>>>>>> this is
>>>>>>>>>>>>> done, a
>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates
>> a
>>>> Disk
>>>>>>>>>> Offering
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created
>> Primary
>>>>>>>>>> Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user
>> creates
>>>> a
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> volume. This
>>>>>>>>>>>>>>>>>>>>>>> leads
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user
>> attaches
>>>>>> the
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> volume to a
>>>>>>>>>>>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
>>>>>>>>>> framework
>>>>>>>>>>>>> calling
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system
>>>> (in
>>>>>> my
>>>>>>>>>> case, a
>>>>>>>>>>>>>>>>> SAN).
>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row
>> with
>>>>>>>>>> applicable
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> (like the
>>>>>>>>>>>>>>>>>>>>>>> IQN of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is
>> only
>>>>>>>>>> invoked if
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state.
>> After
>>>> the
>>>>>>>>>> attach,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a
>>>> detach
>>>>>> disk)
>>>>>>>>>> and
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this
>> SAN
>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-
>> attach
>>>>>> logic
>>>>>>>>>> is run
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> detects
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs
>>>> "assistance"
>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>> form
>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013
>> at
>>>>>> 12:54
>>>>>>>>>> PM,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we
>>>> are in
>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>> terminology,
>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers
>> to
>>>> the
>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>> allocation on
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say
>>>> 'volume',
>>>>>> I
>>>>>>>>>> try to
>>>>>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>>>>>> 'SAN
>>>>>>>>>>>>>>>>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the
>>>> 'volume'
>>>>>> the
>>>>>>>>>> end
>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> make in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack
>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013
>> at
>>>>>> 12:50
>>>>>>>>>> PM,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here
>>>> may
>>>>>>>>>> very
>>>>>>>>>>>>> well
>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> sense, but
>>>>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we
>> should
>>>>>> draw
>>>>>>>>>> Edison
>>>>>>>>>>>>> in on
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> conversation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to
>>>> suggest
>>>>>> the
>>>>>>>>>>>>>>>>> approach I
>>>>>>>>>>>>>>>>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4,
>> 2013
>>>> at
>>>>>> 12:42
>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we
>> are
>>>>>>>>>> combining
>>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>> distinct
>>>>>>>>>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
>> and
>>>>>> storage
>>>>>>>>>>>>> protocols.
>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>> cases, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one
>>>> allows
>>>>>> the
>>>>>>>>>>>>> system to
>>>>>>>>>>>>>>>>>>>> create/delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the
>>>> other
>>>>>>>>>> requires
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> volume to
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the
>>>>>> CloudStack
>>>>>>>>>>>>> context.
>>>>>>>>>>>>>>>>> To
>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology,
>>>> volume,
>>>>>> in
>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>> definitions,
>>>>>>>>>>>>>>>>>>>>>>> refers to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the
>>>>>> device,
>>>>>>>>>>>>> correct?
>>>>>>>>>>>>>>>>>>> Minimally,
>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate
>> with a
>>>>>>>>>> storage
>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> move bits
>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete
>>>> bits,
>>>>>> etc.
>>>>>>>>>>>>>>>>> Optionally, a
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
>>>>>> CloudStack.
>>>>>>>>>>>>> Therefore,
>>>>>>>>>>>>>>>>>>> we can have
>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we
>>>> store a
>>>>>> Xen
>>>>>>>>>> SR,
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> can have a
>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
>>>>>>>>>> CloudStack is
>>>>>>>>>>>>>>>>> capable
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> allocating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally,
>>>>>> while
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>> be able
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may
>> chose
>>>> to
>>>>>>>>>> leave it
>>>>>>>>>>>>>>>>>>> unmanaged by
>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many
>>>>>> services,
>>>>>>>>>> and
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> operator has
>>>>>>>>>>>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
>>>>>>>>>> CloudStack).
>>>>>>>>>>>>> Does
>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my
>>>> thoughts
>>>>>>>>>> above
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> reasonable, it
>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the
>>>> management
>>>>>>>>>>>>> concerns
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device
>> with
>>>> an
>>>>>>>>>>>>> attached
>>>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
>>>>>> CloudStack,
>>>>>>>>>> and
>>>>>>>>>>>>>>>>> establish
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a
>>>> device
>>>>>>>>>> separate
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> associated
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in
>>>> place,
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> drivers can
>>>>>>>>>>>>>>>>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and
>> when
>>>>>> they
>>>>>>>>>>>>> encounter
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
>>>>>> operations
>>>>>>>>>>>>> exposed
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> the driver to
>>>>>>>>>>>>>>>>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these
>>>>>> thoughts/concepts
>>>>>>>>>> make
>>>>>>>>>>>>>>>>> sense,
>>>>>>>>>>>>>>>>>>> then we can
>>>>>>>>>>>>>>>>>>>>>>> sit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed
>>>> design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>>>> 5:25
>>>>>> PM,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the
>>>>>> difference
>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The
>>>> admin
>>>>>>>>>> has to
>>>>>>>>>>>>> go in
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type.
>> At
>>>> this
>>>>>>>>>> point in
>>>>>>>>>>>>> time,
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system
>> (it
>>>> is
>>>>>> pre-
>>>>>>>>>>>>> allocated).
>>>>>>>>>>>>>>>>>>> Future
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the
>> SR
>>>>>> that
>>>>>>>>>> was
>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>> behind the
>>>>>>>>>>>>>>>>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type:
>> The
>>>>>> admin
>>>>>>>>>> has
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> go in
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will
>>>>>> create
>>>>>>>>>> and
>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>> volumes on
>>>>>>>>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as
>> is
>>>>>>>>>> enabled via
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a
>>>> CloudStack
>>>>>>>>>> volume
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>> created,
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
>> create a
>>>>>> new
>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>> After
>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>>>>> done,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor
>> in
>>>>>>>>>> question is
>>>>>>>>>>>>> called.
>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point
>>>> because
>>>>>> the
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must
>> be
>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013
>>>>>> at
>>>>>>>>>> 3:21
>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new
>>>>>> terms,
>>>>>>>>>> so I
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage:
>>>>>> Storage
>>>>>>>>>> that is
>>>>>>>>>>>>>>>>> pre-
>>>>>>>>>>>>>>>>>>> allocated
>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a
>>>> SAN),
>>>>>> then
>>>>>>>>>> a
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage
>> (ex.
>>>>>>>>>> XenServer
>>>>>>>>>>>>> SR),
>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
>>>>>>>>>> CloudStack.
>>>>>>>>>>>>> Disks
>>>>>>>>>>>>>>>>> (VDI)
>>>>>>>>>>>>>>>>>>> are later
>>>>>>>>>>>>>>>>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure
>> as
>>>>>>>>>> needed. In
>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>> cases, the
>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> data
>>>>>>>>>> structure is
>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>> in place
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>> hypervisor
>>>>>> data
>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the VM in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>> storage:
>>>>>>>>>> Storage
>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent
>>>>>> storage,
>>>>>>>>>> this
>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>> be a
>>>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>> (not
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself).
>> The
>>>>>>>>>> hypervisor
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>> must be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is
>>>>>> performed
>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>> types of
>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a
>>>>>> hypervisor
>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> structure by an
>>>>>>>>>>>>>>>>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say,
>> an
>>>> SR
>>>>>> on
>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI
>> within
>>>> the
>>>>>> SR
>>>>>>>>>> to the
>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013
>>>>>> at
>>>>>>>>>> 3:13
>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>> John Burwell
>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>>>>>>>>>>>>> implementation of
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen
>>>> ISCSI
>>>>>>>>>> which
>>>>>>>>>>>>> why I
>>>>>>>>>>>>>>>>> ask
>>>>>>>>>>>>>>>>>>> about the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the
>>>> question --
>>>>>>>>>> what is
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> definition
>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>> 2013,
>>>> at
>>>>>> 5:10
>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I
>>>> know,
>>>>>> the
>>>>>>>>>> iSCSI
>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> uniquely
>>>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set
>> up
>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>>>>>> directly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows
>> you
>>>> to
>>>>>>>>>> skip the
>>>>>>>>>>>>>>>>> step of
>>>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>> repository
>>>>>>>>>> based
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>> target as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I
>> think
>>>>>> this is
>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>> supported for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors,
>>>> you
>>>>>>>>>> must
>>>>>>>>>>>>> first go
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really
>>>>>> know
>>>>>>>>>> what
>>>>>>>>>>>>> RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun 3,
>>>>>> 2013
>>>>>>>>>> at
>>>>>>>>>>>>> 2:13
>>>>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading
>>>>>> through
>>>>>>>>>> the
>>>>>>>>>>>>> code,
>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>> types?
>>>>>>>>>> Why
>>>>>>>>>>>>> isn't
>>>>>>>>>>>>>>>>> RBD
>>>>>>>>>>>>>>>>>>> considered
>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>>>> 2013,
>>>>>> at
>>>>>>>>>> 3:46
>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new
>>>> type
>>>>>> of
>>>>>>>>>>>>> storage is
>>>>>>>>>>>>>>>>>>> defined in
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called
>>>>>> Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public
>> static
>>>>>> enum
>>>>>>>>>>>>>>>>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Filesystem(false),
>>>>>>>>>> //
>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> NetworkFilesystem(true),
>>>>>>>>>>>>>>>>> //
>>>>>>>>>>>>>>>>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IscsiLUN(true),
>>>>>> //
>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>> LUN,
>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Iscsi(true),
>>>> //
>>>>>> for
>>>>>>>>>> e.g.,
>>>>>>>>>>>>> ZFS
>>>>>>>>>>>>>>>>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> ISO(false),
>>>> //
>>>>>> for
>>>>>>>>>> iso
>>>>>>>>>>>>> image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> LVM(false),
>>>> //
>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> VMFS(true),
>>>> //
>>>>>>>>>>>>> VMware
>>>>>>>>>>>>>>>>> VMFS
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> PreSetup(true), //
>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> XenServer, Storage
>>>>>>>>>>>>>>>>>>>>>>> Pool
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> EXT(false),
>>>> //
>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Dynamic(true);
>>>>>> //
>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean
>>>>>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> StoragePoolType(boolean
>>>>>>>>>>>>>>>>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> this.shared
>>>> =
>>>>>>>>>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public
>>>> boolean
>>>>>>>>>>>>> isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return
>>>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>> Jun 3,
>>>>>>>>>> 2013 at
>>>>>>>>>>>>> 1:41
>>>>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>> example,
>>>>>>>>>> let's say
>>>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> company
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>> to
>>>>>>>>>> leverage its
>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> Service
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-
>> wide
>>>>>>>>>> storage, as
>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>> They would
>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>> as
>>>> I've
>>>>>>>>>> made
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>> changes to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-
>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> support
>>>>>> their
>>>>>>>>>> plug-
>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>> Jun
>>>>>> 3,
>>>>>>>>>> 2013
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> 1:39
>>>>>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh,
>> sorry
>>>> to
>>>>>>>>>> imply
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> XenServer code
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>> XenServer
>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>> logic is
>>>>>>>>>>>>>>>>>>> now aware
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and
>>>>>> SolidFire is
>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> implementation of
>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> storage is
>>>>>> new
>>>>>>>>>> to 4.2
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison
>>>>>> created
>>>>>>>>>> a
>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>> framework that
>>>>>>>>>>>>>>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>> volumes
>>>>>>>>>>>>> dynamically.
>>>>>>>>>>>>>>>>>>> However, when I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April,
>>>> we
>>>>>>>>>> realized
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> was not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> nothing
>>>>>>>>>> CloudStack
>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>>> with these
>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> changed
>>>> to
>>>>>>>>>>>>> recognize
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> new type of
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> hypervisor
>>>>>> data
>>>>>>>>>>>>> structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Mon,
>>>>>> Jun 3,
>>>>>>>>>> 2013
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> 1:28
>>>>>>>>>>>>>>>>>>> PM, John
>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>> generally
>>>>>>>>>> odd
>>>>>>>>>>>>> to me
>>>>>>>>>>>>>>>>>>> that any
>>>>>>>>>>>>>>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> understand or
>>>>>>>>>> care
>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> details. I
>>>>>>>>>>>>>>>>>>>>>>> expect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> expose
>>>> a
>>>>>> set
>>>>>>>>>> of
>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>> that can be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> implementations to
>>>>>>>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>>>>>>>> space/create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>> invert
>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> dependency, we are
>>>>>>>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> problem
>>>>>> that
>>>>>>>>>> will
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> enhance.
>>>>>> Am
>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> understanding that the
>>>>>>>>>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> located
>>>> in
>>>>>> the
>>>>>>>>>>>>>>>>>>> CitrixResourceBase
>>>>>>>>>>>>>>>>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Mon,
>>>>>> Jun 3,
>>>>>>>>>>>>> 2013 at
>>>>>>>>>>>>>>>>> 3:12
>>>>>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
>>>> delve
>>>>>> into
>>>>>>>>>> this
>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>> bit
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Prior
>>>> to
>>>>>> 4.2
>>>>>>>>>> and
>>>>>>>>>>>>> aside
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> one setup
>>>>>>>>>>>>>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> first
>>>>>> create
>>>>>>>>>> a
>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> set
>>>> up
>>>>>> a
>>>>>>>>>> data
>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>> to make
>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> repository on
>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>> or a
>>>>>>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> shared
>>>>>> this
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> With
>>>>>> Edison's
>>>>>>>>>> new
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> framework,
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>> you
>>>>>> can
>>>>>>>>>> easily
>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>> a 1:1
>>>>>>>>>>>>>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> volume
>>>>>> and
>>>>>>>>>> the
>>>>>>>>>>>>> VM's
>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>> (necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
>>>> can
>>>>>> now
>>>>>>>>>> write
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> plug-
>>>>>>>>>>>>>>>>>>> in that is
>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> volumes
>>>>>> as
>>>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>> problem
>>>>>>>>>> that
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> deleting
>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor-
>>>>>>>>>>>>>>>>>>> specific data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>>>>>> being
>>>>>>>>>> the
>>>>>>>>>>>>> case,
>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>> enhancing
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> worked
>>>>>> out
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> submitted. I've got
>>>>>>>>>>>>>>>>>>>>>>> ESX(i)
>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>> we
>>>>>> extend
>>>>>>>>>> the
>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Does
>>>> that
>>>>>>>>>> help a
>>>>>>>>>>>>> bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> Mon,
>>>>>> Jun
>>>>>>>>>> 3,
>>>>>>>>>>>>> 2013 at
>>>>>>>>>>>>>>>>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> The
>>>>>> storage
>>>>>>>>>>>>> plug-in -
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> itself - is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> The
>>>>>> issue is
>>>>>>>>>> with
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> volume-attach
>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> framework
>>>>>>>>>> calls
>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> plug-in to
>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> when
>>>>>> the
>>>>>>>>>> time
>>>>>>>>>>>>>>>>> comes to
>>>>>>>>>>>>>>>>>>> attach the
>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> has
>>>> to
>>>>>> be
>>>>>>>>>> smart
>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>> to recognize
>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (where
>>>>>> the
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>>>> just been
>>>>>>>>>>>>>>>>>>>>>>> created)
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> repository
>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>> XenServer) or a
>>>>>>>>>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> volume
>>>>>> that
>>>>>>>>>> was
>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> I've
>>>>>> been
>>>>>>>>>>>>> spending
>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>> of my time
>>>>>>>>>>>>>>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> the
>>>>>> agent
>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Does
>>>>>> that
>>>>>>>>>> clear it
>>>>>>>>>>>>> up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> Mon,
>>>>>>>>>> Jun 3,
>>>>>>>>>>>>> 2013
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> 12:48 PM,
>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Can
>>>>>> you
>>>>>>>>>> explain
>>>>>>>>>>>>>>>>> why
>>>>>>>>>>>>>>>>>>> the the storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>> Jun
>>>>>> 3,
>>>>>>>>>> 2013,
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> 1:21
>>>>>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Yes,
>>>>>>>>>>>>> ultimately I
>>>>>>>>>>>>>>>>>>> would like to
>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> supports.
>>>>>>>>>> I
>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>> just out of
>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Right
>>>>>>>>>> now this
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> regards
>>>>>>>>>> to 4.2
>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>> freeze, I
>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> as
>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Also,
>>>>>> just
>>>>>>>>>> to
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> clear,
>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> plan
>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> support
>>>>>>>>>>>>> Compute
>>>>>>>>>>>>>>>>>>> Offerings post
>>>>>>>>>>>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>> Mon,
>>>>>>>>>> Jun 3,
>>>>>>>>>>>>>>>>> 2013
>>>>>>>>>>>>>>>>>>> at 11:14 AM,
>>>>>>>>>>>>>>>>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Is
>>>>>> there
>>>>>>>>>> any
>>>>>>>>>>>>> plan
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> supporting
>>>>>>>>>>>>>>>>>>>>>>> KVM in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> --
>>>> ---
>>>>>>>>>> Original
>>>>>>>>>>>>>>>>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> From:
>>>>>>>>>> "Mike
>>>>>>>>>>>>>>>>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> To:
>>>>>>>>>>>>>>>>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Sent:
>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>> June
>>>>>>>>>>>>>>>>>>> 3, 2013
>>>>>>>>>>>>>>>>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Subject:
>>>>>>>>>> Re:
>>>>>>>>>>>>>>>>> [MERGE]
>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> I
>>>>>> agree
>>>>>>>>>> on
>>>>>>>>>>>>>>>>> merging
>>>>>>>>>>>>>>>>>>> Wei's feature
>>>>>>>>>>>>>>>>>>>>>>>>> first,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> If
>>>>>> his
>>>>>>>>>> feature
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> KVM only,
>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> KVM in
>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>> Mon,
>>>>>>>>>> Jun
>>>>>>>>>>>>> 3,
>>>>>>>>>>>>>>>>> 2013
>>>>>>>>>>>>>>>>>>> at 8:55 AM,
>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> For
>>>>>>>>>> the
>>>>>>>>>>>>> billing,
>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>> no one works
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> the
>>>>>>>>>> billing
>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> themselves. They
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> disk_offering of
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> VMs and
>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> to
>>>>>> tell
>>>>>>>>>> user
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> exact
>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> rate
>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> nics as
>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> is a
>>>>>>>>>> part of
>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Sorry
>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>> misunstand the second
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Agree
>>>>>>>>>> with
>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>> you said about
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> -
>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>> 2013/6/3
>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>> On
>>>>>>>>>> Jun 3,
>>>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> 2:13 AM, Wei
>>>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>> Hi
>>>>>>>>>> John,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>> I
>>>>>>>>>> hope
>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>> aswer helps you.
>>>>>>>>>>>>>>>>>>>>>>> I am
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>> (1) I
>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>> depend
>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>> limitation.
>>>>>>>>>>>>>>>>> Please
>>>>>>>>>>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>> get
>>>>>>>>>> the
>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>> virtual
>>>>>>>>>>>>>>>>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> Going by
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> AWS
>>>>>>>>>>>>>>>>>>> model,
>>>>>>>>>>>>>>>>>>>> customers
>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>>>>>> as well as,
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/).
>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> option to
>>>>>>>>>>>>>>>>> employ
>>>>>>>>>>>>>>>>>>> similar cost
>>>>>>>>>>>>>>>>>>>>>>>>> models.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> billing
>>>>>>>>>>>>> model
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the current
>>>>>>>>>>>>>>>>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>> (2)
>>>>>>>>>> Do
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>>>>>> IOPS runtime
>>>>>>>>>>>>>>>>>>>>>>>>> change?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>> limitation for
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>>> virtual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>> not support
>>>>>>>>>>>>>>>>>>>>>>>>> changing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>> (computer
>>>>>>>>>>>>>>>>>>> offering or disk
>>>>>>>>>>>>>>>>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> I
>>>>>>>>>> meant at
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Java interface
>>>>>>>>>>>>>>>>>>>>>>> level.
>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>> we
>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> generalize allocation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> describe
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> service guarantees
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>> (3)
>>>>>>>>>> It is a
>>>>>>>>>>>>>>>>> good
>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>> Maybe it
>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike
>>>>>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>> type, I
>>>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> I
>>>>>> will
>>>>>>>>>>>>> expand
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> my thoughts
>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> touch
>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>> between these two
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>> will
>>>>>>>>>> need
>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>> merged before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>> coordination
>>>>>>>>>>>>>>>>>>> between the
>>>>>>>>>>>>>>>>>>>> branches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>> disk_io_throttling)
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>> -
>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>> 2013/6/3
>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> The
>>>>>>>>>>>>> things I
>>>>>>>>>>>>>>>>>>> want to
>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> 1)
>>>>>>>>>> Is
>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>> value in
>>>>>>>>>>>>>>>>>>>> capturing
>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> data
>>>>>>>>>>>>> model
>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> 2)
>>>>>>>>>>>>> Should
>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>> be a common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>> provisioning
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> 3)
>>>>>>>>>> How
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> conflicting
>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> storage device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> is
>>>>>>>>>> lead
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> believe (and
>>>>>>>>>>>>>>>>>>>> billed)
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> could
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>> consistent
>>>>>>>>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>>>>>>>>> between a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> time,
>>>>>>>>>>>>> but a
>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>> modification
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> -
>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> On
>>>>>>>>>> Jun
>>>>>>>>>>>>> 2,
>>>>>>>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>>>>>> at 8:38 PM,
>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> Hi
>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> I
>>>>>>>>>>>>> believe
>>>>>>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>>>>>> feature deals
>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> number
>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> My
>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> focused on
>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> I
>>>>>>>>>> hope
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> On
>>>>>>>>>> Sun,
>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>>>>> 2,
>>>>>>>>>>>>>>>>>>> 2013 at 6:35
>>>>>>>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> My
>>>>>>>>>>>>>>>>> opinion is
>>>>>>>>>>>>>>>>>>> that no
>>>>>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> issues
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> been resolved
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> total
>>>>>>>>>>>>> Ops
>>>>>>>>>>>>>>>>> vs
>>>>>>>>>>>>>>>>>>> discrete
>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> re-
>>>>>>>>>>>>> reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido,
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>> think this
>>>>>>>>>>>>>>>>>>>>>>> criteria
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Also,
>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>> does this work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> also
>>>>>>>>>>>>>>>>> involves
>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>> where provisioned
>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> -
>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> On
>>>>>>>>>>>>> Jun 1,
>>>>>>>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>>>>>> at 6:47 AM,
>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Sure. I
>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>> change it next
>>>>>>>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> -
>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> 2013/6/1
>>>>>>>>>>>>>>>>>>> Wido den Hollander
>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> On
>>>>>>>>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> Exactly. I
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> pushed the
>>>>>>>>>>>>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> If
>>>>>>>>>>>>> anyone
>>>>>>>>>>>>>>>>>>> object thems for
>>>>>>>>>>>>>>>>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> For
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> sake
>>>>>>>>>>>>>>>>>>> of clarity I
>>>>>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>> to R/W IOps
>>>>>>>>>>>>>>>>>>>>>>> asap
>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> only
>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> You
>>>>>>>>>>>>> laid
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> groundwork for
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> however
>>>>>>>>>>>>>>>>>>> prevent that we
>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> -
>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> 2013/5/31
>>>>>>>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> On
>>>>>>>>>>>>>>>>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> +1 --
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> enhancement must
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> don't
>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>> it could be
>>>>>>>>>>>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how
>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> split
>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>> into read
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>> unless/until we
>>>>>>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> What
>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>> meant was merging
>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> 4.2
>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>> and add Read /
>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>> released with Read
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> This is
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> make the May 31st
>>>>>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> (see
>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>> threads) then it
>>>>>>>>>>>>>>>>>>>>>>> won't
>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> I
>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>> completely agree that
>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> disk
>>>>>>>>>>>>> I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> -
>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> On
>>>>>>>>>>>>> May 31,
>>>>>>>>>>>>>>>>>>> 2013, at 9:51 AM,
>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> On
>>>>>>>>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>> Good
>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> I
>>>>>>>>>>>>> thought
>>>>>>>>>>>>>>>>>>> about at the
>>>>>>>>>>>>>>>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> read and write
>>>>>>>>>>>>>>>>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> care
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> difference of sent
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> That
>>>>>>>>>>>>>>>>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Disk
>>>>>>>>>>>>> I/O is
>>>>>>>>>>>>>>>>>>> much more
>>>>>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> Implementing
>>>>>>>>>>>>>>>>>>> it will be some
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> few
>>>>>>>>>>>>> days.
>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>> the deadline
>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> that ,
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> It
>>>>>>>>>>>>> think it's
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> feature we
>>>>>>>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> have to make sure
>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> it
>>>>>>>>>> in
>>>>>>>>>>>>> 4.3,
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> will confuse
>>>>>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> -
>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> 2013/5/31
>>>>>>>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> On
>>>>>>>>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> I
>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> If
>>>>>>>>>>>>> nobody
>>>>>>>>>>>>>>>>>>> object, I will
>>>>>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> The
>>>>>>>>>>>>>>>>> purpose
>>>>>>>>>>>>>>>>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Virtual
>>>>>>>>>>>>>>>>>>> machines are running
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> share
>>>>>>>>>>>>>>>>> strage).
>>>>>>>>>>>>>>>>>>> Because of
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such
>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> iops),
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> one
>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>>>> large disk
>>>>>>>>>>>>>>>>>>>>>>> operation,
>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> other
>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>> running on the
>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> It
>>>>>>>>>> is
>>>>>>>>>>>>>>>>> neccesary
>>>>>>>>>>>>>>>>>>> to set the
>>>>>>>>>>>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> Looking at
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> code I see
>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Qemu
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> libvirt support
>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>> could benefit a
>>>>>>>>>>>>>>>>>>>>>>> lot of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> It's
>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>> strange, in the
>>>>>>>>>>>>>>>>>>>>>>> polling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> IOps,
>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> the throttling
>>>>>>>>>>>>>>>>>>>>>>> side
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Write
>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> usually much
>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> like a
>>>>>>>>>>>>> valid
>>>>>>>>>>>>>>>>>>> use-case where
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> IOps
>>>>>>>>>>>>> vs
>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Since
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>> supports KVM
>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> value
>>>>>>>>>>>>> to at
>>>>>>>>>>>>>>>>>>> least have the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> this
>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>> would be a lot of
>>>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> If
>>>>>>>>>> a
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> write
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> always sum
>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Can
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> explain why you
>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> The
>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> (1) set
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> maximum rate of
>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> (2)
>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> maximum rate
>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> (3)
>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> disk rate
>>>>>>>>>>>>>>>>>>>> (total
>>>>>>>>>>>>>>>>>>>>>>>>> bps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> JIRA
>>>>>>>>>>>>> ticket:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-
>>>>>> 1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 1192
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> FS (I
>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> Merge
>>>>>>>>>>>>>>>>> check
>>>>>>>>>>>>>>>>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> *
>>>>>>>>>> Did
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> check the branch's
>>>>>>>>>>>>>>>>>>>>>>> RAT
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> *
>>>>>>>>>> Are
>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>> new dependencies
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> *
>>>>>>>>>>>>> What
>>>>>>>>>>>>>>>>>>> automated testing
>>>>>>>>>>>>>>>>>>>> (unit
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Unit
>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> *
>>>>>>>>>>>>> What
>>>>>>>>>>>>>>>>>>> testing has been done
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> (1) set
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> bytes rate and
>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> (2)
>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>> deploy,
>>>>>>>>>>>>>>>>> stop,
>>>>>>>>>>>>>>>>>>> start, reboot,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> (3)
>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>> operations,
>>>>>>>>>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Attach,
>>>>>>>>>>>>>>>>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> To
>>>>>>>>>>>>> review
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> code, you can
>>>>>>>>>>>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Best
>>>>>>>>>>>>>>>>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-
>>>> 2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> -
>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> *Mike
>>>>>>>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> *Senior
>>>>>>>>>>>>>>>>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> e:
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>> o:
>>>>>>>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>> Advancing
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> way the world
>>>>>>>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *Mike
>>>>>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *Senior
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> o:
>>>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> Advancing
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *Mike
>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *Senior
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> o:
>>>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> Advancing the
>>>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Mike
>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Senior
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Advancing
>>>>>>>>>> the
>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Mike
>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Senior
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Advancing
>>>>>>>>>> the
>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Senior
>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> Developer, SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Advancing
>>>>>> the
>>>>>>>>>> way
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Advancing
>>>>>> the
>>>>>>>>>> way
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> world
>>>>>>>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Advancing
>>>> the
>>>>>>>>>> way the
>>>>>>>>>>>>>>>>> world
>>>>>>>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> the
>>>>>> way
>>>>>>>>>> the
>>>>>>>>>>>>> world
>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>> way
>>>>>> the
>>>>>>>>>>>>> world
>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way
>>>>>> the
>>>>>>>>>> world
>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>>>> the
>>>>>>>>>> world
>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>>>> world
>>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>>>> world
>>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>>>> world
>>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>> uses
>>>>>>>>>> the
>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>>>> uses
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>> the
>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>> SolidFire
>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *(tm)*****
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*****
>>>>>>>>>>>>
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>>>>>>>
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>>>>>>>
>>>>>>>>>>>> o: 303.746.7302****
>>>>>>>>>>>>
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *(tm)*****
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ****
>>>>>>>>>>>>
>>>>>>>>>>>> ** **
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*****
>>>>>>>>>>>>
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>>>>>>>
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>>>>>>>
>>>>>>>>>>>> o: 303.746.7302****
>>>>>>>>>>>>
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *(tm)*****
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *(tm)*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *(tm)*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *(tm)*
>>>>>
>>>
>
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: John Burwell [mailto:jburwell@basho.com]
> Sent: Monday, June 10, 2013 4:27 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Edison,
>
> Please see my responses in-line below.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 6:52 PM, Edison Su <Ed...@citrix.com> wrote:
>
> >
> >
> >> -----Original Message-----
> >> From: John Burwell [mailto:jburwell@basho.com]
> >> Sent: Monday, June 10, 2013 2:44 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>
> >> Edison,
> >>
> >> Please see my response in-line below.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 4:45 PM, Edison Su <Ed...@citrix.com> wrote:
> >>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: John Burwell [mailto:jburwell@basho.com]
> >>>> Sent: Monday, June 10, 2013 1:25 PM
> >>>> To: dev@cloudstack.apache.org
> >>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>
> >>>> Mike,
> >>>>
> >>>> Yes, a vendor may hand a non-managed plugin. For example, on the
> >>>> secondary storage side, S3 and Swift don't support the concept of
> being
> >>>> managed -- an operator is required to create a bucket and point
> >> CloudStack
> >>>> at it. Another example would be a generic iSCSI device (e.g. OpenFiler,
> >>>> FreeNAS) which CloudStack can not directly manage, but can consume
> a
> >> LUN
> >>>> that an operator has created for it.
> >>>
> >>> Manage or non-managed storage is handled by DataStoreLifeCyclce.
> >>> It depends on storage vendor to implement DataStoreLifeCyclce, which
> >> basically says how to initialize a storage, how to add a storage into a
> cluster
> >> etc.
> >>
> >> So, implementation of the DataStoreLifeCycle interface indicates that
> driver
> >> is capable of managing the device. It does not answer the question of
> >
> > No, the interface only means there is a way to add a storage into cloudstack,
> no matter
> > It's managed or un-managed.
> >
> >> whether or not the administrator/operator wants CloudStack to manage
> the
> >> device. Hence, the need for a flag on the StoragePool to indicate we
> should
> > Cloudstack don't need to care about the storage is managed or un-
> managed, what it cares about
> > Is how to create/delete something on it, it will call storage plugins to do all
> the storage related operations.
> >
> >
> >> be managing it. I am cool with the flag defaulting to true when the driver
> >> supports it, but we must provide the administrator/operator to ability to
> opt-
> >
> > If the storage provider wants to opt-out or wants to do things differently
> based on input from administrator/operator,
> > The storage provider can add parameters in createstoragepoolcmd->details,
> details is a map, which can contain whatever information needed by the
> storage provider.
> >
> > Let's say a storage provider which can support both managed/un-managed
> storage pool, when admin wants a storage pool managed by the storage pool,
> admin can set "managed" = "true" in the details, then
> > The storage provider will know, oh, admin wants a managed storage pool,
> so the DataStoreLifeCyclce->init will create a storage pool dynamically on the
> storage box. While if the "managed" = "false", then DataStoreLifeCyclce->init
> will not create a storage pool, instead, it will check the existence of the
> storage pool, if it's there, then return OK.
> >
> > I don't think cloudstack mgt server needs to know, the storage pool is
> managed/unmanaged, it's up to the implementation of storage provider.
>
> I think we are speaking past each other. I am concerned with the policy the
> operator/administrator intends the implement not the overall capabilities of
> the underlying driver. A driver may be capable of features that an
The admin/operator needs to know what he/she is doing, when adding a storage pool into cloudstack.
Each storage provider has its own way of configure a storage pool, cloudstack doesn't need to know and care about how to do it.
It's a contract between storage provider and admin/operator, cloudstack just helps storage provider and admin to establish this contract.
> operator/administrator has elected not to use. The only place to capture
> that policy is in the management server configuration of the DataStore.
If admin is doing whatever the underlying storage provider doesn't allowed to do, then the storage provider itself will report error, during the createStoragePoolcmd.
>
> >
> >> out. Does this interface provide for a set of administrative credentials
> >> separate from the credentials used to access storage?
> >
> > Yes, createstoragepoolcmd->details, is the place to add whatever
> information when creating a storage pool.
>
> What is the format of the details field? The API does just list it as a non-
> required field in 4.1.
It's a map, it's the place for each storage provider to extend createstoreagepoolcmd api, if the storage provider wants to do that.
CloudStack mgt server just passes the url/details in createstoragepoolcmd to driver's *lifecycle interface, it's driver's responsibility to how to decode the information passed from UI/admin.
>
> >
> >>
> >> Another question comes to mind -- why do we have a Dynamic
> >> StoragePoolType when this interface exists?
> >
> > It shouldn't.
> > Storage provider can return a VolumeTO which can contain whatever
> information about the volume.
> > At the resource side, it may check VolumeTO, if the volume is not a VDI,
> VolumeTO can contain a field called "isManaged", which can be set to true by
> Solidfire's driver, while by default it's false.
> > If isManaged is true, then the resource code can create a SR and create a
> VDI on the fly.
>
> My point is that nothing in the Storage layer should know out an SR or VDI.
Yes, it shouldn't know.
> These are hypervisor concepts -- specifically Xen. The Storage layer should
> only be exposing the discrete operations necessary for the hypervisor to
> determine if an SR and/or VDI are present, as well as, the operations to
> create the structures for them. Fundamentally, it's this circular dependency
> between the Hypervisor and Storage layer that I believe we need to
> eliminate. It is one of the sources of the feature inconsistency across
> hypervisors. For SolidFire, the "managed" flag indicates that the driver
> should programmatically allocate a LUN with a particular IQN -- nothing else.
> The Xen hypervisor plugin should have the knowledge that iSCSI volumes
> need to have an SR created in a particular form on an iSCSI device.
Yes, I think we agree on it.
In CreateAsync, solidfire just creates a LUN.
In VolumeManagerImpl->AttachVolume:
Mgt server will ask solidfire;s driver to return a VolumeTO about the volume,
VolumeTO has a field, called, "isManged", Solidfire's driver needs to set it as true.
On the hypervisor resource code, execute(AttachVolumeCmd) has the following code:
If (volumeTO.isManaged()) {
Get IQN, create SR, create VDI
} {
vid = vid.finduuid()
}
Above code is generic for all the storage vendor, as long as they use the VolumeTO.
>
> >
> >>
> >>>
> >>> Take S3 as an example, if the S3 storage provider provides a
> >> DataStoreLifeCyclce which can create a bucket, then when admin adding a
> S3
> >> storage through API, mgt server will first call DataStoreLifeCyclce, which
> will
> >> create a bucket, then stored in the image_store_details table, then later
> on,
> >> all the referenced to this image store, will contain this bucket information.
> >>
> >> From an S3 perspective, creating a bucket from CloudStack would not a be
> >> security best practice. The general guidance would be for the bucket to
> >> created and associated to a dedicated CloudStack access/secret key pair
> to
> >> properly isolate data. Given this best practice and that only one bucket
> per
> >> S3 instance would be necessary for CloudStack operation, I think most
> >> administrators/operators would want to opt-out CloudStack managing S3
> >> buckets.
> >>
> >>>
> >>> Take an storage box from some vendor as an example, admin wants to
> >> create an ISCSI server on a storage box, which can be used by a
> >> cluster(multiple volumes on a giant LUN ), as long as the storage provider
> can
> >> provide a DataStoreLifeCyclce which can program the storage box, and
> create
> >> an ISCSI server, then stored the information in storage_pool table, then
> all
> >> the referenced to the storage pool will use the dynamically created ISCSI
> LUN.
> >>>
> >>> Take another storage box from some vendor as an example, admin
> wants
> >> to add the whole storage box into cloudstack, which can be used as per
> LUN
> >> per ISCSI, so in this case, DataStoreLifeCyclce won't create an ISCSI server
> >> during the initialize method call. Instead, for every createAsync, it will
> create
> >> an ISCSI LUN, stored the LUN information in volume table and
> volume_detail
> >> table. Whenever the LUN is used by hypervisor, the hypervisor needs to
> take
> >> extra care about the volume.
> >>>
> >>>
> >>>
> >>>
> >>>>
> >>>> From an operational perspective, an administrator may elect to say,
> "Hey,
> >> I
> >>>> know you can manage that device for me, but it is has a wider use than
> >>>> CloudStack can manage. Let me handle it, and just provide the
> >> information
> >>>> CloudStack needs." In this scenario, the operator is accepting a greater
> >>>> management burden in exchange for control. I don't think many will
> elect
> >> to
> >>>> manage these devices themselves, but this ability will be extremely
> >>>> important for the edge cases that require it. Speaking as someone who
> >> has
> >>>> run cloud operations, I can testify to this need -- nothing aggravates
> you
> >>>> more than a system insisting on managing a device or service without
> the
> >>>> opportunity to opt-out
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 10, 2013, at 3:19 PM, Mike Tutkowski
> >> <mi...@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> So, would a storage vendor write a non-CloudStack-managed plug-in
> and
> >>>> maybe
> >>>>> just implement the snapshot feature or something?
> >>>>>
> >>>>> In this case, they'd have to set the hypervisor up ahead of time with a
> >>>>> large portion of their storage, but they'd still get the vendor-snapshot
> >>>>> ability.
> >>>>>
> >>>>> Just trying to undertand this more.
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Edison,
> >>>>>>
> >>>>>> TL;DR Always give operators/system administrators the ability to
> >> manage
> >>>>>> things themselves, if they like.
> >>>>>>
> >>>>>> As more manageable drivers emerge, I think it will be import to allow
> >>>>>> operators to opt-out of CloudStack device management. Therefore,
> a
> >>>> driver
> >>>>>> declares whether or not it can support device management (many,
> in
> >> fact
> >>>>>> most, won't). For those drivers that do support management, allow
> >>>>>> operators to opt-in/out of CloudStack management on a per device
> >> basis.
> >>>> To
> >>>>>> support this notion, we need a flag on the storage_pool table and
> >>>>>> associated classes to indicate whether or not the operator wants
> >>>> CloudStack
> >>>>>> to manage the device.
> >>>>>>
> >>>>>> Does this explanation make more sense?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
> >>>>>>
> >>>>>>> I don't understand why we need to add a "managed" in the UI and
> in
> >>>>>> "storage_pool" table?
> >>>>>>> AFAIK, what you guys trying to do is to enable
> >>>>>> attachVolume/DettachVolume if the volume is created on Solidfire,
> >> right?
> >>>>>>> If the volume is created on Solidfire already, by createAsync, then
> >> why
> >>>>>> we can't just send out all the information about the volume(it's
> >> volumeTO
> >>>>>> which will be returned by each storage driver) to hypervisor, then at
> >> the
> >>>>>> hypervisor side, check the volumeTO, if the volumeTO is managed,
> >> then
> >>>> doing
> >>>>>> whatever necessary operations(create SR, create VDI etc), then
> attach
> >>>> the
> >>>>>> VDI to volume.
> >>>>>>>
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>> Sent: Monday, June 10, 2013 11:10 AM
> >>>>>>>> To: John Burwell
> >>>>>>>> Cc: Edison Su; dev@cloudstack.apache.org
> >>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>
> >>>>>>>> I believe we're on the same page, John.
> >>>>>>>>
> >>>>>>>> I would add a new parameter to the createStoragePool API
> >> command
> >>>> (let's
> >>>>>>>> call it "managed" and it can be true or false). If you pass in "true",
> >>>>>> but
> >>>>>>>> the driver does not support being managed by CloudStack, an
> >>>> exception is
> >>>>>>>> thrown by the driver. If you pass in "false", but the driver does
> not
> >>>>>>>> support being managed outside of CloudStack, an exception is
> >> thrown
> >>>> by
> >>>>>> the
> >>>>>>>> driver. It is possible one could write a driver that can be managed
> >> both
> >>>>>>>> inside or outside of CloudStack (the SolidFire driver requires being
> >>>>>>>> managed only by CloudStack).
> >>>>>>>>
> >>>>>>>> This managed value would end up in the storage_pool table in a
> new
> >>>>>> column
> >>>>>>>> ("managed").
> >>>>>>>>
> >>>>>>>> An example usage: After the storage framework calls on the
> driver's
> >>>>>>>> createAsync method, it will check to see if the driver is managed.
> If
> >>>>>> the
> >>>>>>>> driver is managed, the storage framework will invoke the proper
> >> (new)
> >>>>>>>> command of the hypervisor in use to set up, say, the SR for the
> SAN
> >>>>>> volume.
> >>>>>>>>
> >>>>>>>> There may be other cases when the isManaged() method is
> invoked
> >> on
> >>>> the
> >>>>>>>> driver and - if true is returned - the storage framework sends a
> >>>>>> particular
> >>>>>>>> command to the hypervisor in use.
> >>>>>>>>
> >>>>>>>> ********************
> >>>>>>>>
> >>>>>>>> As far as reworking the default storage plug-in (which sends
> >> commands
> >>>> to
> >>>>>>>> hypervisors) for 4.2, I would check with Edison on his schedule.
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell
> >> <jb...@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> I want to make sure we are on the same page regarding
> isManaged.
> >> I
> >>>> see
> >>>>>>>>> the addition of the following operations:
> >>>>>>>>>
> >>>>>>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates
> >>>> whether
> >>>>>>>> or
> >>>>>>>>> not the driver support/implements the management functions
> >>>>>>>>> * DataStore#isManaged() : boolean -- indicates whether or not a
> >>>> device
> >>>>>> is
> >>>>>>>>> managed by CloudStack
> >>>>>>>>>
> >>>>>>>>> Only DataStores associated with DataStoreDrivers that support
> >>>>>>>> management
> >>>>>>>>> would be able to have the management flag enabled. This
> >> behavior
> >>>>>> allows
> >>>>>>>>> operators to declare their intention to allow CloudStack to
> manage
> >> a
> >>>>>>>>> device. This behavior would need to be exposed as flag on the
> >> HTTP
> >>>>>> API.
> >>>>>>>>> Additionally, this flag would only be settable when a device is
> >>>>>>>>> defined/created in the system (i.e. once a device is managed by
> >>>>>> CloudStack,
> >>>>>>>>> you can't unmanaged it without dissociating it from CloudStack).
> In
> >>>>>> the
> >>>>>>>>> future, we may consider relaxing that rule, but, in the short term,
> it
> >>>>>>>>> greatly decreases the alternative flows.
> >>>>>>>>>
> >>>>>>>>> In terms of removing the dependencies from Storage to
> Hypervisor,
> >> I
> >>>>>>>> would
> >>>>>>>>> really like to see this accomplished in 4.2. It will be much more
> >>>>>>>>> difficult to address this issue post 4.2 because we will have more
> >>>>>> plugins
> >>>>>>>>> to address. I also see this issue as a critical architectural issues
> >>>>>> that
> >>>>>>>>> for the management server. I recommend we identify the
> >>>> dependencies
> >>>>>>>> that
> >>>>>>>>> need to be inverted, and see if we can divide the work across a
> few
> >>>>>> more
> >>>>>>>>> resources.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> >>>>>>>> <mi...@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Sounds good, Edison :)
> >>>>>>>>>
> >>>>>>>>> I can implement John's isManaged() logic and resubmit my code
> for
> >>>>>> review.
> >>>>>>>>>
> >>>>>>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
> >>>>>> requires
> >>>>>>>>> some refactoring of the storage framework, then I could do that.
> >>>>>>>>>
> >>>>>>>>> Also, the default storage plug-in does have hypervisor logic in it
> in
> >>>>>> 4.2.
> >>>>>>>>> I recommend we leave this as is for 4.2 and it can be refactored
> in
> >> 4.3
> >>>>>>>>> with the above work I mentioned.
> >>>>>>>>>
> >>>>>>>>> Are you OK with that, guys?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su
> <Ed...@citrix.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
> >>>>>>>>>> *To:* dev@cloudstack.apache.org
> >>>>>>>>>> *Cc:* Edison Su; John Burwell
> >>>>>>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> Another thing we might want to consider for 4.2 is that the way
> >> the
> >>>>>> code
> >>>>>>>>>> is currently implemented in master, I don't think it's even
> possible
> >>>>>> to
> >>>>>>>>>> write a plug-in that connects storage directly to a VM because
> the
> >>>>>>>>>> CloudStack storage framework will call the "attach" method of
> the
> >>>>>>>>>> hypervisor in use and this method will fail because it assumes
> >> there
> >>>>>> is
> >>>>>>>>>> (talking Xen here) an SR existent and in this situation there isn't
> >>>>>> one.*
> >>>>>>>>>> ***
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> My thinking is that we should ignore the direct-attach-of-
> volume-
> >> to-
> >>>> VM
> >>>>>>>>>> use case for 4.2. I think it will require too much of a rewrite at
> >>>>>> this
> >>>>>>>>>> point in development (fewer than three weeks left until code
> >>>>>>>> freeze).****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> If the plug-in implements an isManaged() method (as John
> >>>> suggested),
> >>>>>>>> the
> >>>>>>>>>> storage framework can invoke the necessary commands on the
> >>>>>>>> hypervisor in
> >>>>>>>>>> use.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> For example, the createAsync method is called. The storage
> plug-
> >> in
> >>>>>>>>>> creates a SAN volume and updates the CS DB. The storage
> >>>> framework
> >>>>>>>> then asks
> >>>>>>>>>> the driver if it's managed. If it is, the storage framework
> executes
> >> a
> >>>>>>>>>> command against the hypervisor in use to have it create, say,
> an
> >> SR
> >>>>>> (with
> >>>>>>>> a
> >>>>>>>>>> single VDI that takes up the entire SR). Now, when the storage
> >>>>>>>> framework
> >>>>>>>>>> executes the "attach" command against the hypervisor, this
> >>>> command
> >>>>>> will
> >>>>>>>>>> work, as well (because that logic assumes the prior existence of
> >> an
> >>>>>> SR and
> >>>>>>>>>> it will, in fact, be present).****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:****
> >>>>>>>>>>
> >>>>>>>>>> Hi Edison and John,****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> I wanted to point out something I thought was relevant to this
> >>>>>>>>>> conversation.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> My plug-in supports both XenServer and ESX(i).****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> These two hypervisors handle their respective hypervisor data
> >>>>>> structures
> >>>>>>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> In XenServer, the SR that resides on the SAN LUN is simply
> >>>> "forgotten"
> >>>>>>>>>> when you issue a remove-SR command to XenServer.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> When you later want to access the contents of that data, it is
> still
> >>>>>>>>>> safely stored on the LUN and can be accessed from XenServer
> >> (from
> >>>> the
> >>>>>>>> same
> >>>>>>>>>> or different cluster (what XenServer calls a resource pool)) by
> >>>>>> creating a
> >>>>>>>>>> new SR that is based on the same IQN and LUN.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> XenServer SRs exist at the cluster level.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
> >>>>>>>>>> destroyed when you issue a remove-datastore command to
> >> ESX(i).
> >>>> In
> >>>>>>>> other
> >>>>>>>>>> words, the contents of the SAN LUN are destroyed.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> ESX(i) datastores exist at the datacenter level.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> The XenServer behavior (not destroying the contents of the
> LUN
> >>>> when
> >>>>>>>> the
> >>>>>>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
> >>>>>> contents
> >>>>>>>>>> of the LUN when the datastore is removed) is not
> desirable.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> I can get around the ESX(i) behavior by removing the
> referenced
> >> IQN
> >>>>>>>> from
> >>>>>>>>>> each host in the cluster when the CS storage framework
> notifies
> >> my
> >>>>>> plug-
> >>>>>>>> in
> >>>>>>>>>> of a detach-volume event. When the CS volume is later
> attached
> >>>> again
> >>>>>>>>>> (either to the same or a different ESX(i) cluster), I can have
> each
> >>>>>> host in
> >>>>>>>>>> the relevant ESX(i) cluster reference the IQN and the datastore
> >> will
> >>>>>> be
> >>>>>>>>>> back again.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> This sequence of events is critical for my plug-in to support
> both
> >>>>>>>>>> environments:****
> >>>>>>>>>>
> >>>>>>>>>> *XenServer flow (events for Driver class):*****
> >>>>>>>>>>
> >>>>>>>>>> createAsync (called right before first attach) = creates SAN
> >>>>>> volume****
> >>>>>>>>>>
> >>>>>>>>>> preAttachVolume = creates SR and creates VDI or just
> introduces
> >>>> SR****
> >>>>>>>>>>
> >>>>>>>>>> (CS volume gets attached and later detached)****
> >>>>>>>>>>
> >>>>>>>>>> postDetachVolume = removes SR (this does not delete data on
> >> SAN
> >>>>>>>> volume)**
> >>>>>>>>>> **
> >>>>>>>>>>
> >>>>>>>>>> deleteAsync (can be called when volume is not attached) =
> >> deletes
> >>>> the
> >>>>>>>> SAN
> >>>>>>>>>> volume****
> >>>>>>>>>>
> >>>>>>>>>> *VMware flow (events for Driver class):*****
> >>>>>>>>>>
> >>>>>>>>>> createAsync (called right before first attach) = creates SAN
> >> volume,
> >>>>>>>>>> creates datastore, and creates VMDK file****
> >>>>>>>>>>
> >>>>>>>>>> preAttachVolume = adds iSCSI target to each host in
> cluster****
> >>>>>>>>>>
> >>>>>>>>>> (CS volume gets attached and later detached)****
> >>>>>>>>>>
> >>>>>>>>>> postDetachVolume = removes iSCSI target from each host in
> >>>> cluster****
> >>>>>>>>>>
> >>>>>>>>>> deleteAsync (can be called when volume is not attached) =
> >> deletes
> >>>>>>>>>> datastore and deletes the SAN volume****
> >>>>>>>>>>
> >>>>>>>>>> ************************
> >>>>>>>>>>
> >>>>>>>>>> So, the way I have it written up here, the driver is talking to the
> >>>>>>>>>> hypervisor. As John has pointed out, this may not be ideal.****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> Whatever we decide on, it needs to be able to support this kind
> of
> >>>>>> flow.
> >>>>>>>>>> :)****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> We could go John's route, where the driver says if it's managed
> or
> >>>>>> not.**
> >>>>>>>>>> **
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> If it is, then the storage framework could send the necessary
> >>>>>> messages to
> >>>>>>>>>> the hypervisor in question at the right times (just migrate the
> >>>>>> hypervisor
> >>>>>>>>>> logic above from the driver to the right parts of the storage
> >>>>>> framework).
> >>>>>>>>>> ****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> This is fine, except that it does not resolve Edison's use case of
> >>>>>>>>>> desiring to have the storage LUN potentially connected directly
> to
> >> a
> >>>>>> VM.*
> >>>>>>>>>> ***
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> I'm not sure we should design around that use case, though. Is
> it
> >>>>>> common
> >>>>>>>>>> enough? Has anyone asked for it? Is it even desirable? How are
> >>>> usage
> >>>>>>>>>> statistics gathered for such volumes? It's nice to offer the
> >>>>>> flexibility,
> >>>>>>>>>> but is it a good idea in this case?****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> Thanks!****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> >>>>>> wrote:**
> >>>>>>>>>> **
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>> Sent: Friday, June 07, 2013 4:08 PM
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> Hi John,
> >>>>>>>>>>>
> >>>>>>>>>>> Yes, you are correct that Xen must layer an SR on top of the
> iSCSI
> >>>>>>>>>> volume
> >>>>>>>>>>> to use it. Same story for VMware: ESX(i) must layer a
> datastore
> >> on
> >>>>>> top
> >>>>>>>>>> of
> >>>>>>>>>>> the iSCSI volume to use it. I look at it like they are layering a
> >>>>>>>>>> clustered
> >>>>>>>>>>> file system on the SAN volume so the hypervisors can share
> >> access
> >>>> to
> >>>>>>>> the
> >>>>>>>>>>> contents of the volume.
> >>>>>>>>>>>
> >>>>>>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called,
> >> the
> >>>>>>>>>> SR or
> >>>>>>>>>>> datastore has already been created (usually manually by an
> >> admin).
> >>>>>>>>>>>
> >>>>>>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in
> an
> >>>>>>>>>>> environment where each CloudStack volume that a user
> creates
> >> is
> >>>>>>>> mapped
> >>>>>>>>>>> to a
> >>>>>>>>>>> single SAN volume (via the SR).
> >>>>>>>>>>>
> >>>>>>>>>>> The question comes down to who should allocate the SR
> >>>> dynamically.
> >>>>>>>>>>>
> >>>>>>>>>>> We could have the storage framework ask the storage plug-in
> if
> >> it is
> >>>>>>>>>>> managed. If it is, then the storage framework could send a
> >>>> message to
> >>>>>>>>>> the
> >>>>>>>>>>> hypervisor in question to create (let's talk Xen here) the SR
> >> ahead
> >>>>>> of
> >>>>>>>>>>> time. Then, when the storage framework next sends the
> attach-
> >>>> volume
> >>>>>>>>>>> command
> >>>>>>>>>>> to the hypervisor, it should work without changes to that
> >>>>>> attach-volume
> >>>>>>>>>>> logic (because - from the point of view of the attach logic - the
> SR
> >>>>>> is
> >>>>>>>>>>> already existent, as expected).
> >>>>>>>>>>>
> >>>>>>>>>>> Now, as Edison has pointed out, this limits the power of the
> >> storage
> >>>>>>>>>>> plug-in. A storage plug-in in this model cannot directly attach
> >>>>>> storage
> >>>>>>>>>> to
> >>>>>>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK.
> We
> >>>> need
> >>>>>>>> to
> >>>>>>>>>>> make a call on that.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Yes, I think we need to address this usage case, and won't limit
> >> the
> >>>>>>>>>> power of the storage plug-in.
> >>>>>>>>>> John, what's your idea to fit this usage case into your
> abstraction?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> If we want to give more power to the storage vendor, we
> would
> >>>> have to
> >>>>>>>>>>> have
> >>>>>>>>>>> the storage framework call into the storage plug-in to send
> the
> >>>>>>>>>> appropriate
> >>>>>>>>>>> attach commands to the hypervisor.
> >>>>>>>>>>>
> >>>>>>>>>>> Let's discuss and come to a consensus (or at least agree on a
> >> path)
> >>>>>> as
> >>>>>>>>>> soon
> >>>>>>>>>>> as we can. :)
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell
> >> <jb...@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> My understanding is the Xen expects an SR structure on all
> >> ISCSI
> >>>>>>>>>> devices
> >>>>>>>>>>>> -- at least that is how I read the code in your patch. Is my
> >>>>>>>>>> understanding
> >>>>>>>>>>>> correct? If so, the Xen plugin should be able to query the
> >> storage
> >>>>>>>>>> device
> >>>>>>>>>>>> to determine the presence of the SR structure and create it
> if it
> >>>>>>>>>> does not
> >>>>>>>>>>>> exist. Am I missing something in the implementation that
> >> makes
> >>>> that
> >>>>>>>>>> type
> >>>>>>>>>>>> of implementation impossible?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Yeah, if a storage vendor wanted to attach a volume
> directly
> >> to a
> >>>>>>>>>> VM in
> >>>>>>>>>>>> 4.2
> >>>>>>>>>>>>> today, it would probably fail because the attach-volume
> logic
> >>>>>>>>>> assumes
> >>>>>>>>>>> the
> >>>>>>>>>>>>> existence of the necessary hypervisor data structure (ex.
> SR
> >> on
> >>>>>>>>>> Xen).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If we wanted to enable such an attach, we could do it the
> way
> >>>>>>>> Edison
> >>>>>>>>>>>>> suggests.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su
> >> <Ed...@citrix.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>>> From: Mike Tutkowski
> >> [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
> >>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> As we only have three weeks until feature freeze, we
> >> should
> >>>>>>>> come
> >>>>>>>>>> to
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>> consensus on this design point as soon as possible.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Right now, if the storage framework asks my driver it is is
> >>>>>>>>>> managed, it
> >>>>>>>>>>>>>>> will say 'yes.' This means the framework will tell the
> driver
> >> to
> >>>>>>>>>>>> perform
> >>>>>>>>>>>>>>> its management activities. This then means the driver
> will
> >> call
> >>>>>>>>>> into
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to
> >>>> perform
> >>>>>>>> the
> >>>>>>>>>>>>>> activity
> >>>>>>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on
> ESX.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The driver doesn't know which hypervisor it's talking to,
> it
> >> just
> >>>>>>>>>>>> sends a
> >>>>>>>>>>>>>>> message to the host to perform the necessary pre-
> attach
> >>>> work.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Could we just expose a method like
> >>>>>>>> "attachVolume/dettachVolume" on
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> PrimaryDataStoreDriver driver?
> >>>>>>>>>>>>>> For most of cases, the implementation of each driver
> would
> >>>> just
> >>>>>>>>>> send
> >>>>>>>>>>>>>> attachvolumecommand/dettachvolumecmd to
> >> hypervisor(we
> >>>> can
> >>>>>>>> put
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> implementation in a base class, so that can be shared by
> all
> >> of
> >>>>>>>>>> this
> >>>>>>>>>>>>>> drivers), and in each hypervisor resource code, which may
> >> just
> >>>>>> call
> >>>>>>>>>>>>>> hypervisor's api to attach the volume to VM, while for
> >> certain
> >>>>>>>>>> storage,
> >>>>>>>>>>>>>> like SolidFire, may need to create a SR first, and create
> >> volume
> >>>>>>>>>> on it,
> >>>>>>>>>>>>>> then call hypervisor's API to attach volume to VM.
> >>>>>>>>>>>>>> While for some other storage vendor, may want to bypass
> >>>>>>>> hypervisor
> >>>>>>>>>>>> during
> >>>>>>>>>>>>>> attaching volume, so inside driver's attachvolume
> >>>> implementation,
> >>>>>>>>>> the
> >>>>>>>>>>>>>> driver can do some magic, such as, directly talk to an agent
> >>>>>>>>>> inside VM
> >>>>>>>>>>>>>> instance, then create a disk inside VM.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> How do you guys think?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su
> >>>> <Ed...@citrix.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>>>>> From: Mike Tutkowski
> >>>> [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> How's about this:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The driver can implement an isManaged() method.
> The
> >>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
> >>>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>> (which is responsible for calling into the hypervisor to
> >>>> attach
> >>>>>>>>>> the
> >>>>>>>>>>>>>> disk)
> >>>>>>>>>>>>>>>>> can call into the hypervisor to create the necessary
> >>>> hypervisor
> >>>>>>>>>> data
> >>>>>>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The problem here is that storage vendor may work
> >>>> differently
> >>>>>>>> with
> >>>>>>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN,
> >> while
> >>>>>>>>>> maybe
> >>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign
> the
> >>>> LUN
> >>>>>>>>>>> directly
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> VM instance, see the discuss(
> >>>>>>>>>>>>>>>> http://mail-
> archives.apache.org/mod_mbox/cloudstack-
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>>>>>>>>>>>>>> %3E
> >>>>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>>>> So I would let storage provider to implement attach
> disk
> >> to
> >>>> VM,
> >>>>>>>>>>>>>> instead of
> >>>>>>>>>>>>>>>> implemented by cloudstack itself.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If that's what you're going for, that works for me. By
> the
> >>>> way,
> >>>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>>>> default storage plug-in (which handles the default
> >> storage
> >>>>>>>>>> behavior
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include
> code
> >> that
> >>>>>>>>>> talks to
> >>>>>>>>>>>>>>>>> hypervisors. You might want to contact him and
> inform
> >> him
> >>>> of
> >>>>>>>>>> your
> >>>>>>>>>>>>>>>> concerns
> >>>>>>>>>>>>>>>>> or that logic (as is) will make it to production.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Please let me know if what I wrote in above (for my
> >>>> solution)
> >>>>>>>>>> is OK
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Please see my responses in-line below.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I
> >> build
> >>>>>>>>>> up
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>> file, but this is the gist of what I've done:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * During a volume-attach operation, after
> >>>>>>>> VolumeManagerImpl
> >>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a
> >> volume, I
> >>>>>>>> have
> >>>>>>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask
> >> the
> >>>>>>>> driver if
> >>>>>>>>>> it
> >>>>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver
> >>>> perform
> >>>>>>>>>>>>>> whatever
> >>>>>>>>>>>>>>>>>>> activity is required. In my case, this includes sending
> a
> >>>>>>>>>>>>>> message to
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> host where the VM is running to have, say
> XenServer,
> >>>> add a
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the
> >> IQN
> >>>> of
> >>>>>>>> the
> >>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the
> >> space
> >>>> it
> >>>>>>>>>> can
> >>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> storage repository). After this, the normal attach-
> >> volume
> >>>>>>>>>>>>>> message is
> >>>>>>>>>>>>>>>> sent
> >>>>>>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> There should be **no** code from a storage driver
> to
> >> a
> >>>>>>>>>> hypervisor.
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>> apologize for the repetition, but we simply can not
> >> have
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> specific code in the storage layer. The circular
> >>>> dependencies
> >>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> two layers are not sustainable in the long term.
> Either
> >> the
> >>>>>>>>>>>>>>>> VirtualManager
> >>>>>>>>>>>>>>>>>> or Xen hypervisor plugin needs to be
> >> refactored/modified
> >>>> to
> >>>>>>>>>>>>>>> coordinate
> >>>>>>>>>>>>>>>>>> volume creation and then populating the SR. Ideally,
> >> we
> >>>> can
> >>>>>>>>>>>>>>>> generalize the
> >>>>>>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>> plugin
> >>>>>>>>>>>>>>>>>> would only implement callbacks to perform the
> attach
> >>>> action
> >>>>>>>> and
> >>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver
> >> should
> >>>>>>>>>> only be
> >>>>>>>>>>>>>>>>>> allocating space and providing information about
> >> contents
> >>>>>>>> (e.g.
> >>>>>>>>>>>>>> space
> >>>>>>>>>>>>>>>>>> available, space consumed, streams to a URI, file
> >> handle
> >>>> for a
> >>>>>>>>>> URI,
> >>>>>>>>>>>>>>>> etc)
> >>>>>>>>>>>>>>>>>> and capabilities.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * The reverse is performed for a detach-volume
> >>>> command.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged()
> in
> >> my
> >>>>>>>>>> driver.
> >>>>>>>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>>>>>> default driver simply does a "return false;". We
> could
> >>>> add a
> >>>>>>>>>> new
> >>>>>>>>>>>>>>>>>> parameter
> >>>>>>>>>>>>>>>>>>> to the createStoragePool API command, if we want,
> >> to
> >>>>>>>> remove
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>>>>>>>> return values in the drivers (although my driver will
> >>>>>> probably
> >>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>> this parameter and always return true since it
> >> wouldn't
> >>>> make
> >>>>>>>>>>>>>> sense
> >>>>>>>>>>>>>>>> for it
> >>>>>>>>>>>>>>>>>>> to ever return false). We'd need another column in
> >> the
> >>>>>>>>>>>>>> storage_pool
> >>>>>>>>>>>>>>>>> table
> >>>>>>>>>>>>>>>>>>> to store this value.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Yes, I think we should have a parameter added to
> the
> >>>>>>>>>>>>>>> createStoragePool
> >>>>>>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to
> be
> >>>>>>>>>> configured
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> management when their underlying drivers support
> it.
> >> To
> >>>>>>>>>> simplify
> >>>>>>>>>>>>>>>> things,
> >>>>>>>>>>>>>>>>>> this flag should only be mutable when the DataStore
> is
> >>>>>>>>>> created. It
> >>>>>>>>>>>>>>>> would be
> >>>>>>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to
> >>>> unmanaged
> >>>>>>>> after
> >>>>>>>>>>>>>>>> creation.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have
> a
> >>>>>>>> working
> >>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>> to review (in that document), if you'd like to check
> it
> >>>> out.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed
> >>>> together to
> >>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor
> >>>> throttled
> >>>>>>>> IOPs
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the
> >> potential
> >>>> for
> >>>>>>>>>>>>>>>> confusion,
> >>>>>>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these
> >> features
> >>>> will
> >>>>>>>>>>>>>> help
> >>>>>>>>>>>>>>>>>>>>> minimize any potential confusion.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will
> >> work
> >>>>>>>>>> itself
> >>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
> >>>>>> hypervisor
> >>>>>>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the
> Google
> >>>> doc I
> >>>>>>>> sent
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> other day to see how the GUI looks for
> >> provisioned
> >>>>>>>> storage
> >>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the
> e-
> >> mail
> >>>>>> list
> >>>>>>>>>>>>>> and we
> >>>>>>>>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the
> Add
> >>>> Disk
> >>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>> dialog.
> >>>>>>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS,
> >> so
> >>>> they
> >>>>>>>>>>> should
> >>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they,
> say,
> >>>> only
> >>>>>>>>>> use
> >>>>>>>>>>>>>> Max
> >>>>>>>>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for
> >>>> storage
> >>>>>>>>>> that
> >>>>>>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size
> >> field,
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> admin can
> >>>>>>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max,
> >> and
> >>>>>>>> Burst
> >>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either
> Wei's
> >>>> feature
> >>>>>>>> or
> >>>>>>>>>>>>>> mine,
> >>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> decided by the admin).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two
> >>>> features
> >>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN
> express
> >> the
> >>>>>>>> IOPS
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>> QoS in
> >>>>>>>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in
> into
> >>>>>>>>>> read/write
> >>>>>>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>>>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on
> the
> >>>>>>>>>>>>>>> Storage/Hypervisor
> >>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the
> >>>> behavior
> >>>>>>>> of
> >>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I
> see
> >>>> the
> >>>>>>>>>>>>>> following
> >>>>>>>>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>>>>>>>> outstanding:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as
> >>>> discrete
> >>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are
> >> expressed
> >>>> as
> >>>>>>>> total
> >>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS
> >>>> attached
> >>>>>>>> to
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for
> >> throttled
> >>>> and
> >>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>> that will permit providers to discriminate
> these
> >>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled
> and
> >>>>>>>>>> provisioned
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would
> >>>> neatly
> >>>>>>>>>> solve
> >>>>>>>>>>>>>>>> items 1
> >>>>>>>>>>>>>>>>>>>>> and 2.
> >>>>>>>>>>>>>>>>>>>>>>> Since the two facilities would not be
> permitted
> >> to
> >>>>>>>> operate
> >>>>>>>>>>>>>>>> together,
> >>>>>>>>>>>>>>>>>>>>> they
> >>>>>>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I
> >> think
> >>>> item
> >>>>>>>> 3
> >>>>>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage
> >> records.
> >>>> As
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>> Item 4,
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>>>>>>>> not familiar with how these two
> enhancements
> >> are
> >>>>>>>>>>> depicted in
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
> >>>>>>>> enhancements
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>> enhancements and ensure they are
> consistent.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >>>>>>>>>>> <us...@gmail.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I was busy working on other issues
> >> (CLOUDSTACK-
> >>>>>>>>>>> 2780/2729,
> >>>>>>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865,
> CLOUDSTACK-
> >>>> 2823 ,
> >>>>>>>>>>>>>>> CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 2875 ) this
> >>>>>>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes
> >>>> tomorrow,
> >>>>>>>> and
> >>>>>>>>>>>>>> ask
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> second
> >>>>>>>>>>>>>>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> >>>>>>>> <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going
> >> with
> >>>> this,
> >>>>>>>>>> John,
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John
> Burwell
> >>>>>>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike
> Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager
> >>>> determining
> >>>>>>>>>>> whether or
> >>>>>>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
> >>>> StoragePoolType,
> >>>>>>>> but an
> >>>>>>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
> >>>> allocate
> >>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an
> >>>> isManaged()
> >>>>>>>>>>> method
> >>>>>>>>>>>>>>>>> added to
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it
> >> returns
> >>>> true,
> >>>>>>>>>>>>>> then the
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or
> >>>> whatever
> >>>>>>>> name)
> >>>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver
> interface,
> >>>> as
> >>>>>>>> well)
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code
> to
> >>>> create,
> >>>>>>>> say
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() :
> >> boolean
> >>>> on
> >>>>>>>>>>>>>>>>> DataStoreDriver.
> >>>>>>>>>>>>>>>>>>>>> Since
> >>>>>>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to
> >> primary
> >>>>>>>> storage,
> >>>>>>>>>>> I
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage
> >> (Storage
> >>>> is a
> >>>>>>>>>>> straw
> >>>>>>>>>>>>>>> man
> >>>>>>>>>>>>>>>>> term
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> something other than volume) methods
> to
> >>>>>>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed
> is
> >> a
> >>>>>>>> mutable
> >>>>>>>>>>>>>>> property
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the
> >> underlying
> >>>>>>>>>>>>>>> DataStoreDriver
> >>>>>>>>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
> >>>>>>>>>>> manageability
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>> devices.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for
> SR,
> >>>> the
> >>>>>>>> Xen
> >>>>>>>>>>>>>> plugin
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> responsible for composing
> >> DataStore/Volume
> >>>>>>>> methods
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> create any
> >>>>>>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR.
> >> There
> >>>>>>>>>> should
> >>>>>>>>>>>>>> be no
> >>>>>>>>>>>>>>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer.
> As
> >> I
> >>>> said
> >>>>>>>>>>>>>> earlier,
> >>>>>>>>>>>>>>>> such
> >>>>>>>>>>>>>>>>>>>>>>>>> circular
> >>>>>>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
> >>>>>>>> unmaintainable
> >>>>>>>>>>> mess.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same
> page
> >>>> with
> >>>>>>>> you.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John
> Burwell
> >>>>>>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a
> >>>> Storage
> >>>>>>>> layer
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> supports n
> >>>>>>>>>>>>>>>>>>>>>>>>>>> devices types with each specific
> behaviors
> >> of
> >>>> m
> >>>>>>>>>>>>>> hypervisors.
> >>>>>>>>>>>>>>>>>> Such
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable
> and
> >>>>>>>> untestable
> >>>>>>>>>>>>>>> beast.
> >>>>>>>>>>>>>>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are
> driven
> >> to
> >>>>>>>> evolve
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager
> >>>> determining
> >>>>>>>>>>> whether
> >>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
> >>>> StoragePoolType,
> >>>>>>>> but
> >>>>>>>>>>> an
> >>>>>>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
> >>>> allocate
> >>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device
> >>>> driver
> >>>>>>>>>> needs
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> indicate
> >>>>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>>>> or not it supports management
> operations.
> >>>>>>>> Finally, I
> >>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to
> >> elect
> >>>> to
> >>>>>>>>>> have
> >>>>>>>>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the
> driver
> >> is
> >>>>>>>> capable
> >>>>>>>>>>>>>>> managing
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to
> leave it
> >>>>>>>>>>> unmanaged).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume
> >> should
> >>>> be
> >>>>>>>> done
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods
> exposed
> >> by
> >>>> the
> >>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>> layer to
> >>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make
> directories,
> >>>>>>>> create a
> >>>>>>>>>>>>>> file,
> >>>>>>>>>>>>>>>> etc).
> >>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of
> the
> >>>> Xen
> >>>>>>>>>>>>>>>> hypervisor, as
> >>>>>>>>>>>>>>>>>>>>> such,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation.
> >>>> From
> >>>>>>>> my
> >>>>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>>>> nothing
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned
> with
> >>>>>>>> content.
> >>>>>>>>>>> From
> >>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It
> provides
> >>>> the
> >>>>>>>> means
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>> query
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the
> content
> >> by
> >>>>>>>> higher-
> >>>>>>>>>>> level
> >>>>>>>>>>>>>>>>> layers
> >>>>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should
> be
> >> a
> >>>>>>>>>>> composition
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based
> on
> >>>> the
> >>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> protocol
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic
> is
> >>>>>>>>>>> implemented
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>> patch,
> >>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code
> >> alone.
> >>>> In
> >>>>>>>>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and
> >> send it
> >>>> to
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor.
> >>>>>>>>>>>>>>>>>> Before
> >>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see
> if
> >>>> we're
> >>>>>>>>>>> dealing
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>> (or
> >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and
> - if
> >> so
> >>>> -
> >>>>>>>>>> send a
> >>>>>>>>>>>>>>>> "Create
> >>>>>>>>>>>>>>>>>>>>> SR"
> >>>>>>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this
> returns
> >> OK,
> >>>> we
> >>>>>>>>>>> would
> >>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>> proceed
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the
> same
> >>>> and
> >>>>>>>> we just
> >>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called
> >> for
> >>>> this
> >>>>>>>>>>>>>> particular
> >>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer
> >> today
> >>>> that
> >>>>>>>>>>> outlines
> >>>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will
> >>>> probably
> >>>>>>>> be
> >>>>>>>>>> of
> >>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> you, as
> >>>>>>>>>>>>>>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you
> >> (you
> >>>>>>>> should
> >>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I
> >> some
> >>>>>>>>>>> thoughts,
> >>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas
> against
> >> the
> >>>>>>>>>> solidfire,
> >>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to
> >> collect
> >>>> them
> >>>>>>>> in a
> >>>>>>>>>>>>>>>>> coherent
> >>>>>>>>>>>>>>>>>>>>> form
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> >>>>>>>> introduces
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in
> just
> >>>>>>>> introduces
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that
> leverages
> >>>> the
> >>>>>>>> plug-in
> >>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>> incomplete,
> >>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a
> >> managed
> >>>> iSCSI
> >>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've
> been
> >>>>>>>> spending on
> >>>>>>>>>>>>>> this.
> >>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>> Do
> >>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we
> should
> >>>>>>>> accomplish
> >>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>> you're
> >>>>>>>>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM,
> John
> >>>> Burwell
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> >>>>>>>> introduces
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> >>>>>>>> management
> >>>>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>>>>> (i.e.
> >>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an
> >> operator
> >>>> want
> >>>>>>>> it
> >>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we
> should
> >>>> end up
> >>>>>>>> with
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> semantic
> >>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device
> to
> >> be
> >>>>>>>> managed.
> >>>>>>>>>>> I
> >>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the
> storage
> >>>> types
> >>>>>>>> from
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike
> >>>> Tutkowski
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below
> in
> >> red.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM,
> >> John
> >>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the
> >>>> following
> >>>>>>>>>>> questions
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> me
> >>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
> >> iSCSI
> >>>>>>>> LUNs
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to
> a
> >>>> single
> >>>>>>>> LUN
> >>>>>>>>>>> (LUN
> >>>>>>>>>>>>>>> 0).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
> >> SAN
> >>>>>>>> Volumes
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack
> >> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> >>>>>>>> management
> >>>>>>>>>>>>>>> server or
> >>>>>>>>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire
> plug-
> >> in,
> >>>> a
> >>>>>>>> SAN
> >>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>> (same
> >>>>>>>>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management
> server
> >>>> (via
> >>>>>>>> the
> >>>>>>>>>>> plug-in)
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to
> a
> >>>>>>>> hypervisor.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the
> >> SolidFire
> >>>> plug-
> >>>>>>>> in,
> >>>>>>>>>>> but
> >>>>>>>>>>>>>>>> still
> >>>>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can
> do
> >>>> this
> >>>>>>>> already
> >>>>>>>>>>>>>>> today
> >>>>>>>>>>>>>>>>>> (prior
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN
> >>>> volume
> >>>>>>>> and -
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> case -
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN
> >> volume.
> >>>>>>>> While
> >>>>>>>>>>> you
> >>>>>>>>>>>>>>> can do
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce
> >>>> storage
> >>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> >>>>>>>> management
> >>>>>>>>>>> server
> >>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is
> used,
> >> the
> >>>>>>>> SAN
> >>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via
> the
> >>>> plug-
> >>>>>>>> in).
> >>>>>>>>>>> There
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping
> >> between
> >>>> a
> >>>>>>>> SAN
> >>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>> and a
> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any
> storage
> >>>> vendor
> >>>>>>>> that
> >>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>> true
> >>>>>>>>>>>>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these
> >>>> pieces
> >>>>>>>> are
> >>>>>>>>>>>>>> related
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM,
> Mike
> >>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we
> ensure
> >>>> that
> >>>>>>>>>>> multiple
> >>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the
> >> underlying
> >>>>>>>> storage."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we
> >> need
> >>>> to
> >>>>>>>> map a
> >>>>>>>>>>>>>>> single
> >>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We
> >>>> cannot
> >>>>>>>> have
> >>>>>>>>>>>>>>> multiple
> >>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume
> and
> >> still
> >>>>>>>>>>> guarantee
> >>>>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a
> single
> >>>> SAN
> >>>>>>>>>>> volume
> >>>>>>>>>>>>>>> house
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can
> do
> >>>> that
> >>>>>>>> today
> >>>>>>>>>>>>>>> without
> >>>>>>>>>>>>>>>>> any of
> >>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM,
> >>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will
> allocate
> >> a
> >>>> SAN
> >>>>>>>>>>> volume
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes
> will
> >> be
> >>>>>>>> created."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail
> on
> >> this.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41
> PM,
> >>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part
> >> which
> >>>>>>>>>>> concerns me
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into
> >>>> storage
> >>>>>>>> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM,
> >> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the
> >> iSCSI
> >>>> type
> >>>>>>>> is
> >>>>>>>>>>> today
> >>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer
> >> (when
> >>>> you
> >>>>>>>> have
> >>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> PreSetup an
> >>>>>>>>>>>>>>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your
> iSCSI
> >>>>>>>> volume
> >>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> VMware, it
> >>>>>>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your
> iSCSI
> >>>>>>>> volume
> >>>>>>>>>>> from
> >>>>>>>>>>>>>>> KVM,
> >>>>>>>>>>>>>>>>> it uses
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and
> >> Edison's
> >>>>>>>> thinking
> >>>>>>>>>>>>>>> here
> >>>>>>>>>>>>>>>>> was to
> >>>>>>>>>>>>>>>>>>>>> make a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this
> >>>> dynamic
> >>>>>>>> ability
> >>>>>>>>>>>>>>> Edison
> >>>>>>>>>>>>>>>>> added
> >>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it
> should
> >> be
> >>>>>>>> more
> >>>>>>>>>>>>>>> specificy,
> >>>>>>>>>>>>>>>>> though:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27
> >> PM,
> >>>>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> >>>>>>>> shouldn't
> >>>>>>>>>>> know
> >>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>> care
> >>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be
> >> able
> >>>> to
> >>>>>>>> answer
> >>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern
> here
> >> is
> >>>> that
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage
> so
> >>>> that
> >>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to
> >> removing
> >>>> that
> >>>>>>>>>>> constraint
> >>>>>>>>>>>>>>>>> and just
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just
> >>>> need a
> >>>>>>>> way
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new
> >> requirement
> >>>> and
> >>>>>>>>>>> perform
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> 1:24
> >> PM,
> >>>>>>>> John
> >>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-
> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10
> PM,
> >>>> Mike
> >>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture
> this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while
> CloudStack
> >>>> may
> >>>>>>>> be
> >>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it
> >>>> unmanaged by
> >>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> (e.g. the
> >>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and
> the
> >>>>>>>> operator has
> >>>>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does
> my
> >>>>>>>> reasoning
> >>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure
> how
> >>>>>>>> creating a
> >>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>> volume via
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> >>>>>>>> hypervisor)
> >>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> work unless
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the
> >> form
> >>>> of,
> >>>>>>>> say,
> >>>>>>>>>>> an
> >>>>>>>>>>>>>>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that,
> >>>> independent
> >>>>>>>> of
> >>>>>>>>>>>>>>>>> CloudStack, an
> >>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to
> >>>> CloudStack,
> >>>>>>>> and
> >>>>>>>>>>>>>>> exposes it
> >>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of
> >> that
> >>>> LUN
> >>>>>>>> over
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate
> >>>> anymore
> >>>>>>>> LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic
> stands
> >>>> prior
> >>>>>>>> to my
> >>>>>>>>>>>>>>>>> changes, we
> >>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does
> >> not
> >>>> have
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we
> >> should
> >>>>>>>> maybe
> >>>>>>>>>>> have
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN
> volume
> >>>> needs
> >>>>>>>>>>> support
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent
> code
> >>>>>>>> specifically
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent
> >> code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> >>>>>>>> management
> >>>>>>>>>>> plugin
> >>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> >>>>>>>> determine
> >>>>>>>>>>>>>>> available
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would
> allow
> >>>> the
> >>>>>>>>>>>>>>> interrogation of
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The
> storage
> >>>>>>>> device
> >>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>> shouldn't
> >>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen
> SR -
> >> -
> >>>>>>>> simply be
> >>>>>>>>>>> able
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> answer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> >> 1:01
> >>>> PM,
> >>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as
> >> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers
> >> the
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage).
> Once
> >>>> this is
> >>>>>>>>>>> done, a
> >>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates
> a
> >> Disk
> >>>>>>>> Offering
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> references
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created
> Primary
> >>>>>>>> Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user
> creates
> >> a
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> volume. This
> >>>>>>>>>>>>>>>>>>>>> leads
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user
> attaches
> >>>> the
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> volume to a
> >>>>>>>>>>>>>>>>>>>>> VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> >>>>>>>> framework
> >>>>>>>>>>> calling
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system
> >> (in
> >>>> my
> >>>>>>>> case, a
> >>>>>>>>>>>>>>> SAN).
> >>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row
> with
> >>>>>>>> applicable
> >>>>>>>>>>> data
> >>>>>>>>>>>>>>>>> (like the
> >>>>>>>>>>>>>>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is
> only
> >>>>>>>> invoked if
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state.
> After
> >> the
> >>>>>>>> attach,
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a
> >> detach
> >>>> disk)
> >>>>>>>> and
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this
> SAN
> >>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-
> attach
> >>>> logic
> >>>>>>>> is run
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> detects
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs
> >> "assistance"
> >>>> in
> >>>>>>>> the
> >>>>>>>>>>> form
> >>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013
> at
> >>>> 12:54
> >>>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we
> >> are in
> >>>>>>>> sync on
> >>>>>>>>>>>>>>>>> terminology,
> >>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers
> to
> >> the
> >>>>>>>> physical
> >>>>>>>>>>>>>>>>> allocation on
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say
> >> 'volume',
> >>>> I
> >>>>>>>> try to
> >>>>>>>>>>>>>>> mean
> >>>>>>>>>>>>>>>>> 'SAN
> >>>>>>>>>>>>>>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the
> >> 'volume'
> >>>> the
> >>>>>>>> end
> >>>>>>>>>>> user
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> make in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack
> >> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013
> at
> >>>> 12:50
> >>>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here
> >> may
> >>>>>>>> very
> >>>>>>>>>>> well
> >>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>> sense, but
> >>>>>>>>>>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we
> should
> >>>> draw
> >>>>>>>> Edison
> >>>>>>>>>>> in on
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to
> >> suggest
> >>>> the
> >>>>>>>>>>>>>>> approach I
> >>>>>>>>>>>>>>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4,
> 2013
> >> at
> >>>> 12:42
> >>>>>>>> PM,
> >>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we
> are
> >>>>>>>> combining
> >>>>>>>>>>> two
> >>>>>>>>>>>>>>>>> distinct
> >>>>>>>>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
> and
> >>>> storage
> >>>>>>>>>>> protocols.
> >>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one
> >> allows
> >>>> the
> >>>>>>>>>>> system to
> >>>>>>>>>>>>>>>>>> create/delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the
> >> other
> >>>>>>>> requires
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> volume to
> >>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the
> >>>> CloudStack
> >>>>>>>>>>> context.
> >>>>>>>>>>>>>>> To
> >>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology,
> >> volume,
> >>>> in
> >>>>>>>> these
> >>>>>>>>>>>>>>>>> definitions,
> >>>>>>>>>>>>>>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the
> >>>> device,
> >>>>>>>>>>> correct?
> >>>>>>>>>>>>>>>>> Minimally,
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate
> with a
> >>>>>>>> storage
> >>>>>>>>>>> device
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> move bits
> >>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete
> >> bits,
> >>>> etc.
> >>>>>>>>>>>>>>> Optionally, a
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
> >>>> CloudStack.
> >>>>>>>>>>> Therefore,
> >>>>>>>>>>>>>>>>> we can have
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we
> >> store a
> >>>> Xen
> >>>>>>>> SR,
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> can have a
> >>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> >>>>>>>> CloudStack is
> >>>>>>>>>>>>>>> capable
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally,
> >>>> while
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>> be able
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may
> chose
> >> to
> >>>>>>>> leave it
> >>>>>>>>>>>>>>>>> unmanaged by
> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many
> >>>> services,
> >>>>>>>> and
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> operator has
> >>>>>>>>>>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> >>>>>>>> CloudStack).
> >>>>>>>>>>> Does
> >>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my
> >> thoughts
> >>>>>>>> above
> >>>>>>>>>>> are
> >>>>>>>>>>>>>>>>> reasonable, it
> >>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the
> >> management
> >>>>>>>>>>> concerns
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device
> with
> >> an
> >>>>>>>>>>> attached
> >>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
> >>>> CloudStack,
> >>>>>>>> and
> >>>>>>>>>>>>>>> establish
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a
> >> device
> >>>>>>>> separate
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> associated
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in
> >> place,
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> drivers can
> >>>>>>>>>>>>>>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and
> when
> >>>> they
> >>>>>>>>>>> encounter
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
> >>>> operations
> >>>>>>>>>>> exposed
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>> the driver to
> >>>>>>>>>>>>>>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these
> >>>> thoughts/concepts
> >>>>>>>> make
> >>>>>>>>>>>>>>> sense,
> >>>>>>>>>>>>>>>>> then we can
> >>>>>>>>>>>>>>>>>>>>> sit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed
> >> design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> >> 5:25
> >>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the
> >>>> difference
> >>>>>>>>>>> between
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> current
> >>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The
> >> admin
> >>>>>>>> has to
> >>>>>>>>>>> go in
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type.
> At
> >> this
> >>>>>>>> point in
> >>>>>>>>>>> time,
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system
> (it
> >> is
> >>>> pre-
> >>>>>>>>>>> allocated).
> >>>>>>>>>>>>>>>>> Future
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the
> SR
> >>>> that
> >>>>>>>> was
> >>>>>>>>>>> created
> >>>>>>>>>>>>>>>>> behind the
> >>>>>>>>>>>>>>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type:
> The
> >>>> admin
> >>>>>>>> has
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>> go in
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will
> >>>> create
> >>>>>>>> and
> >>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>> volumes on
> >>>>>>>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as
> is
> >>>>>>>> enabled via
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a
> >> CloudStack
> >>>>>>>> volume
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>> created,
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> create a
> >>>> new
> >>>>>>>>>>> volume.
> >>>>>>>>>>>>>>> After
> >>>>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>>>>>> done,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor
> in
> >>>>>>>> question is
> >>>>>>>>>>> called.
> >>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point
> >> because
> >>>> the
> >>>>>>>>>>> volume
> >>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must
> be
> >>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>> at
> >>>>>>>> 3:21
> >>>>>>>>>>> PM,
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new
> >>>> terms,
> >>>>>>>> so I
> >>>>>>>>>>> should
> >>>>>>>>>>>>>>>>> probably
> >>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage:
> >>>> Storage
> >>>>>>>> that is
> >>>>>>>>>>>>>>> pre-
> >>>>>>>>>>>>>>>>> allocated
> >>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a
> >> SAN),
> >>>> then
> >>>>>>>> a
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage
> (ex.
> >>>>>>>> XenServer
> >>>>>>>>>>> SR),
> >>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> >>>>>>>> CloudStack.
> >>>>>>>>>>> Disks
> >>>>>>>>>>>>>>> (VDI)
> >>>>>>>>>>>>>>>>> are later
> >>>>>>>>>>>>>>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure
> as
> >>>>>>>> needed. In
> >>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>> cases, the
> >>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> data
> >>>>>>>> structure is
> >>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>> in place
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >> hypervisor
> >>>> data
> >>>>>>>>>>> structure
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> the VM in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> storage:
> >>>>>>>> Storage
> >>>>>>>>>>> that is
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent
> >>>> storage,
> >>>>>>>> this
> >>>>>>>>>>> could
> >>>>>>>>>>>>>>> be a
> >>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>> (not
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself).
> The
> >>>>>>>> hypervisor
> >>>>>>>>>>> data
> >>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>> must be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is
> >>>> performed
> >>>>>>>>>>> because
> >>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>> types of
> >>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a
> >>>> hypervisor
> >>>>>>>> data
> >>>>>>>>>>>>>>>>> structure by an
> >>>>>>>>>>>>>>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say,
> an
> >> SR
> >>>> on
> >>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI
> within
> >> the
> >>>> SR
> >>>>>>>> to the
> >>>>>>>>>>> VM
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>> at
> >>>>>>>> 3:13
> >>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>> John Burwell
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >>>>>>>>>>> implementation of
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen
> >> ISCSI
> >>>>>>>> which
> >>>>>>>>>>> why I
> >>>>>>>>>>>>>>> ask
> >>>>>>>>>>>>>>>>> about the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the
> >> question --
> >>>>>>>> what is
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> definition
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> 2013,
> >> at
> >>>> 5:10
> >>>>>>>> PM,
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I
> >> know,
> >>>> the
> >>>>>>>> iSCSI
> >>>>>>>>>>> type
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> uniquely
> >>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set
> up
> >>>>>>>> Primary
> >>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>>>> directly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows
> you
> >> to
> >>>>>>>> skip the
> >>>>>>>>>>>>>>> step of
> >>>>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>> repository
> >>>>>>>> based
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I
> think
> >>>> this is
> >>>>>>>> only
> >>>>>>>>>>>>>>>>> supported for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors,
> >> you
> >>>>>>>> must
> >>>>>>>>>>> first go
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really
> >>>> know
> >>>>>>>> what
> >>>>>>>>>>> RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >>>> 2013
> >>>>>>>> at
> >>>>>>>>>>> 2:13
> >>>>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading
> >>>> through
> >>>>>>>> the
> >>>>>>>>>>> code,
> >>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>> is the
> >>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >> types?
> >>>>>>>> Why
> >>>>>>>>>>> isn't
> >>>>>>>>>>>>>>> RBD
> >>>>>>>>>>>>>>>>> considered
> >>>>>>>>>>>>>>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> >> 2013,
> >>>> at
> >>>>>>>> 3:46
> >>>>>>>>>>> PM,
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new
> >> type
> >>>> of
> >>>>>>>>>>> storage is
> >>>>>>>>>>>>>>>>> defined in
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called
> >>>> Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public
> static
> >>>> enum
> >>>>>>>>>>>>>>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Filesystem(false),
> >>>>>>>> //
> >>>>>>>>>>> local
> >>>>>>>>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> NetworkFilesystem(true),
> >>>>>>>>>>>>>>> //
> >>>>>>>>>>>>>>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IscsiLUN(true),
> >>>> //
> >>>>>>>>>>> shared
> >>>>>>>>>>>>>>> LUN,
> >>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Iscsi(true),
> >> //
> >>>> for
> >>>>>>>> e.g.,
> >>>>>>>>>>> ZFS
> >>>>>>>>>>>>>>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> ISO(false),
> >> //
> >>>> for
> >>>>>>>> iso
> >>>>>>>>>>> image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> LVM(false),
> >> //
> >>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> VMFS(true),
> >> //
> >>>>>>>>>>> VMware
> >>>>>>>>>>>>>>> VMFS
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> PreSetup(true), //
> >>>>>>>> for
> >>>>>>>>>>>>>>>>> XenServer, Storage
> >>>>>>>>>>>>>>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> EXT(false),
> >> //
> >>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Dynamic(true);
> >>>> //
> >>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean
> >>>> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> StoragePoolType(boolean
> >>>>>>>>>>>>>>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> this.shared
> >> =
> >>>>>>>> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public
> >> boolean
> >>>>>>>>>>> isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return
> >> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun 3,
> >>>>>>>> 2013 at
> >>>>>>>>>>> 1:41
> >>>>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >> example,
> >>>>>>>> let's say
> >>>>>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> to
> >>>>>>>> leverage its
> >>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>> Service
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-
> wide
> >>>>>>>> storage, as
> >>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>> They would
> >>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> as
> >> I've
> >>>>>>>> made
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-
> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> support
> >>>> their
> >>>>>>>> plug-
> >>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun
> >>>> 3,
> >>>>>>>> 2013
> >>>>>>>>>>> at
> >>>>>>>>>>>>>>> 1:39
> >>>>>>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh,
> sorry
> >> to
> >>>>>>>> imply
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> XenServer code
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>> XenServer
> >>>>>>>>>>> attach
> >>>>>>>>>>>>>>> logic is
> >>>>>>>>>>>>>>>>> now aware
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and
> >>>> SolidFire is
> >>>>>>>> an
> >>>>>>>>>>>>>>>>> implementation of
> >>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> storage is
> >>>> new
> >>>>>>>> to 4.2
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison
> >>>> created
> >>>>>>>> a
> >>>>>>>>>>> new
> >>>>>>>>>>>>>>>>> framework that
> >>>>>>>>>>>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> volumes
> >>>>>>>>>>> dynamically.
> >>>>>>>>>>>>>>>>> However, when I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April,
> >> we
> >>>>>>>> realized
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> was not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> nothing
> >>>>>>>> CloudStack
> >>>>>>>>>>> could
> >>>>>>>>>>>>>>> do
> >>>>>>>>>>>>>>>>> with these
> >>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> changed
> >> to
> >>>>>>>>>>> recognize
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> new type of
> >>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> hypervisor
> >>>> data
> >>>>>>>>>>> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Mon,
> >>>> Jun 3,
> >>>>>>>> 2013
> >>>>>>>>>>> at
> >>>>>>>>>>>>>>> 1:28
> >>>>>>>>>>>>>>>>> PM, John
> >>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>> generally
> >>>>>>>> odd
> >>>>>>>>>>> to me
> >>>>>>>>>>>>>>>>> that any
> >>>>>>>>>>>>>>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> understand or
> >>>>>>>> care
> >>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>> details. I
> >>>>>>>>>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> expose
> >> a
> >>>> set
> >>>>>>>> of
> >>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>> that can be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> implementations to
> >>>>>>>>>>>>>>>>> allocate
> >>>>>>>>>>>>>>>>>> space/create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>> invert
> >>>>>>>> this
> >>>>>>>>>>>>>>>>> dependency, we are
> >>>>>>>>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> problem
> >>>> that
> >>>>>>>> will
> >>>>>>>>>>> make
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> enhance.
> >>>> Am
> >>>>>>>> I
> >>>>>>>>>>>>>>>>> understanding that the
> >>>>>>>>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> located
> >> in
> >>>> the
> >>>>>>>>>>>>>>>>> CitrixResourceBase
> >>>>>>>>>>>>>>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Mon,
> >>>> Jun 3,
> >>>>>>>>>>> 2013 at
> >>>>>>>>>>>>>>> 3:12
> >>>>>>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> >> delve
> >>>> into
> >>>>>>>> this
> >>>>>>>>>>> in a
> >>>>>>>>>>>>>>> bit
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Prior
> >> to
> >>>> 4.2
> >>>>>>>> and
> >>>>>>>>>>> aside
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> one setup
> >>>>>>>>>>>>>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> first
> >>>> create
> >>>>>>>> a
> >>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> set
> >> up
> >>>> a
> >>>>>>>> data
> >>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>> to make
> >>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> repository on
> >>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>> or a
> >>>>>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> shared
> >>>> this
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> With
> >>>> Edison's
> >>>>>>>> new
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> framework,
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> you
> >>>> can
> >>>>>>>> easily
> >>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>> a 1:1
> >>>>>>>>>>>>>>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> volume
> >>>> and
> >>>>>>>> the
> >>>>>>>>>>> VM's
> >>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>> (necessary
> >>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
> >> can
> >>>> now
> >>>>>>>> write
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>> plug-
> >>>>>>>>>>>>>>>>> in that is
> >>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> volumes
> >>>> as
> >>>>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>> problem
> >>>>>>>> that
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> deleting
> >>>> the
> >>>>>>>>>>>>>>> hypervisor-
> >>>>>>>>>>>>>>>>> specific data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> >>>> being
> >>>>>>>> the
> >>>>>>>>>>> case,
> >>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>> enhancing
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> worked
> >>>> out
> >>>>>>>> and
> >>>>>>>>>>>>>>>>> submitted. I've got
> >>>>>>>>>>>>>>>>>>>>> ESX(i)
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> we
> >>>> extend
> >>>>>>>> the
> >>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Does
> >> that
> >>>>>>>> help a
> >>>>>>>>>>> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >> Mon,
> >>>> Jun
> >>>>>>>> 3,
> >>>>>>>>>>> 2013 at
> >>>>>>>>>>>>>>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> >> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >>>> storage
> >>>>>>>>>>> plug-in -
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>> itself - is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >>>> issue is
> >>>>>>>> with
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> volume-attach
> >>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> framework
> >>>>>>>> calls
> >>>>>>>>>>> into
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> plug-in to
> >>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> when
> >>>> the
> >>>>>>>> time
> >>>>>>>>>>>>>>> comes to
> >>>>>>>>>>>>>>>>> attach the
> >>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> has
> >> to
> >>>> be
> >>>>>>>> smart
> >>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>> to recognize
> >>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (where
> >>>> the
> >>>>>>>>>>> volume
> >>>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>>>> just been
> >>>>>>>>>>>>>>>>>>>>> created)
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> repository
> >>>>>>>> (for
> >>>>>>>>>>>>>>>>> XenServer) or a
> >>>>>>>>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> volume
> >>>> that
> >>>>>>>> was
> >>>>>>>>>>> just
> >>>>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I've
> >>>> been
> >>>>>>>>>>> spending
> >>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>> of my time
> >>>>>>>>>>>>>>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> the
> >>>> agent
> >>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Does
> >>>> that
> >>>>>>>> clear it
> >>>>>>>>>>> up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >> Mon,
> >>>>>>>> Jun 3,
> >>>>>>>>>>> 2013
> >>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>> 12:48 PM,
> >>>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Can
> >>>> you
> >>>>>>>> explain
> >>>>>>>>>>>>>>> why
> >>>>>>>>>>>>>>>>> the the storage
> >>>>>>>>>>>>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> Jun
> >>>> 3,
> >>>>>>>> 2013,
> >>>>>>>>>>> at
> >>>>>>>>>>>>>>> 1:21
> >>>>>>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Yes,
> >>>>>>>>>>> ultimately I
> >>>>>>>>>>>>>>>>> would like to
> >>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> supports.
> >>>>>>>> I
> >>>>>>>>>>> think
> >>>>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>> just out of
> >>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Right
> >>>>>>>> now this
> >>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> regards
> >>>>>>>> to 4.2
> >>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>> freeze, I
> >>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> as
> >>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Also,
> >>>> just
> >>>>>>>> to
> >>>>>>>>>>> be
> >>>>>>>>>>>>>>> clear,
> >>>>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> plan
> >> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> support
> >>>>>>>>>>> Compute
> >>>>>>>>>>>>>>>>> Offerings post
> >>>>>>>>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >>>> Mon,
> >>>>>>>> Jun 3,
> >>>>>>>>>>>>>>> 2013
> >>>>>>>>>>>>>>>>> at 11:14 AM,
> >>>>>>>>>>>>>>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Is
> >>>> there
> >>>>>>>> any
> >>>>>>>>>>> plan
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> supporting
> >>>>>>>>>>>>>>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> --
> >> ---
> >>>>>>>> Original
> >>>>>>>>>>>>>>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> From:
> >>>>>>>> "Mike
> >>>>>>>>>>>>>>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> To:
> >>>>>>>>>>>>>>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Sent:
> >>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>> June
> >>>>>>>>>>>>>>>>> 3, 2013
> >>>>>>>>>>>>>>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Subject:
> >>>>>>>> Re:
> >>>>>>>>>>>>>>> [MERGE]
> >>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>> agree
> >>>>>>>> on
> >>>>>>>>>>>>>>> merging
> >>>>>>>>>>>>>>>>> Wei's feature
> >>>>>>>>>>>>>>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> If
> >>>> his
> >>>>>>>> feature
> >>>>>>>>>>> is
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> KVM only,
> >>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> KVM in
> >>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>> Mon,
> >>>>>>>> Jun
> >>>>>>>>>>> 3,
> >>>>>>>>>>>>>>> 2013
> >>>>>>>>>>>>>>>>> at 8:55 AM,
> >>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> For
> >>>>>>>> the
> >>>>>>>>>>> billing,
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>> no one works
> >>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> the
> >>>>>>>> billing
> >>>>>>>>>>> by
> >>>>>>>>>>>>>>>>> themselves. They
> >>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> disk_offering of
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> VMs and
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> to
> >>>> tell
> >>>>>>>> user
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> exact
> >>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> rate
> >>>>>>>>>>> limitation
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> nics as
> >>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> is a
> >>>>>>>> part of
> >>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Sorry
> >>>>>>>> my
> >>>>>>>>>>>>>>>>> misunstand the second
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Agree
> >>>>>>>> with
> >>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>> you said about
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>> 2013/6/3
> >>>>>>>>>>> John
> >>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>> On
> >>>>>>>> Jun 3,
> >>>>>>>>>>> 2013,
> >>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>> 2:13 AM, Wei
> >>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>> Hi
> >>>>>>>> John,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>> I
> >>>>>>>> hope
> >>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>> aswer helps you.
> >>>>>>>>>>>>>>>>>>>>> I am
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>> (1) I
> >>>>>>>>>>> think
> >>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>> depend
> >>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>> limitation.
> >>>>>>>>>>>>>>> Please
> >>>>>>>>>>>>>>>>> review
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>> get
> >>>>>>>> the
> >>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>> virtual
> >>>>>>>>>>>>>>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> Going by
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> AWS
> >>>>>>>>>>>>>>>>> model,
> >>>>>>>>>>>>>>>>>> customers
> >>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>>>> as well as,
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/).
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> option to
> >>>>>>>>>>>>>>> employ
> >>>>>>>>>>>>>>>>> similar cost
> >>>>>>>>>>>>>>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> billing
> >>>>>>>>>>> model
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the current
> >>>>>>>>>>>>>>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>> (2)
> >>>>>>>> Do
> >>>>>>>>>>> you
> >>>>>>>>>>>>>>> mean
> >>>>>>>>>>>>>>>>> IOPS runtime
> >>>>>>>>>>>>>>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>> limitation for
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>>>>>> virtual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>> not support
> >>>>>>>>>>>>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>> (computer
> >>>>>>>>>>>>>>>>> offering or disk
> >>>>>>>>>>>>>>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> I
> >>>>>>>> meant at
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> Java interface
> >>>>>>>>>>>>>>>>>>>>> level.
> >>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>> we
> >>>>>>>> more
> >>>>>>>>>>>>>>>>> generalize allocation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> describe
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> service guarantees
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>> (3)
> >>>>>>>> It is a
> >>>>>>>>>>>>>>> good
> >>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>> Maybe it
> >>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>> disk_io_throttling as Mike
> >>>>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>> type, I
> >>>>>>>>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> I
> >>>> will
> >>>>>>>>>>> expand
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> my thoughts
> >>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> touch
> >>>>>>>>>>> points
> >>>>>>>>>>>>>>>>> between these two
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>> will
> >>>>>>>> need
> >>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>> merged before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>> coordination
> >>>>>>>>>>>>>>>>> between the
> >>>>>>>>>>>>>>>>>> branches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>> disk_io_throttling)
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>> -
> >>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>> 2013/6/3
> >>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> The
> >>>>>>>>>>> things I
> >>>>>>>>>>>>>>>>> want to
> >>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> 1)
> >>>>>>>> Is
> >>>>>>>>>>> there
> >>>>>>>>>>>>>>>>> value in
> >>>>>>>>>>>>>>>>>> capturing
> >>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> data
> >>>>>>>>>>> model
> >>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> 2)
> >>>>>>>>>>> Should
> >>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>> be a common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>> provisioning
> >>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> 3)
> >>>>>>>> How
> >>>>>>>>>>> are
> >>>>>>>>>>>>>>>>> conflicting
> >>>>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> storage device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> is
> >>>>>>>> lead
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> believe (and
> >>>>>>>>>>>>>>>>>> billed)
> >>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> storage
> >>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> could
> >>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>> consistent
> >>>>>>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>>>>>> between a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> time,
> >>>>>>>>>>> but a
> >>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>> modification
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> -
> >>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> On
> >>>>>>>> Jun
> >>>>>>>>>>> 2,
> >>>>>>>>>>>>>>> 2013,
> >>>>>>>>>>>>>>>>> at 8:38 PM,
> >>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> Hi
> >>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> I
> >>>>>>>>>>> believe
> >>>>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>>>> feature deals
> >>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> number
> >> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> My
> >>>>>>>>>>> feature
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> focused on
> >>>>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> I
> >>>>>>>> hope
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> On
> >>>>>>>> Sun,
> >>>>>>>>>>> Jun
> >>>>>>>>>>>>>>> 2,
> >>>>>>>>>>>>>>>>> 2013 at 6:35
> >>>>>>>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> My
> >>>>>>>>>>>>>>> opinion is
> >>>>>>>>>>>>>>>>> that no
> >>>>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> issues
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> been resolved
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> total
> >>>>>>>>>>> Ops
> >>>>>>>>>>>>>>> vs
> >>>>>>>>>>>>>>>>> discrete
> >>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> re-
> >>>>>>>>>>> reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido,
> >>>>>>>>>>> I
> >>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>> think this
> >>>>>>>>>>>>>>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Also,
> >>>>>>>>>>> how
> >>>>>>>>>>>>>>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> also
> >>>>>>>>>>>>>>> involves
> >>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>> where provisioned
> >>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> -
> >>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> On
> >>>>>>>>>>> Jun 1,
> >>>>>>>>>>>>>>> 2013,
> >>>>>>>>>>>>>>>>> at 6:47 AM,
> >>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Sure. I
> >>>>>>>>>>> will
> >>>>>>>>>>>>>>>>> change it next
> >>>>>>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> -
> >>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> 2013/6/1
> >>>>>>>>>>>>>>>>> Wido den Hollander
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> On
> >>>>>>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> Exactly. I
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> pushed the
> >>>>>>>>>>>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> If
> >>>>>>>>>>> anyone
> >>>>>>>>>>>>>>>>> object thems for
> >>>>>>>>>>>>>>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> For
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> sake
> >>>>>>>>>>>>>>>>> of clarity I
> >>>>>>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> the
> >>>>>>>>>>> total
> >>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>> to R/W IOps
> >>>>>>>>>>>>>>>>>>>>> asap
> >>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> only
> >>>>>>>>>>> total
> >>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> You
> >>>>>>>>>>> laid
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> groundwork for
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> however
> >>>>>>>>>>>>>>>>> prevent that we
> >>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> -
> >>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> 2013/5/31
> >>>>>>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> On
> >>>>>>>>>>>>>>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> +1 --
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> enhancement must
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> don't
> >>>>>>>>>>> see
> >>>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>>> it could be
> >>>>>>>>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how
> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> split
> >>>>>>>>>>> total
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> into read
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> total
> >>>>>>>>>>>>>>>>> unless/until we
> >>>>>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> What
> >>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>> meant was merging
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> 4.2
> >>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>> and add Read /
> >>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> will be
> >>>>>>>>>>>>>>>>> released with Read
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> This is
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> make the May 31st
> >>>>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> (see
> >>>>>>>>>>> other
> >>>>>>>>>>>>>>>>> threads) then it
> >>>>>>>>>>>>>>>>>>>>> won't
> >>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> I
> >>>>>>>> also
> >>>>>>>>>>>>>>>>> completely agree that
> >>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> disk
> >>>>>>>>>>> I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> -
> >>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> On
> >>>>>>>>>>> May 31,
> >>>>>>>>>>>>>>>>> 2013, at 9:51 AM,
> >>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> On
> >>>>>>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Hi
> >>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> Thanks.
> >>>>>>>>>>>>>>> Good
> >>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> I
> >>>>>>>>>>> thought
> >>>>>>>>>>>>>>>>> about at the
> >>>>>>>>>>>>>>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> difference
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> read and write
> >>>>>>>>>>>>>>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> care
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> difference of sent
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> That
> >>>>>>>>>>>>>>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Disk
> >>>>>>>>>>> I/O is
> >>>>>>>>>>>>>>>>> much more
> >>>>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> Implementing
> >>>>>>>>>>>>>>>>> it will be some
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> few
> >>>>>>>>>>> days.
> >>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>> the deadline
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> that ,
> >>>>>>>>>>> if
> >>>>>>>>>>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> It
> >>>>>>>>>>> think it's
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> feature we
> >>>>>>>>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> window
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> have to make sure
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> it
> >>>>>>>> in
> >>>>>>>>>>> 4.3,
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> will confuse
> >>>>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> -
> >>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> 2013/5/31
> >>>>>>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> On
> >>>>>>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> I
> >>>>>>>>>>> would
> >>>>>>>>>>>>>>> like to
> >>>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> If
> >>>>>>>>>>> nobody
> >>>>>>>>>>>>>>>>> object, I will
> >>>>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> The
> >>>>>>>>>>>>>>> purpose
> >>>>>>>>>>>>>>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Virtual
> >>>>>>>>>>>>>>>>> machines are running
> >>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> share
> >>>>>>>>>>>>>>> strage).
> >>>>>>>>>>>>>>>>> Because of
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such
> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> iops),
> >>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> one
> >>>>>>>>>>> VM
> >>>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>>>> large disk
> >>>>>>>>>>>>>>>>>>>>> operation,
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> other
> >>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>> running on the
> >>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> It
> >>>>>>>> is
> >>>>>>>>>>>>>>> neccesary
> >>>>>>>>>>>>>>>>> to set the
> >>>>>>>>>>>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> Looking at
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> code I see
> >>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Qemu
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> libvirt support
> >>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> IOps
> >>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>> could benefit a
> >>>>>>>>>>>>>>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> It's
> >>>>>>>>>>> also
> >>>>>>>>>>>>>>>>> strange, in the
> >>>>>>>>>>>>>>>>>>>>> polling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> IOps,
> >>>>>>>>>>> but
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> the throttling
> >>>>>>>>>>>>>>>>>>>>> side
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Write
> >>>>>>>>>>> IOps
> >>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>> usually much
> >>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> like a
> >>>>>>>>>>> valid
> >>>>>>>>>>>>>>>>> use-case where
> >>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> IOps
> >>>>>>>>>>> vs
> >>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Since
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>> supports KVM
> >>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> value
> >>>>>>>>>>> to at
> >>>>>>>>>>>>>>>>> least have the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> this
> >>>>>>>>>>> later
> >>>>>>>>>>>>>>>>> would be a lot of
> >>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> If
> >>>>>>>> a
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> doesn't
> >>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> write
> >>>>>>>>>>> you
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> always sum
> >>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Can
> >>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> explain why you
> >>>>>>>>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> The
> >>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> (1) set
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> maximum rate of
> >>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> (2)
> >>>>>>>>>>> change
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> maximum rate
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> (3)
> >>>>>>>>>>> limit
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> disk rate
> >>>>>>>>>>>>>>>>>> (total
> >>>>>>>>>>>>>>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> JIRA
> >>>>>>>>>>> ticket:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-
> >>>> 1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 1192
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> FS (I
> >>>>>>>>>>> will
> >>>>>>>>>>>>>>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> Merge
> >>>>>>>>>>>>>>> check
> >>>>>>>>>>>>>>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> *
> >>>>>>>> Did
> >>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> check the branch's
> >>>>>>>>>>>>>>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> *
> >>>>>>>> Are
> >>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>> new dependencies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> *
> >>>>>>>>>>> What
> >>>>>>>>>>>>>>>>> automated testing
> >>>>>>>>>>>>>>>>>> (unit
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Unit
> >>>>>>>>>>> tests
> >>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> *
> >>>>>>>>>>> What
> >>>>>>>>>>>>>>>>> testing has been done
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> (1) set
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> bytes rate and
> >>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> (2)
> >>>>>>>>>>> VM
> >>>>>>>>>>>>>>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>> deploy,
> >>>>>>>>>>>>>>> stop,
> >>>>>>>>>>>>>>>>> start, reboot,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> (3)
> >>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>> operations,
> >>>>>>>>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Attach,
> >>>>>>>>>>>>>>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> To
> >>>>>>>>>>> review
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> code, you can
> >>>>>>>>>>>>>>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Best
> >>>>>>>>>>>>>>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> >>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-
> >> 2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> -
> >>>>>>>>>>> VM
> >>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> *Mike
> >>>>>>>>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> *Senior
> >>>>>>>>>>>>>>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> e:
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>> o:
> >>>>>>>>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>> Advancing
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> way the world
> >>>>>>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Mike
> >>>>>>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Senior
> >>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> o:
> >>>>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> Advancing
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Mike
> >>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Senior
> >>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> o:
> >>>>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> Advancing the
> >>>>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Mike
> >>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Senior
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Advancing
> >>>>>>>> the
> >>>>>>>>>>> way
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Mike
> >>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Senior
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Advancing
> >>>>>>>> the
> >>>>>>>>>>> way
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> Developer, SolidFire
> >>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Advancing
> >>>> the
> >>>>>>>> way
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>>> CloudStack
> >>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> >>>> the
> >>>>>>>> way
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> world
> >>>>>>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>>> CloudStack
> >>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> >> the
> >>>>>>>> way the
> >>>>>>>>>>>>>>> world
> >>>>>>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >>>> way
> >>>>>>>> the
> >>>>>>>>>>> world
> >>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>> the
> >>>>>>>>>>> world
> >>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> >>>> the
> >>>>>>>> world
> >>>>>>>>>>> uses
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >> the
> >>>>>>>> world
> >>>>>>>>>>> uses
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> >>>> world
> >>>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> >>>> world
> >>>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>> world
> >>>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>> uses
> >>>>>>>> the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >>>> uses
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *(tm)*****
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> ****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*****
> >>>>>>>>>>
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>>>>>
> >>>>>>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>>>>>
> >>>>>>>>>> o: 303.746.7302****
> >>>>>>>>>>
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *(tm)*****
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> ****
> >>>>>>>>>>
> >>>>>>>>>> ** **
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*****
> >>>>>>>>>>
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>>>>>
> >>>>>>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>>>>>
> >>>>>>>>>> o: 303.746.7302****
> >>>>>>>>>>
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *(tm)*****
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *(tm)*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *(tm)*
> >>>
> >
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Edison,
Please see my responses in-line below.
Thanks,
-John
On Jun 10, 2013, at 6:52 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: John Burwell [mailto:jburwell@basho.com]
>> Sent: Monday, June 10, 2013 2:44 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>> Edison,
>>
>> Please see my response in-line below.
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 4:45 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>> Sent: Monday, June 10, 2013 1:25 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>
>>>> Mike,
>>>>
>>>> Yes, a vendor may hand a non-managed plugin. For example, on the
>>>> secondary storage side, S3 and Swift don't support the concept of being
>>>> managed -- an operator is required to create a bucket and point
>> CloudStack
>>>> at it. Another example would be a generic iSCSI device (e.g. OpenFiler,
>>>> FreeNAS) which CloudStack can not directly manage, but can consume a
>> LUN
>>>> that an operator has created for it.
>>>
>>> Manage or non-managed storage is handled by DataStoreLifeCyclce.
>>> It depends on storage vendor to implement DataStoreLifeCyclce, which
>> basically says how to initialize a storage, how to add a storage into a cluster
>> etc.
>>
>> So, implementation of the DataStoreLifeCycle interface indicates that driver
>> is capable of managing the device. It does not answer the question of
>
> No, the interface only means there is a way to add a storage into cloudstack, no matter
> It's managed or un-managed.
>
>> whether or not the administrator/operator wants CloudStack to manage the
>> device. Hence, the need for a flag on the StoragePool to indicate we should
> Cloudstack don't need to care about the storage is managed or un-managed, what it cares about
> Is how to create/delete something on it, it will call storage plugins to do all the storage related operations.
>
>
>> be managing it. I am cool with the flag defaulting to true when the driver
>> supports it, but we must provide the administrator/operator to ability to opt-
>
> If the storage provider wants to opt-out or wants to do things differently based on input from administrator/operator,
> The storage provider can add parameters in createstoragepoolcmd->details, details is a map, which can contain whatever information needed by the storage provider.
>
> Let's say a storage provider which can support both managed/un-managed storage pool, when admin wants a storage pool managed by the storage pool, admin can set "managed" = "true" in the details, then
> The storage provider will know, oh, admin wants a managed storage pool, so the DataStoreLifeCyclce->init will create a storage pool dynamically on the storage box. While if the "managed" = "false", then DataStoreLifeCyclce->init will not create a storage pool, instead, it will check the existence of the storage pool, if it's there, then return OK.
>
> I don't think cloudstack mgt server needs to know, the storage pool is managed/unmanaged, it's up to the implementation of storage provider.
I think we are speaking past each other. I am concerned with the policy the operator/administrator intends the implement not the overall capabilities of the underlying driver. A driver may be capable of features that an operator/administrator has elected not to use. The only place to capture that policy is in the management server configuration of the DataStore.
>
>> out. Does this interface provide for a set of administrative credentials
>> separate from the credentials used to access storage?
>
> Yes, createstoragepoolcmd->details, is the place to add whatever information when creating a storage pool.
What is the format of the details field? The API does just list it as a non-required field in 4.1.
>
>>
>> Another question comes to mind -- why do we have a Dynamic
>> StoragePoolType when this interface exists?
>
> It shouldn't.
> Storage provider can return a VolumeTO which can contain whatever information about the volume.
> At the resource side, it may check VolumeTO, if the volume is not a VDI, VolumeTO can contain a field called "isManaged", which can be set to true by Solidfire's driver, while by default it's false.
> If isManaged is true, then the resource code can create a SR and create a VDI on the fly.
My point is that nothing in the Storage layer should know out an SR or VDI. These are hypervisor concepts -- specifically Xen. The Storage layer should only be exposing the discrete operations necessary for the hypervisor to determine if an SR and/or VDI are present, as well as, the operations to create the structures for them. Fundamentally, it's this circular dependency between the Hypervisor and Storage layer that I believe we need to eliminate. It is one of the sources of the feature inconsistency across hypervisors. For SolidFire, the "managed" flag indicates that the driver should programmatically allocate a LUN with a particular IQN -- nothing else. The Xen hypervisor plugin should have the knowledge that iSCSI volumes need to have an SR created in a particular form on an iSCSI device.
>
>>
>>>
>>> Take S3 as an example, if the S3 storage provider provides a
>> DataStoreLifeCyclce which can create a bucket, then when admin adding a S3
>> storage through API, mgt server will first call DataStoreLifeCyclce, which will
>> create a bucket, then stored in the image_store_details table, then later on,
>> all the referenced to this image store, will contain this bucket information.
>>
>> From an S3 perspective, creating a bucket from CloudStack would not a be
>> security best practice. The general guidance would be for the bucket to
>> created and associated to a dedicated CloudStack access/secret key pair to
>> properly isolate data. Given this best practice and that only one bucket per
>> S3 instance would be necessary for CloudStack operation, I think most
>> administrators/operators would want to opt-out CloudStack managing S3
>> buckets.
>>
>>>
>>> Take an storage box from some vendor as an example, admin wants to
>> create an ISCSI server on a storage box, which can be used by a
>> cluster(multiple volumes on a giant LUN ), as long as the storage provider can
>> provide a DataStoreLifeCyclce which can program the storage box, and create
>> an ISCSI server, then stored the information in storage_pool table, then all
>> the referenced to the storage pool will use the dynamically created ISCSI LUN.
>>>
>>> Take another storage box from some vendor as an example, admin wants
>> to add the whole storage box into cloudstack, which can be used as per LUN
>> per ISCSI, so in this case, DataStoreLifeCyclce won't create an ISCSI server
>> during the initialize method call. Instead, for every createAsync, it will create
>> an ISCSI LUN, stored the LUN information in volume table and volume_detail
>> table. Whenever the LUN is used by hypervisor, the hypervisor needs to take
>> extra care about the volume.
>>>
>>>
>>>
>>>
>>>>
>>>> From an operational perspective, an administrator may elect to say, "Hey,
>> I
>>>> know you can manage that device for me, but it is has a wider use than
>>>> CloudStack can manage. Let me handle it, and just provide the
>> information
>>>> CloudStack needs." In this scenario, the operator is accepting a greater
>>>> management burden in exchange for control. I don't think many will elect
>> to
>>>> manage these devices themselves, but this ability will be extremely
>>>> important for the edge cases that require it. Speaking as someone who
>> has
>>>> run cloud operations, I can testify to this need -- nothing aggravates you
>>>> more than a system insisting on managing a device or service without the
>>>> opportunity to opt-out
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 10, 2013, at 3:19 PM, Mike Tutkowski
>> <mi...@solidfire.com>
>>>> wrote:
>>>>
>>>>> So, would a storage vendor write a non-CloudStack-managed plug-in and
>>>> maybe
>>>>> just implement the snapshot feature or something?
>>>>>
>>>>> In this case, they'd have to set the hypervisor up ahead of time with a
>>>>> large portion of their storage, but they'd still get the vendor-snapshot
>>>>> ability.
>>>>>
>>>>> Just trying to undertand this more.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Edison,
>>>>>>
>>>>>> TL;DR Always give operators/system administrators the ability to
>> manage
>>>>>> things themselves, if they like.
>>>>>>
>>>>>> As more manageable drivers emerge, I think it will be import to allow
>>>>>> operators to opt-out of CloudStack device management. Therefore, a
>>>> driver
>>>>>> declares whether or not it can support device management (many, in
>> fact
>>>>>> most, won't). For those drivers that do support management, allow
>>>>>> operators to opt-in/out of CloudStack management on a per device
>> basis.
>>>> To
>>>>>> support this notion, we need a flag on the storage_pool table and
>>>>>> associated classes to indicate whether or not the operator wants
>>>> CloudStack
>>>>>> to manage the device.
>>>>>>
>>>>>> Does this explanation make more sense?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>>>
>>>>>>> I don't understand why we need to add a "managed" in the UI and in
>>>>>> "storage_pool" table?
>>>>>>> AFAIK, what you guys trying to do is to enable
>>>>>> attachVolume/DettachVolume if the volume is created on Solidfire,
>> right?
>>>>>>> If the volume is created on Solidfire already, by createAsync, then
>> why
>>>>>> we can't just send out all the information about the volume(it's
>> volumeTO
>>>>>> which will be returned by each storage driver) to hypervisor, then at
>> the
>>>>>> hypervisor side, check the volumeTO, if the volumeTO is managed,
>> then
>>>> doing
>>>>>> whatever necessary operations(create SR, create VDI etc), then attach
>>>> the
>>>>>> VDI to volume.
>>>>>>>
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>> Sent: Monday, June 10, 2013 11:10 AM
>>>>>>>> To: John Burwell
>>>>>>>> Cc: Edison Su; dev@cloudstack.apache.org
>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>
>>>>>>>> I believe we're on the same page, John.
>>>>>>>>
>>>>>>>> I would add a new parameter to the createStoragePool API
>> command
>>>> (let's
>>>>>>>> call it "managed" and it can be true or false). If you pass in "true",
>>>>>> but
>>>>>>>> the driver does not support being managed by CloudStack, an
>>>> exception is
>>>>>>>> thrown by the driver. If you pass in "false", but the driver does not
>>>>>>>> support being managed outside of CloudStack, an exception is
>> thrown
>>>> by
>>>>>> the
>>>>>>>> driver. It is possible one could write a driver that can be managed
>> both
>>>>>>>> inside or outside of CloudStack (the SolidFire driver requires being
>>>>>>>> managed only by CloudStack).
>>>>>>>>
>>>>>>>> This managed value would end up in the storage_pool table in a new
>>>>>> column
>>>>>>>> ("managed").
>>>>>>>>
>>>>>>>> An example usage: After the storage framework calls on the driver's
>>>>>>>> createAsync method, it will check to see if the driver is managed. If
>>>>>> the
>>>>>>>> driver is managed, the storage framework will invoke the proper
>> (new)
>>>>>>>> command of the hypervisor in use to set up, say, the SR for the SAN
>>>>>> volume.
>>>>>>>>
>>>>>>>> There may be other cases when the isManaged() method is invoked
>> on
>>>> the
>>>>>>>> driver and - if true is returned - the storage framework sends a
>>>>>> particular
>>>>>>>> command to the hypervisor in use.
>>>>>>>>
>>>>>>>> ********************
>>>>>>>>
>>>>>>>> As far as reworking the default storage plug-in (which sends
>> commands
>>>> to
>>>>>>>> hypervisors) for 4.2, I would check with Edison on his schedule.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell
>> <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> I want to make sure we are on the same page regarding isManaged.
>> I
>>>> see
>>>>>>>>> the addition of the following operations:
>>>>>>>>>
>>>>>>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates
>>>> whether
>>>>>>>> or
>>>>>>>>> not the driver support/implements the management functions
>>>>>>>>> * DataStore#isManaged() : boolean -- indicates whether or not a
>>>> device
>>>>>> is
>>>>>>>>> managed by CloudStack
>>>>>>>>>
>>>>>>>>> Only DataStores associated with DataStoreDrivers that support
>>>>>>>> management
>>>>>>>>> would be able to have the management flag enabled. This
>> behavior
>>>>>> allows
>>>>>>>>> operators to declare their intention to allow CloudStack to manage
>> a
>>>>>>>>> device. This behavior would need to be exposed as flag on the
>> HTTP
>>>>>> API.
>>>>>>>>> Additionally, this flag would only be settable when a device is
>>>>>>>>> defined/created in the system (i.e. once a device is managed by
>>>>>> CloudStack,
>>>>>>>>> you can't unmanaged it without dissociating it from CloudStack). In
>>>>>> the
>>>>>>>>> future, we may consider relaxing that rule, but, in the short term, it
>>>>>>>>> greatly decreases the alternative flows.
>>>>>>>>>
>>>>>>>>> In terms of removing the dependencies from Storage to Hypervisor,
>> I
>>>>>>>> would
>>>>>>>>> really like to see this accomplished in 4.2. It will be much more
>>>>>>>>> difficult to address this issue post 4.2 because we will have more
>>>>>> plugins
>>>>>>>>> to address. I also see this issue as a critical architectural issues
>>>>>> that
>>>>>>>>> for the management server. I recommend we identify the
>>>> dependencies
>>>>>>>> that
>>>>>>>>> need to be inverted, and see if we can divide the work across a few
>>>>>> more
>>>>>>>>> resources.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
>>>>>>>> <mi...@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Sounds good, Edison :)
>>>>>>>>>
>>>>>>>>> I can implement John's isManaged() logic and resubmit my code for
>>>>>> review.
>>>>>>>>>
>>>>>>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
>>>>>> requires
>>>>>>>>> some refactoring of the storage framework, then I could do that.
>>>>>>>>>
>>>>>>>>> Also, the default storage plug-in does have hypervisor logic in it in
>>>>>> 4.2.
>>>>>>>>> I recommend we leave this as is for 4.2 and it can be refactored in
>> 4.3
>>>>>>>>> with the above work I mentioned.
>>>>>>>>>
>>>>>>>>> Are you OK with that, guys?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
>>>>>>>>>> *To:* dev@cloudstack.apache.org
>>>>>>>>>> *Cc:* Edison Su; John Burwell
>>>>>>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> Another thing we might want to consider for 4.2 is that the way
>> the
>>>>>> code
>>>>>>>>>> is currently implemented in master, I don't think it's even possible
>>>>>> to
>>>>>>>>>> write a plug-in that connects storage directly to a VM because the
>>>>>>>>>> CloudStack storage framework will call the "attach" method of the
>>>>>>>>>> hypervisor in use and this method will fail because it assumes
>> there
>>>>>> is
>>>>>>>>>> (talking Xen here) an SR existent and in this situation there isn't
>>>>>> one.*
>>>>>>>>>> ***
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> My thinking is that we should ignore the direct-attach-of-volume-
>> to-
>>>> VM
>>>>>>>>>> use case for 4.2. I think it will require too much of a rewrite at
>>>>>> this
>>>>>>>>>> point in development (fewer than three weeks left until code
>>>>>>>> freeze).****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> If the plug-in implements an isManaged() method (as John
>>>> suggested),
>>>>>>>> the
>>>>>>>>>> storage framework can invoke the necessary commands on the
>>>>>>>> hypervisor in
>>>>>>>>>> use.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> For example, the createAsync method is called. The storage plug-
>> in
>>>>>>>>>> creates a SAN volume and updates the CS DB. The storage
>>>> framework
>>>>>>>> then asks
>>>>>>>>>> the driver if it's managed. If it is, the storage framework executes
>> a
>>>>>>>>>> command against the hypervisor in use to have it create, say, an
>> SR
>>>>>> (with
>>>>>>>> a
>>>>>>>>>> single VDI that takes up the entire SR). Now, when the storage
>>>>>>>> framework
>>>>>>>>>> executes the "attach" command against the hypervisor, this
>>>> command
>>>>>> will
>>>>>>>>>> work, as well (because that logic assumes the prior existence of
>> an
>>>>>> SR and
>>>>>>>>>> it will, in fact, be present).****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:****
>>>>>>>>>>
>>>>>>>>>> Hi Edison and John,****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> I wanted to point out something I thought was relevant to this
>>>>>>>>>> conversation.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> My plug-in supports both XenServer and ESX(i).****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> These two hypervisors handle their respective hypervisor data
>>>>>> structures
>>>>>>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> In XenServer, the SR that resides on the SAN LUN is simply
>>>> "forgotten"
>>>>>>>>>> when you issue a remove-SR command to XenServer.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> When you later want to access the contents of that data, it is still
>>>>>>>>>> safely stored on the LUN and can be accessed from XenServer
>> (from
>>>> the
>>>>>>>> same
>>>>>>>>>> or different cluster (what XenServer calls a resource pool)) by
>>>>>> creating a
>>>>>>>>>> new SR that is based on the same IQN and LUN.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> XenServer SRs exist at the cluster level.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
>>>>>>>>>> destroyed when you issue a remove-datastore command to
>> ESX(i).
>>>> In
>>>>>>>> other
>>>>>>>>>> words, the contents of the SAN LUN are destroyed.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> ESX(i) datastores exist at the datacenter level.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> The XenServer behavior (not destroying the contents of the LUN
>>>> when
>>>>>>>> the
>>>>>>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
>>>>>> contents
>>>>>>>>>> of the LUN when the datastore is removed) is not desirable.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> I can get around the ESX(i) behavior by removing the referenced
>> IQN
>>>>>>>> from
>>>>>>>>>> each host in the cluster when the CS storage framework notifies
>> my
>>>>>> plug-
>>>>>>>> in
>>>>>>>>>> of a detach-volume event. When the CS volume is later attached
>>>> again
>>>>>>>>>> (either to the same or a different ESX(i) cluster), I can have each
>>>>>> host in
>>>>>>>>>> the relevant ESX(i) cluster reference the IQN and the datastore
>> will
>>>>>> be
>>>>>>>>>> back again.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> This sequence of events is critical for my plug-in to support both
>>>>>>>>>> environments:****
>>>>>>>>>>
>>>>>>>>>> *XenServer flow (events for Driver class):*****
>>>>>>>>>>
>>>>>>>>>> createAsync (called right before first attach) = creates SAN
>>>>>> volume****
>>>>>>>>>>
>>>>>>>>>> preAttachVolume = creates SR and creates VDI or just introduces
>>>> SR****
>>>>>>>>>>
>>>>>>>>>> (CS volume gets attached and later detached)****
>>>>>>>>>>
>>>>>>>>>> postDetachVolume = removes SR (this does not delete data on
>> SAN
>>>>>>>> volume)**
>>>>>>>>>> **
>>>>>>>>>>
>>>>>>>>>> deleteAsync (can be called when volume is not attached) =
>> deletes
>>>> the
>>>>>>>> SAN
>>>>>>>>>> volume****
>>>>>>>>>>
>>>>>>>>>> *VMware flow (events for Driver class):*****
>>>>>>>>>>
>>>>>>>>>> createAsync (called right before first attach) = creates SAN
>> volume,
>>>>>>>>>> creates datastore, and creates VMDK file****
>>>>>>>>>>
>>>>>>>>>> preAttachVolume = adds iSCSI target to each host in cluster****
>>>>>>>>>>
>>>>>>>>>> (CS volume gets attached and later detached)****
>>>>>>>>>>
>>>>>>>>>> postDetachVolume = removes iSCSI target from each host in
>>>> cluster****
>>>>>>>>>>
>>>>>>>>>> deleteAsync (can be called when volume is not attached) =
>> deletes
>>>>>>>>>> datastore and deletes the SAN volume****
>>>>>>>>>>
>>>>>>>>>> ************************
>>>>>>>>>>
>>>>>>>>>> So, the way I have it written up here, the driver is talking to the
>>>>>>>>>> hypervisor. As John has pointed out, this may not be ideal.****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> Whatever we decide on, it needs to be able to support this kind of
>>>>>> flow.
>>>>>>>>>> :)****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> We could go John's route, where the driver says if it's managed or
>>>>>> not.**
>>>>>>>>>> **
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> If it is, then the storage framework could send the necessary
>>>>>> messages to
>>>>>>>>>> the hypervisor in question at the right times (just migrate the
>>>>>> hypervisor
>>>>>>>>>> logic above from the driver to the right parts of the storage
>>>>>> framework).
>>>>>>>>>> ****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> This is fine, except that it does not resolve Edison's use case of
>>>>>>>>>> desiring to have the storage LUN potentially connected directly to
>> a
>>>>>> VM.*
>>>>>>>>>> ***
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> I'm not sure we should design around that use case, though. Is it
>>>>>> common
>>>>>>>>>> enough? Has anyone asked for it? Is it even desirable? How are
>>>> usage
>>>>>>>>>> statistics gathered for such volumes? It's nice to offer the
>>>>>> flexibility,
>>>>>>>>>> but is it a good idea in this case?****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> Thanks!****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
>>>>>> wrote:**
>>>>>>>>>> **
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>> Sent: Friday, June 07, 2013 4:08 PM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>
>>>>>>>>>>> Hi John,
>>>>>>>>>>>
>>>>>>>>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
>>>>>>>>>> volume
>>>>>>>>>>> to use it. Same story for VMware: ESX(i) must layer a datastore
>> on
>>>>>> top
>>>>>>>>>> of
>>>>>>>>>>> the iSCSI volume to use it. I look at it like they are layering a
>>>>>>>>>> clustered
>>>>>>>>>>> file system on the SAN volume so the hypervisors can share
>> access
>>>> to
>>>>>>>> the
>>>>>>>>>>> contents of the volume.
>>>>>>>>>>>
>>>>>>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called,
>> the
>>>>>>>>>> SR or
>>>>>>>>>>> datastore has already been created (usually manually by an
>> admin).
>>>>>>>>>>>
>>>>>>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
>>>>>>>>>>> environment where each CloudStack volume that a user creates
>> is
>>>>>>>> mapped
>>>>>>>>>>> to a
>>>>>>>>>>> single SAN volume (via the SR).
>>>>>>>>>>>
>>>>>>>>>>> The question comes down to who should allocate the SR
>>>> dynamically.
>>>>>>>>>>>
>>>>>>>>>>> We could have the storage framework ask the storage plug-in if
>> it is
>>>>>>>>>>> managed. If it is, then the storage framework could send a
>>>> message to
>>>>>>>>>> the
>>>>>>>>>>> hypervisor in question to create (let's talk Xen here) the SR
>> ahead
>>>>>> of
>>>>>>>>>>> time. Then, when the storage framework next sends the attach-
>>>> volume
>>>>>>>>>>> command
>>>>>>>>>>> to the hypervisor, it should work without changes to that
>>>>>> attach-volume
>>>>>>>>>>> logic (because - from the point of view of the attach logic - the SR
>>>>>> is
>>>>>>>>>>> already existent, as expected).
>>>>>>>>>>>
>>>>>>>>>>> Now, as Edison has pointed out, this limits the power of the
>> storage
>>>>>>>>>>> plug-in. A storage plug-in in this model cannot directly attach
>>>>>> storage
>>>>>>>>>> to
>>>>>>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We
>>>> need
>>>>>>>> to
>>>>>>>>>>> make a call on that.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, I think we need to address this usage case, and won't limit
>> the
>>>>>>>>>> power of the storage plug-in.
>>>>>>>>>> John, what's your idea to fit this usage case into your abstraction?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> If we want to give more power to the storage vendor, we would
>>>> have to
>>>>>>>>>>> have
>>>>>>>>>>> the storage framework call into the storage plug-in to send the
>>>>>>>>>> appropriate
>>>>>>>>>>> attach commands to the hypervisor.
>>>>>>>>>>>
>>>>>>>>>>> Let's discuss and come to a consensus (or at least agree on a
>> path)
>>>>>> as
>>>>>>>>>> soon
>>>>>>>>>>> as we can. :)
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell
>> <jb...@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> My understanding is the Xen expects an SR structure on all
>> ISCSI
>>>>>>>>>> devices
>>>>>>>>>>>> -- at least that is how I read the code in your patch. Is my
>>>>>>>>>> understanding
>>>>>>>>>>>> correct? If so, the Xen plugin should be able to query the
>> storage
>>>>>>>>>> device
>>>>>>>>>>>> to determine the presence of the SR structure and create it if it
>>>>>>>>>> does not
>>>>>>>>>>>> exist. Am I missing something in the implementation that
>> makes
>>>> that
>>>>>>>>>> type
>>>>>>>>>>>> of implementation impossible?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Yeah, if a storage vendor wanted to attach a volume directly
>> to a
>>>>>>>>>> VM in
>>>>>>>>>>>> 4.2
>>>>>>>>>>>>> today, it would probably fail because the attach-volume logic
>>>>>>>>>> assumes
>>>>>>>>>>> the
>>>>>>>>>>>>> existence of the necessary hypervisor data structure (ex. SR
>> on
>>>>>>>>>> Xen).
>>>>>>>>>>>>>
>>>>>>>>>>>>> If we wanted to enable such an attach, we could do it the way
>>>>>>>> Edison
>>>>>>>>>>>>> suggests.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su
>> <Ed...@citrix.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Mike Tutkowski
>> [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As we only have three weeks until feature freeze, we
>> should
>>>>>>>> come
>>>>>>>>>> to
>>>>>>>>>>> a
>>>>>>>>>>>>>>> consensus on this design point as soon as possible.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Right now, if the storage framework asks my driver it is is
>>>>>>>>>> managed, it
>>>>>>>>>>>>>>> will say 'yes.' This means the framework will tell the driver
>> to
>>>>>>>>>>>> perform
>>>>>>>>>>>>>>> its management activities. This then means the driver will
>> call
>>>>>>>>>> into
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to
>>>> perform
>>>>>>>> the
>>>>>>>>>>>>>> activity
>>>>>>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The driver doesn't know which hypervisor it's talking to, it
>> just
>>>>>>>>>>>> sends a
>>>>>>>>>>>>>>> message to the host to perform the necessary pre-attach
>>>> work.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Could we just expose a method like
>>>>>>>> "attachVolume/dettachVolume" on
>>>>>>>>>>> the
>>>>>>>>>>>>>> PrimaryDataStoreDriver driver?
>>>>>>>>>>>>>> For most of cases, the implementation of each driver would
>>>> just
>>>>>>>>>> send
>>>>>>>>>>>>>> attachvolumecommand/dettachvolumecmd to
>> hypervisor(we
>>>> can
>>>>>>>> put
>>>>>>>>>>> the
>>>>>>>>>>>>>> implementation in a base class, so that can be shared by all
>> of
>>>>>>>>>> this
>>>>>>>>>>>>>> drivers), and in each hypervisor resource code, which may
>> just
>>>>>> call
>>>>>>>>>>>>>> hypervisor's api to attach the volume to VM, while for
>> certain
>>>>>>>>>> storage,
>>>>>>>>>>>>>> like SolidFire, may need to create a SR first, and create
>> volume
>>>>>>>>>> on it,
>>>>>>>>>>>>>> then call hypervisor's API to attach volume to VM.
>>>>>>>>>>>>>> While for some other storage vendor, may want to bypass
>>>>>>>> hypervisor
>>>>>>>>>>>> during
>>>>>>>>>>>>>> attaching volume, so inside driver's attachvolume
>>>> implementation,
>>>>>>>>>> the
>>>>>>>>>>>>>> driver can do some magic, such as, directly talk to an agent
>>>>>>>>>> inside VM
>>>>>>>>>>>>>> instance, then create a disk inside VM.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> How do you guys think?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su
>>>> <Ed...@citrix.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>> From: Mike Tutkowski
>>>> [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> How's about this:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The driver can implement an isManaged() method. The
>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
>>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>> (which is responsible for calling into the hypervisor to
>>>> attach
>>>>>>>>>> the
>>>>>>>>>>>>>> disk)
>>>>>>>>>>>>>>>>> can call into the hypervisor to create the necessary
>>>> hypervisor
>>>>>>>>>> data
>>>>>>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The problem here is that storage vendor may work
>>>> differently
>>>>>>>> with
>>>>>>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN,
>> while
>>>>>>>>>> maybe
>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the
>>>> LUN
>>>>>>>>>>> directly
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> VM instance, see the discuss(
>>>>>>>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>
>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>>>>>>>>>>>>>>> %3E
>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>> So I would let storage provider to implement attach disk
>> to
>>>> VM,
>>>>>>>>>>>>>> instead of
>>>>>>>>>>>>>>>> implemented by cloudstack itself.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If that's what you're going for, that works for me. By the
>>>> way,
>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>> default storage plug-in (which handles the default
>> storage
>>>>>>>>>> behavior
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code
>> that
>>>>>>>>>> talks to
>>>>>>>>>>>>>>>>> hypervisors. You might want to contact him and inform
>> him
>>>> of
>>>>>>>>>> your
>>>>>>>>>>>>>>>> concerns
>>>>>>>>>>>>>>>>> or that logic (as is) will make it to production.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please let me know if what I wrote in above (for my
>>>> solution)
>>>>>>>>>> is OK
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please see my responses in-line below.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I
>> build
>>>>>>>>>> up
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>> file, but this is the gist of what I've done:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * During a volume-attach operation, after
>>>>>>>> VolumeManagerImpl
>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a
>> volume, I
>>>>>>>> have
>>>>>>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask
>> the
>>>>>>>> driver if
>>>>>>>>>> it
>>>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver
>>>> perform
>>>>>>>>>>>>>> whatever
>>>>>>>>>>>>>>>>>>> activity is required. In my case, this includes sending a
>>>>>>>>>>>>>> message to
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> host where the VM is running to have, say XenServer,
>>>> add a
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the
>> IQN
>>>> of
>>>>>>>> the
>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the
>> space
>>>> it
>>>>>>>>>> can
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> storage repository). After this, the normal attach-
>> volume
>>>>>>>>>>>>>> message is
>>>>>>>>>>>>>>>> sent
>>>>>>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> There should be **no** code from a storage driver to
>> a
>>>>>>>>>> hypervisor.
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> apologize for the repetition, but we simply can not
>> have
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> specific code in the storage layer. The circular
>>>> dependencies
>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> two layers are not sustainable in the long term. Either
>> the
>>>>>>>>>>>>>>>> VirtualManager
>>>>>>>>>>>>>>>>>> or Xen hypervisor plugin needs to be
>> refactored/modified
>>>> to
>>>>>>>>>>>>>>> coordinate
>>>>>>>>>>>>>>>>>> volume creation and then populating the SR. Ideally,
>> we
>>>> can
>>>>>>>>>>>>>>>> generalize the
>>>>>>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
>>>>>>>> hypervisor
>>>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>>>> would only implement callbacks to perform the attach
>>>> action
>>>>>>>> and
>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver
>> should
>>>>>>>>>> only be
>>>>>>>>>>>>>>>>>> allocating space and providing information about
>> contents
>>>>>>>> (e.g.
>>>>>>>>>>>>>> space
>>>>>>>>>>>>>>>>>> available, space consumed, streams to a URI, file
>> handle
>>>> for a
>>>>>>>>>> URI,
>>>>>>>>>>>>>>>> etc)
>>>>>>>>>>>>>>>>>> and capabilities.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * The reverse is performed for a detach-volume
>>>> command.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in
>> my
>>>>>>>>>> driver.
>>>>>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>>>> default driver simply does a "return false;". We could
>>>> add a
>>>>>>>>>> new
>>>>>>>>>>>>>>>>>> parameter
>>>>>>>>>>>>>>>>>>> to the createStoragePool API command, if we want,
>> to
>>>>>>>> remove
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>>>>>> return values in the drivers (although my driver will
>>>>>> probably
>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>> this parameter and always return true since it
>> wouldn't
>>>> make
>>>>>>>>>>>>>> sense
>>>>>>>>>>>>>>>> for it
>>>>>>>>>>>>>>>>>>> to ever return false). We'd need another column in
>> the
>>>>>>>>>>>>>> storage_pool
>>>>>>>>>>>>>>>>> table
>>>>>>>>>>>>>>>>>>> to store this value.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yes, I think we should have a parameter added to the
>>>>>>>>>>>>>>> createStoragePool
>>>>>>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
>>>>>>>>>> configured
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> management when their underlying drivers support it.
>> To
>>>>>>>>>> simplify
>>>>>>>>>>>>>>>> things,
>>>>>>>>>>>>>>>>>> this flag should only be mutable when the DataStore is
>>>>>>>>>> created. It
>>>>>>>>>>>>>>>> would be
>>>>>>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to
>>>> unmanaged
>>>>>>>> after
>>>>>>>>>>>>>>>> creation.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
>>>>>>>> working
>>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>> to review (in that document), if you'd like to check it
>>>> out.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed
>>>> together to
>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor
>>>> throttled
>>>>>>>> IOPs
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the
>> potential
>>>> for
>>>>>>>>>>>>>>>> confusion,
>>>>>>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these
>> features
>>>> will
>>>>>>>>>>>>>> help
>>>>>>>>>>>>>>>>>>>>> minimize any potential confusion.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will
>> work
>>>>>>>>>> itself
>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
>>>>>> hypervisor
>>>>>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google
>>>> doc I
>>>>>>>> sent
>>>>>>>>>>> to
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> other day to see how the GUI looks for
>> provisioned
>>>>>>>> storage
>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-
>> mail
>>>>>> list
>>>>>>>>>>>>>> and we
>>>>>>>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add
>>>> Disk
>>>>>>>>>>> Offering
>>>>>>>>>>>>>>>> dialog.
>>>>>>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS,
>> so
>>>> they
>>>>>>>>>>> should
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say,
>>>> only
>>>>>>>>>> use
>>>>>>>>>>>>>> Max
>>>>>>>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for
>>>> storage
>>>>>>>>>> that
>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size
>> field,
>>>>>>>>>> the
>>>>>>>>>>>>>>>> admin can
>>>>>>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max,
>> and
>>>>>>>> Burst
>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's
>>>> feature
>>>>>>>> or
>>>>>>>>>>>>>> mine,
>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> decided by the admin).
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two
>>>> features
>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> speed
>>>>>>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express
>> the
>>>>>>>> IOPS
>>>>>>>>>> for
>>>>>>>>>>>>>>>> QoS in
>>>>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
>>>>>>>>>> read/write
>>>>>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
>>>>>>>>>>>>>>> Storage/Hypervisor
>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the
>>>> behavior
>>>>>>>> of
>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see
>>>> the
>>>>>>>>>>>>>> following
>>>>>>>>>>>>>>>>>> issues
>>>>>>>>>>>>>>>>>>>>>>> outstanding:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as
>>>> discrete
>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are
>> expressed
>>>> as
>>>>>>>> total
>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS
>>>> attached
>>>>>>>> to
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for
>> throttled
>>>> and
>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>> that will permit providers to discriminate these
>>>>>>>>>> guaranteed
>>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
>>>>>>>>>> provisioned
>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would
>>>> neatly
>>>>>>>>>> solve
>>>>>>>>>>>>>>>> items 1
>>>>>>>>>>>>>>>>>>>>> and 2.
>>>>>>>>>>>>>>>>>>>>>>> Since the two facilities would not be permitted
>> to
>>>>>>>> operate
>>>>>>>>>>>>>>>> together,
>>>>>>>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I
>> think
>>>> item
>>>>>>>> 3
>>>>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>>>> resolved
>>>>>>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage
>> records.
>>>> As
>>>>>>>>>> for
>>>>>>>>>>>>>>>> Item 4,
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>>>>>>>> not familiar with how these two enhancements
>> are
>>>>>>>>>>> depicted in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
>>>>>>>> enhancements
>>>>>>>>>>> for
>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>>>>>>>>>>> <us...@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> John and Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I was busy working on other issues
>> (CLOUDSTACK-
>>>>>>>>>>> 2780/2729,
>>>>>>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-
>>>> 2823 ,
>>>>>>>>>>>>>>> CLOUDSTACK-
>>>>>>>>>>>>>>>>> 2875 ) this
>>>>>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes
>>>> tomorrow,
>>>>>>>> and
>>>>>>>>>>>>>> ask
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> second
>>>>>>>>>>>>>>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
>>>>>>>> <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going
>> with
>>>> this,
>>>>>>>>>> John,
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>>>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager
>>>> determining
>>>>>>>>>>> whether or
>>>>>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
>>>> StoragePoolType,
>>>>>>>> but an
>>>>>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
>>>> allocate
>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an
>>>> isManaged()
>>>>>>>>>>> method
>>>>>>>>>>>>>>>>> added to
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it
>> returns
>>>> true,
>>>>>>>>>>>>>> then the
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or
>>>> whatever
>>>>>>>> name)
>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface,
>>>> as
>>>>>>>> well)
>>>>>>>>>>> and
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to
>>>> create,
>>>>>>>> say
>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>> SR?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() :
>> boolean
>>>> on
>>>>>>>>>>>>>>>>> DataStoreDriver.
>>>>>>>>>>>>>>>>>>>>> Since
>>>>>>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to
>> primary
>>>>>>>> storage,
>>>>>>>>>>> I
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage
>> (Storage
>>>> is a
>>>>>>>>>>> straw
>>>>>>>>>>>>>>> man
>>>>>>>>>>>>>>>>> term
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> something other than volume) methods to
>>>>>>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is
>> a
>>>>>>>> mutable
>>>>>>>>>>>>>>> property
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> DataStore
>>>>>>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the
>> underlying
>>>>>>>>>>>>>>> DataStoreDriver
>>>>>>>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
>>>>>>>>>>> manageability
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> devices.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR,
>>>> the
>>>>>>>> Xen
>>>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> responsible for composing
>> DataStore/Volume
>>>>>>>> methods
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> create any
>>>>>>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR.
>> There
>>>>>>>>>> should
>>>>>>>>>>>>>> be no
>>>>>>>>>>>>>>>>>>>>>>>>> dependencies
>>>>>>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As
>> I
>>>> said
>>>>>>>>>>>>>> earlier,
>>>>>>>>>>>>>>>> such
>>>>>>>>>>>>>>>>>>>>>>>>> circular
>>>>>>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
>>>>>>>> unmaintainable
>>>>>>>>>>> mess.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page
>>>> with
>>>>>>>> you.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>>>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a
>>>> Storage
>>>>>>>> layer
>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> supports n
>>>>>>>>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors
>> of
>>>> m
>>>>>>>>>>>>>> hypervisors.
>>>>>>>>>>>>>>>>>> Such
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
>>>>>>>> untestable
>>>>>>>>>>>>>>> beast.
>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven
>> to
>>>>>>>> evolve
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager
>>>> determining
>>>>>>>>>>> whether
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
>>>> StoragePoolType,
>>>>>>>> but
>>>>>>>>>>> an
>>>>>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
>>>> allocate
>>>>>>>>>>>>>> resources
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device
>>>> driver
>>>>>>>>>> needs
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> indicate
>>>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
>>>>>>>> Finally, I
>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to
>> elect
>>>> to
>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver
>> is
>>>>>>>> capable
>>>>>>>>>>>>>>> managing
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
>>>>>>>>>>> unmanaged).
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume
>> should
>>>> be
>>>>>>>> done
>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed
>> by
>>>> the
>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>> layer to
>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
>>>>>>>> create a
>>>>>>>>>>>>>> file,
>>>>>>>>>>>>>>>> etc).
>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the
>>>> Xen
>>>>>>>>>>>>>>>> hypervisor, as
>>>>>>>>>>>>>>>>>>>>> such,
>>>>>>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation.
>>>> From
>>>>>>>> my
>>>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>>>> nothing
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
>>>>>>>> content.
>>>>>>>>>>> From
>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides
>>>> the
>>>>>>>> means
>>>>>>>>>>> to
>>>>>>>>>>>>>>>> query
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the content
>> by
>>>>>>>> higher-
>>>>>>>>>>> level
>>>>>>>>>>>>>>>>> layers
>>>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be
>> a
>>>>>>>>>>> composition
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on
>>>> the
>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> protocol
>>>>>>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
>>>>>>>>>>> implemented
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>> patch,
>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code
>> alone.
>>>> In
>>>>>>>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and
>> send it
>>>> to
>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor.
>>>>>>>>>>>>>>>>>> Before
>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if
>>>> we're
>>>>>>>>>>> dealing
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>> (or
>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if
>> so
>>>> -
>>>>>>>>>> send a
>>>>>>>>>>>>>>>> "Create
>>>>>>>>>>>>>>>>>>>>> SR"
>>>>>>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns
>> OK,
>>>> we
>>>>>>>>>>> would
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>> proceed
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same
>>>> and
>>>>>>>> we just
>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called
>> for
>>>> this
>>>>>>>>>>>>>> particular
>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer
>> today
>>>> that
>>>>>>>>>>> outlines
>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will
>>>> probably
>>>>>>>> be
>>>>>>>>>> of
>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> you, as
>>>>>>>>>>>>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you
>> (you
>>>>>>>> should
>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John
>>>> Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I
>> some
>>>>>>>>>>> thoughts,
>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against
>> the
>>>>>>>>>> solidfire,
>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to
>> collect
>>>> them
>>>>>>>> in a
>>>>>>>>>>>>>>>>> coherent
>>>>>>>>>>>>>>>>>>>>> form
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
>>>>>>>> introduces
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
>>>>>>>> introduces
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages
>>>> the
>>>>>>>> plug-in
>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>> incomplete,
>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a
>> managed
>>>> iSCSI
>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
>>>>>>>> spending on
>>>>>>>>>>>>>> this.
>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>> Do
>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
>>>>>>>> accomplish
>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>> you're
>>>>>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John
>>>> Burwell
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
>>>>>>>> introduces
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
>>>>>>>> management
>>>>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>>>>> (i.e.
>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an
>> operator
>>>> want
>>>>>>>> it
>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should
>>>> end up
>>>>>>>> with
>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> semantic
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to
>> be
>>>>>>>> managed.
>>>>>>>>>>> I
>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage
>>>> types
>>>>>>>> from
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike
>>>> Tutkowski
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in
>> red.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM,
>> John
>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the
>>>> following
>>>>>>>>>>> questions
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
>> iSCSI
>>>>>>>> LUNs
>>>>>>>>>>> and
>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>> volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a
>>>> single
>>>>>>>> LUN
>>>>>>>>>>> (LUN
>>>>>>>>>>>>>>> 0).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
>> SAN
>>>>>>>> Volumes
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack
>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
>>>>>>>> management
>>>>>>>>>>>>>>> server or
>>>>>>>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-
>> in,
>>>> a
>>>>>>>> SAN
>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> (same
>>>>>>>>>>>>>>>>>> as a
>>>>>>>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server
>>>> (via
>>>>>>>> the
>>>>>>>>>>> plug-in)
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> first
>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
>>>>>>>> hypervisor.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the
>> SolidFire
>>>> plug-
>>>>>>>> in,
>>>>>>>>>>> but
>>>>>>>>>>>>>>>> still
>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do
>>>> this
>>>>>>>> already
>>>>>>>>>>>>>>> today
>>>>>>>>>>>>>>>>>> (prior
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN
>>>> volume
>>>>>>>> and -
>>>>>>>>>>> in
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> case -
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN
>> volume.
>>>>>>>> While
>>>>>>>>>>> you
>>>>>>>>>>>>>>> can do
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce
>>>> storage
>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
>>>>>>>> management
>>>>>>>>>>> server
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used,
>> the
>>>>>>>> SAN
>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the
>>>> plug-
>>>>>>>> in).
>>>>>>>>>>> There
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping
>> between
>>>> a
>>>>>>>> SAN
>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> and a
>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage
>>>> vendor
>>>>>>>> that
>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>> true
>>>>>>>>>>>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these
>>>> pieces
>>>>>>>> are
>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure
>>>> that
>>>>>>>>>>> multiple
>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the
>> underlying
>>>>>>>> storage."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we
>> need
>>>> to
>>>>>>>> map a
>>>>>>>>>>>>>>> single
>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We
>>>> cannot
>>>>>>>> have
>>>>>>>>>>>>>>> multiple
>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and
>> still
>>>>>>>>>>> guarantee
>>>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single
>>>> SAN
>>>>>>>>>>> volume
>>>>>>>>>>>>>>> house
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do
>>>> that
>>>>>>>> today
>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>> any of
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM,
>>>> Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate
>> a
>>>> SAN
>>>>>>>>>>> volume
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will
>> be
>>>>>>>> created."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on
>> this.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM,
>>>> John
>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part
>> which
>>>>>>>>>>> concerns me
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into
>>>> storage
>>>>>>>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM,
>> Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the
>> iSCSI
>>>> type
>>>>>>>> is
>>>>>>>>>>> today
>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>> (as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer
>> (when
>>>> you
>>>>>>>> have
>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> PreSetup an
>>>>>>>>>>>>>>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>>>>>>>> volume
>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> VMware, it
>>>>>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>>>>>>>> volume
>>>>>>>>>>> from
>>>>>>>>>>>>>>> KVM,
>>>>>>>>>>>>>>>>> it uses
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and
>> Edison's
>>>>>>>> thinking
>>>>>>>>>>>>>>> here
>>>>>>>>>>>>>>>>> was to
>>>>>>>>>>>>>>>>>>>>> make a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this
>>>> dynamic
>>>>>>>> ability
>>>>>>>>>>>>>>> Edison
>>>>>>>>>>>>>>>>> added
>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should
>> be
>>>>>>>> more
>>>>>>>>>>>>>>> specificy,
>>>>>>>>>>>>>>>>> though:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27
>> PM,
>>>>>>>> Mike
>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
>>>>>>>> shouldn't
>>>>>>>>>>> know
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> care
>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be
>> able
>>>> to
>>>>>>>> answer
>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here
>> is
>>>> that
>>>>>>>> the
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so
>>>> that
>>>>>>>> the
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>> knows
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to
>> removing
>>>> that
>>>>>>>>>>> constraint
>>>>>>>>>>>>>>>>> and just
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just
>>>> need a
>>>>>>>> way
>>>>>>>>>>> for
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new
>> requirement
>>>> and
>>>>>>>>>>> perform
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24
>> PM,
>>>>>>>> John
>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM,
>>>> Mike
>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack
>>>> may
>>>>>>>> be
>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it
>>>> unmanaged by
>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> (e.g. the
>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
>>>>>>>> operator has
>>>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
>>>>>>>> reasoning
>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
>>>>>>>> creating a
>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>> volume via
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
>>>>>>>> hypervisor)
>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> work unless
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the
>> form
>>>> of,
>>>>>>>> say,
>>>>>>>>>>> an
>>>>>>>>>>>>>>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that,
>>>> independent
>>>>>>>> of
>>>>>>>>>>>>>>>>> CloudStack, an
>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to
>>>> CloudStack,
>>>>>>>> and
>>>>>>>>>>>>>>> exposes it
>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of
>> that
>>>> LUN
>>>>>>>> over
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate
>>>> anymore
>>>>>>>> LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands
>>>> prior
>>>>>>>> to my
>>>>>>>>>>>>>>>>> changes, we
>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does
>> not
>>>> have
>>>>>>>> the
>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we
>> should
>>>>>>>> maybe
>>>>>>>>>>> have
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume
>>>> needs
>>>>>>>>>>> support
>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
>>>>>>>> specifically
>>>>>>>>>>> to
>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent
>> code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
>>>>>>>> management
>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
>>>>>>>> determine
>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow
>>>> the
>>>>>>>>>>>>>>> interrogation of
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
>>>>>>>> device
>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>> shouldn't
>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -
>> -
>>>>>>>> simply be
>>>>>>>>>>> able
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> answer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>> 1:01
>>>> PM,
>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as
>> follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers
>> the
>>>>>>>> SolidFire
>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once
>>>> this is
>>>>>>>>>>> done, a
>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a
>> Disk
>>>>>>>> Offering
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
>>>>>>>> Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates
>> a
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> volume. This
>>>>>>>>>>>>>>>>>>>>> leads
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches
>>>> the
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> volume to a
>>>>>>>>>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
>>>>>>>> framework
>>>>>>>>>>> calling
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system
>> (in
>>>> my
>>>>>>>> case, a
>>>>>>>>>>>>>>> SAN).
>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
>>>>>>>> applicable
>>>>>>>>>>> data
>>>>>>>>>>>>>>>>> (like the
>>>>>>>>>>>>>>>>>>>>> IQN of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
>>>>>>>> invoked if
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After
>> the
>>>>>>>> attach,
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a
>> detach
>>>> disk)
>>>>>>>> and
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach
>>>> logic
>>>>>>>> is run
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> detects
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs
>> "assistance"
>>>> in
>>>>>>>> the
>>>>>>>>>>> form
>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>>>> 12:54
>>>>>>>> PM,
>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we
>> are in
>>>>>>>> sync on
>>>>>>>>>>>>>>>>> terminology,
>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to
>> the
>>>>>>>> physical
>>>>>>>>>>>>>>>>> allocation on
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say
>> 'volume',
>>>> I
>>>>>>>> try to
>>>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>>>> 'SAN
>>>>>>>>>>>>>>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the
>> 'volume'
>>>> the
>>>>>>>> end
>>>>>>>>>>> user
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> make in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack
>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>>>> 12:50
>>>>>>>> PM,
>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here
>> may
>>>>>>>> very
>>>>>>>>>>> well
>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>> sense, but
>>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should
>>>> draw
>>>>>>>> Edison
>>>>>>>>>>> in on
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> conversation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to
>> suggest
>>>> the
>>>>>>>>>>>>>>> approach I
>>>>>>>>>>>>>>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013
>> at
>>>> 12:42
>>>>>>>> PM,
>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
>>>>>>>> combining
>>>>>>>>>>> two
>>>>>>>>>>>>>>>>> distinct
>>>>>>>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and
>>>> storage
>>>>>>>>>>> protocols.
>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>> cases, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one
>> allows
>>>> the
>>>>>>>>>>> system to
>>>>>>>>>>>>>>>>>> create/delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the
>> other
>>>>>>>> requires
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> volume to
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the
>>>> CloudStack
>>>>>>>>>>> context.
>>>>>>>>>>>>>>> To
>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology,
>> volume,
>>>> in
>>>>>>>> these
>>>>>>>>>>>>>>>>> definitions,
>>>>>>>>>>>>>>>>>>>>> refers to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the
>>>> device,
>>>>>>>>>>> correct?
>>>>>>>>>>>>>>>>> Minimally,
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
>>>>>>>> storage
>>>>>>>>>>> device
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> move bits
>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete
>> bits,
>>>> etc.
>>>>>>>>>>>>>>> Optionally, a
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
>>>> CloudStack.
>>>>>>>>>>> Therefore,
>>>>>>>>>>>>>>>>> we can have
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we
>> store a
>>>> Xen
>>>>>>>> SR,
>>>>>>>>>>> and
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> can have a
>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
>>>>>>>> CloudStack is
>>>>>>>>>>>>>>> capable
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> allocating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally,
>>>> while
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>> be able
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose
>> to
>>>>>>>> leave it
>>>>>>>>>>>>>>>>> unmanaged by
>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many
>>>> services,
>>>>>>>> and
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> operator has
>>>>>>>>>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
>>>>>>>> CloudStack).
>>>>>>>>>>> Does
>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my
>> thoughts
>>>>>>>> above
>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> reasonable, it
>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the
>> management
>>>>>>>>>>> concerns
>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with
>> an
>>>>>>>>>>> attached
>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
>>>> CloudStack,
>>>>>>>> and
>>>>>>>>>>>>>>> establish
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a
>> device
>>>>>>>> separate
>>>>>>>>>>> that
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> associated
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in
>> place,
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> drivers can
>>>>>>>>>>>>>>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when
>>>> they
>>>>>>>>>>> encounter
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
>>>> operations
>>>>>>>>>>> exposed
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> the driver to
>>>>>>>>>>>>>>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these
>>>> thoughts/concepts
>>>>>>>> make
>>>>>>>>>>>>>>> sense,
>>>>>>>>>>>>>>>>> then we can
>>>>>>>>>>>>>>>>>>>>> sit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed
>> design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>> 5:25
>>>> PM,
>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the
>>>> difference
>>>>>>>>>>> between
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The
>> admin
>>>>>>>> has to
>>>>>>>>>>> go in
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At
>> this
>>>>>>>> point in
>>>>>>>>>>> time,
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it
>> is
>>>> pre-
>>>>>>>>>>> allocated).
>>>>>>>>>>>>>>>>> Future
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR
>>>> that
>>>>>>>> was
>>>>>>>>>>> created
>>>>>>>>>>>>>>>>> behind the
>>>>>>>>>>>>>>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The
>>>> admin
>>>>>>>> has
>>>>>>>>>>> to
>>>>>>>>>>>>>>> go in
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will
>>>> create
>>>>>>>> and
>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>> volumes on
>>>>>>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
>>>>>>>> enabled via
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a
>> CloudStack
>>>>>>>> volume
>>>>>>>>>>> that
>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>> created,
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a
>>>> new
>>>>>>>>>>> volume.
>>>>>>>>>>>>>>> After
>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>>> done,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
>>>>>>>> question is
>>>>>>>>>>> called.
>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point
>> because
>>>> the
>>>>>>>>>>> volume
>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013
>>>> at
>>>>>>>> 3:21
>>>>>>>>>>> PM,
>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new
>>>> terms,
>>>>>>>> so I
>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage:
>>>> Storage
>>>>>>>> that is
>>>>>>>>>>>>>>> pre-
>>>>>>>>>>>>>>>>> allocated
>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a
>> SAN),
>>>> then
>>>>>>>> a
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
>>>>>>>> XenServer
>>>>>>>>>>> SR),
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
>>>>>>>> CloudStack.
>>>>>>>>>>> Disks
>>>>>>>>>>>>>>> (VDI)
>>>>>>>>>>>>>>>>> are later
>>>>>>>>>>>>>>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
>>>>>>>> needed. In
>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>> cases, the
>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>>>>>>>> structure is
>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>> in place
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>> hypervisor
>>>> data
>>>>>>>>>>> structure
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> the VM in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
>>>>>>>> Storage
>>>>>>>>>>> that is
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent
>>>> storage,
>>>>>>>> this
>>>>>>>>>>> could
>>>>>>>>>>>>>>> be a
>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>> (not
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
>>>>>>>> hypervisor
>>>>>>>>>>> data
>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>> must be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is
>>>> performed
>>>>>>>>>>> because
>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>> types of
>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a
>>>> hypervisor
>>>>>>>> data
>>>>>>>>>>>>>>>>> structure by an
>>>>>>>>>>>>>>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an
>> SR
>>>> on
>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within
>> the
>>>> SR
>>>>>>>> to the
>>>>>>>>>>> VM
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013
>>>> at
>>>>>>>> 3:13
>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>> John Burwell
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>>>>>>>>>>> implementation of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen
>> ISCSI
>>>>>>>> which
>>>>>>>>>>> why I
>>>>>>>>>>>>>>> ask
>>>>>>>>>>>>>>>>> about the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the
>> question --
>>>>>>>> what is
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> definition
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
>> at
>>>> 5:10
>>>>>>>> PM,
>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I
>> know,
>>>> the
>>>>>>>> iSCSI
>>>>>>>>>>> type
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> uniquely
>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
>>>>>>>> Primary
>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>>>> directly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you
>> to
>>>>>>>> skip the
>>>>>>>>>>>>>>> step of
>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>> repository
>>>>>>>> based
>>>>>>>>>>> on
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>> target as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think
>>>> this is
>>>>>>>> only
>>>>>>>>>>>>>>>>> supported for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors,
>> you
>>>>>>>> must
>>>>>>>>>>> first go
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really
>>>> know
>>>>>>>> what
>>>>>>>>>>> RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013
>>>>>>>> at
>>>>>>>>>>> 2:13
>>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading
>>>> through
>>>>>>>> the
>>>>>>>>>>> code,
>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>> types?
>>>>>>>> Why
>>>>>>>>>>> isn't
>>>>>>>>>>>>>>> RBD
>>>>>>>>>>>>>>>>> considered
>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>> 2013,
>>>> at
>>>>>>>> 3:46
>>>>>>>>>>> PM,
>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new
>> type
>>>> of
>>>>>>>>>>> storage is
>>>>>>>>>>>>>>>>> defined in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called
>>>> Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static
>>>> enum
>>>>>>>>>>>>>>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Filesystem(false),
>>>>>>>> //
>>>>>>>>>>> local
>>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> NetworkFilesystem(true),
>>>>>>>>>>>>>>> //
>>>>>>>>>>>>>>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IscsiLUN(true),
>>>> //
>>>>>>>>>>> shared
>>>>>>>>>>>>>>> LUN,
>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true),
>> //
>>>> for
>>>>>>>> e.g.,
>>>>>>>>>>> ZFS
>>>>>>>>>>>>>>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false),
>> //
>>>> for
>>>>>>>> iso
>>>>>>>>>>> image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false),
>> //
>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true),
>> //
>>>>>>>>>>> VMware
>>>>>>>>>>>>>>> VMFS
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> PreSetup(true), //
>>>>>>>> for
>>>>>>>>>>>>>>>>> XenServer, Storage
>>>>>>>>>>>>>>>>>>>>> Pool
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false),
>> //
>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Dynamic(true);
>>>> //
>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean
>>>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> StoragePoolType(boolean
>>>>>>>>>>>>>>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared
>> =
>>>>>>>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public
>> boolean
>>>>>>>>>>> isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return
>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun 3,
>>>>>>>> 2013 at
>>>>>>>>>>> 1:41
>>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>> example,
>>>>>>>> let's say
>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> company
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
>>>>>>>> leverage its
>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> Service
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>> storage, as
>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>> They would
>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as
>> I've
>>>>>>>> made
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>> changes to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support
>>>> their
>>>>>>>> plug-
>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun
>>>> 3,
>>>>>>>> 2013
>>>>>>>>>>> at
>>>>>>>>>>>>>>> 1:39
>>>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry
>> to
>>>>>>>> imply
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> XenServer code
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>> XenServer
>>>>>>>>>>> attach
>>>>>>>>>>>>>>> logic is
>>>>>>>>>>>>>>>>> now aware
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and
>>>> SolidFire is
>>>>>>>> an
>>>>>>>>>>>>>>>>> implementation of
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is
>>>> new
>>>>>>>> to 4.2
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison
>>>> created
>>>>>>>> a
>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> framework that
>>>>>>>>>>>>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> volumes
>>>>>>>>>>> dynamically.
>>>>>>>>>>>>>>>>> However, when I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April,
>> we
>>>>>>>> realized
>>>>>>>>>>> that
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> was not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
>>>>>>>> CloudStack
>>>>>>>>>>> could
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>> with these
>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed
>> to
>>>>>>>>>>> recognize
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> new type of
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> hypervisor
>>>> data
>>>>>>>>>>> structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>> Jun 3,
>>>>>>>> 2013
>>>>>>>>>>> at
>>>>>>>>>>>>>>> 1:28
>>>>>>>>>>>>>>>>> PM, John
>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>> generally
>>>>>>>> odd
>>>>>>>>>>> to me
>>>>>>>>>>>>>>>>> that any
>>>>>>>>>>>>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> understand or
>>>>>>>> care
>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>> details. I
>>>>>>>>>>>>>>>>>>>>> expect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose
>> a
>>>> set
>>>>>>>> of
>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>> that can be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> implementations to
>>>>>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>>>>>> space/create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>> invert
>>>>>>>> this
>>>>>>>>>>>>>>>>> dependency, we are
>>>>>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> problem
>>>> that
>>>>>>>> will
>>>>>>>>>>> make
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> enhance.
>>>> Am
>>>>>>>> I
>>>>>>>>>>>>>>>>> understanding that the
>>>>>>>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located
>> in
>>>> the
>>>>>>>>>>>>>>>>> CitrixResourceBase
>>>>>>>>>>>>>>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>> Jun 3,
>>>>>>>>>>> 2013 at
>>>>>>>>>>>>>>> 3:12
>>>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
>> delve
>>>> into
>>>>>>>> this
>>>>>>>>>>> in a
>>>>>>>>>>>>>>> bit
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior
>> to
>>>> 4.2
>>>>>>>> and
>>>>>>>>>>> aside
>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> one setup
>>>>>>>>>>>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first
>>>> create
>>>>>>>> a
>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set
>> up
>>>> a
>>>>>>>> data
>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>> to make
>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> repository on
>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>> or a
>>>>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
>>>> this
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>> Edison's
>>>>>>>> new
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> framework,
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> you
>>>> can
>>>>>>>> easily
>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>> a 1:1
>>>>>>>>>>>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> volume
>>>> and
>>>>>>>> the
>>>>>>>>>>> VM's
>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>> (necessary
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
>> can
>>>> now
>>>>>>>> write
>>>>>>>>>>> a
>>>>>>>>>>>>>>> plug-
>>>>>>>>>>>>>>>>> in that is
>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> volumes
>>>> as
>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>> problem
>>>>>>>> that
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> deleting
>>>> the
>>>>>>>>>>>>>>> hypervisor-
>>>>>>>>>>>>>>>>> specific data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>>>> being
>>>>>>>> the
>>>>>>>>>>> case,
>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>> enhancing
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> worked
>>>> out
>>>>>>>> and
>>>>>>>>>>>>>>>>> submitted. I've got
>>>>>>>>>>>>>>>>>>>>> ESX(i)
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we
>>>> extend
>>>>>>>> the
>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does
>> that
>>>>>>>> help a
>>>>>>>>>>> bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Mon,
>>>> Jun
>>>>>>>> 3,
>>>>>>>>>>> 2013 at
>>>>>>>>>>>>>>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>> storage
>>>>>>>>>>> plug-in -
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> itself - is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>> issue is
>>>>>>>> with
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> volume-attach
>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> framework
>>>>>>>> calls
>>>>>>>>>>> into
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> plug-in to
>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>> the
>>>>>>>> time
>>>>>>>>>>>>>>> comes to
>>>>>>>>>>>>>>>>> attach the
>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has
>> to
>>>> be
>>>>>>>> smart
>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>> to recognize
>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (where
>>>> the
>>>>>>>>>>> volume
>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>> just been
>>>>>>>>>>>>>>>>>>>>> created)
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> repository
>>>>>>>> (for
>>>>>>>>>>>>>>>>> XenServer) or a
>>>>>>>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> volume
>>>> that
>>>>>>>> was
>>>>>>>>>>> just
>>>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>> been
>>>>>>>>>>> spending
>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>> of my time
>>>>>>>>>>>>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>> agent
>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does
>>>> that
>>>>>>>> clear it
>>>>>>>>>>> up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Mon,
>>>>>>>> Jun 3,
>>>>>>>>>>> 2013
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> 12:48 PM,
>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>> you
>>>>>>>> explain
>>>>>>>>>>>>>>> why
>>>>>>>>>>>>>>>>> the the storage
>>>>>>>>>>>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Jun
>>>> 3,
>>>>>>>> 2013,
>>>>>>>>>>> at
>>>>>>>>>>>>>>> 1:21
>>>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Yes,
>>>>>>>>>>> ultimately I
>>>>>>>>>>>>>>>>> would like to
>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> supports.
>>>>>>>> I
>>>>>>>>>>> think
>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>> just out of
>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Right
>>>>>>>> now this
>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> regards
>>>>>>>> to 4.2
>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>> freeze, I
>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Also,
>>>> just
>>>>>>>> to
>>>>>>>>>>> be
>>>>>>>>>>>>>>> clear,
>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan
>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> support
>>>>>>>>>>> Compute
>>>>>>>>>>>>>>>>> Offerings post
>>>>>>>>>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> Mon,
>>>>>>>> Jun 3,
>>>>>>>>>>>>>>> 2013
>>>>>>>>>>>>>>>>> at 11:14 AM,
>>>>>>>>>>>>>>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is
>>>> there
>>>>>>>> any
>>>>>>>>>>> plan
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> supporting
>>>>>>>>>>>>>>>>>>>>> KVM in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> ---
>>>>>>>> Original
>>>>>>>>>>>>>>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> From:
>>>>>>>> "Mike
>>>>>>>>>>>>>>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> To:
>>>>>>>>>>>>>>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Sent:
>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>> June
>>>>>>>>>>>>>>>>> 3, 2013
>>>>>>>>>>>>>>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Subject:
>>>>>>>> Re:
>>>>>>>>>>>>>>> [MERGE]
>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>> agree
>>>>>>>> on
>>>>>>>>>>>>>>> merging
>>>>>>>>>>>>>>>>> Wei's feature
>>>>>>>>>>>>>>>>>>>>>>> first,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>> his
>>>>>>>> feature
>>>>>>>>>>> is
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> KVM only,
>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> KVM in
>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>> Mon,
>>>>>>>> Jun
>>>>>>>>>>> 3,
>>>>>>>>>>>>>>> 2013
>>>>>>>>>>>>>>>>> at 8:55 AM,
>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> For
>>>>>>>> the
>>>>>>>>>>> billing,
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>> no one works
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> the
>>>>>>>> billing
>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> themselves. They
>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> disk_offering of
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> VMs and
>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> to
>>>> tell
>>>>>>>> user
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> exact
>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> rate
>>>>>>>>>>> limitation
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> nics as
>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> is a
>>>>>>>> part of
>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Sorry
>>>>>>>> my
>>>>>>>>>>>>>>>>> misunstand the second
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Agree
>>>>>>>> with
>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>> you said about
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> -
>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> 2013/6/3
>>>>>>>>>>> John
>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>>>> Jun 3,
>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> 2:13 AM, Wei
>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> Hi
>>>>>>>> John,
>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> I
>>>>>>>> hope
>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>> aswer helps you.
>>>>>>>>>>>>>>>>>>>>> I am
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> (1) I
>>>>>>>>>>> think
>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>> depend
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>> limitation.
>>>>>>>>>>>>>>> Please
>>>>>>>>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> get
>>>>>>>> the
>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>> virtual
>>>>>>>>>>>>>>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> Going by
>>>>>>>>>>> the
>>>>>>>>>>>>>>> AWS
>>>>>>>>>>>>>>>>> model,
>>>>>>>>>>>>>>>>>> customers
>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>>>> as well as,
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/).
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> option to
>>>>>>>>>>>>>>> employ
>>>>>>>>>>>>>>>>> similar cost
>>>>>>>>>>>>>>>>>>>>>>> models.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> billing
>>>>>>>>>>> model
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the current
>>>>>>>>>>>>>>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> (2)
>>>>>>>> Do
>>>>>>>>>>> you
>>>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>>>> IOPS runtime
>>>>>>>>>>>>>>>>>>>>>>> change?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>> limitation for
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>> virtual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>> not support
>>>>>>>>>>>>>>>>>>>>>>> changing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>> (computer
>>>>>>>>>>>>>>>>> offering or disk
>>>>>>>>>>>>>>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> I
>>>>>>>> meant at
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Java interface
>>>>>>>>>>>>>>>>>>>>> level.
>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> we
>>>>>>>> more
>>>>>>>>>>>>>>>>> generalize allocation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> describe
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> service guarantees
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> (3)
>>>>>>>> It is a
>>>>>>>>>>>>>>> good
>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>> Maybe it
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>> disk_io_throttling as Mike
>>>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>> type, I
>>>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> I
>>>> will
>>>>>>>>>>> expand
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> my thoughts
>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> touch
>>>>>>>>>>> points
>>>>>>>>>>>>>>>>> between these two
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> will
>>>>>>>> need
>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>> merged before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> coordination
>>>>>>>>>>>>>>>>> between the
>>>>>>>>>>>>>>>>>> branches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> disk_io_throttling)
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>> -
>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>> 2013/6/3
>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> The
>>>>>>>>>>> things I
>>>>>>>>>>>>>>>>> want to
>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> 1)
>>>>>>>> Is
>>>>>>>>>>> there
>>>>>>>>>>>>>>>>> value in
>>>>>>>>>>>>>>>>>> capturing
>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> data
>>>>>>>>>>> model
>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> 2)
>>>>>>>>>>> Should
>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>> be a common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>> provisioning
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> 3)
>>>>>>>> How
>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> conflicting
>>>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> storage device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> is
>>>>>>>> lead
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> believe (and
>>>>>>>>>>>>>>>>>> billed)
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> storage
>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> could
>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>> consistent
>>>>>>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>>>>>>> between a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> time,
>>>>>>>>>>> but a
>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>> modification
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> -
>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> On
>>>>>>>> Jun
>>>>>>>>>>> 2,
>>>>>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>>>> at 8:38 PM,
>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> Hi
>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> I
>>>>>>>>>>> believe
>>>>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>>>> feature deals
>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number
>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> the
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> My
>>>>>>>>>>> feature
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> focused on
>>>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> I
>>>>>>>> hope
>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> On
>>>>>>>> Sun,
>>>>>>>>>>> Jun
>>>>>>>>>>>>>>> 2,
>>>>>>>>>>>>>>>>> 2013 at 6:35
>>>>>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> My
>>>>>>>>>>>>>>> opinion is
>>>>>>>>>>>>>>>>> that no
>>>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> issues
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> been resolved
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> total
>>>>>>>>>>> Ops
>>>>>>>>>>>>>>> vs
>>>>>>>>>>>>>>>>> discrete
>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> re-
>>>>>>>>>>> reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido,
>>>>>>>>>>> I
>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>> think this
>>>>>>>>>>>>>>>>>>>>> criteria
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Also,
>>>>>>>>>>> how
>>>>>>>>>>>>>>>>> does this work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> also
>>>>>>>>>>>>>>> involves
>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>> where provisioned
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> -
>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> On
>>>>>>>>>>> Jun 1,
>>>>>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>>>> at 6:47 AM,
>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Sure. I
>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> change it next
>>>>>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> -
>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> 2013/6/1
>>>>>>>>>>>>>>>>> Wido den Hollander
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> On
>>>>>>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> Exactly. I
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> pushed the
>>>>>>>>>>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> If
>>>>>>>>>>> anyone
>>>>>>>>>>>>>>>>> object thems for
>>>>>>>>>>>>>>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> For
>>>>>>>>>>> the
>>>>>>>>>>>>>>> sake
>>>>>>>>>>>>>>>>> of clarity I
>>>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> the
>>>>>>>>>>> total
>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>> to R/W IOps
>>>>>>>>>>>>>>>>>>>>> asap
>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> only
>>>>>>>>>>> total
>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> You
>>>>>>>>>>> laid
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> groundwork for
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> however
>>>>>>>>>>>>>>>>> prevent that we
>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> -
>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> 2013/5/31
>>>>>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> On
>>>>>>>>>>>>>>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> +1 --
>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> enhancement must
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> don't
>>>>>>>>>>> see
>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>> it could be
>>>>>>>>>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> split
>>>>>>>>>>> total
>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>> into read
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> total
>>>>>>>>>>>>>>>>> unless/until we
>>>>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> What
>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>> meant was merging
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> 4.2
>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>> and add Read /
>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> will be
>>>>>>>>>>>>>>>>> released with Read
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> This is
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> make the May 31st
>>>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> (see
>>>>>>>>>>> other
>>>>>>>>>>>>>>>>> threads) then it
>>>>>>>>>>>>>>>>>>>>> won't
>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> I
>>>>>>>> also
>>>>>>>>>>>>>>>>> completely agree that
>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> disk
>>>>>>>>>>> I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> -
>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> On
>>>>>>>>>>> May 31,
>>>>>>>>>>>>>>>>> 2013, at 9:51 AM,
>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> On
>>>>>>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Hi
>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>> Good
>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> I
>>>>>>>>>>> thought
>>>>>>>>>>>>>>>>> about at the
>>>>>>>>>>>>>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> difference
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> read and write
>>>>>>>>>>>>>>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> care
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> difference of sent
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> That
>>>>>>>>>>>>>>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Disk
>>>>>>>>>>> I/O is
>>>>>>>>>>>>>>>>> much more
>>>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> Implementing
>>>>>>>>>>>>>>>>> it will be some
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> few
>>>>>>>>>>> days.
>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>> the deadline
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> that ,
>>>>>>>>>>> if
>>>>>>>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> It
>>>>>>>>>>> think it's
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> feature we
>>>>>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> window
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> have to make sure
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> it
>>>>>>>> in
>>>>>>>>>>> 4.3,
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> will confuse
>>>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> -
>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> 2013/5/31
>>>>>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> On
>>>>>>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> I
>>>>>>>>>>> would
>>>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> If
>>>>>>>>>>> nobody
>>>>>>>>>>>>>>>>> object, I will
>>>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> The
>>>>>>>>>>>>>>> purpose
>>>>>>>>>>>>>>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Virtual
>>>>>>>>>>>>>>>>> machines are running
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> share
>>>>>>>>>>>>>>> strage).
>>>>>>>>>>>>>>>>> Because of
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> iops),
>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> one
>>>>>>>>>>> VM
>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>> large disk
>>>>>>>>>>>>>>>>>>>>> operation,
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> other
>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>> running on the
>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> It
>>>>>>>> is
>>>>>>>>>>>>>>> neccesary
>>>>>>>>>>>>>>>>> to set the
>>>>>>>>>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> Looking at
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> code I see
>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Qemu
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> libvirt support
>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> IOps
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>> could benefit a
>>>>>>>>>>>>>>>>>>>>> lot of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> It's
>>>>>>>>>>> also
>>>>>>>>>>>>>>>>> strange, in the
>>>>>>>>>>>>>>>>>>>>> polling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> IOps,
>>>>>>>>>>> but
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> the throttling
>>>>>>>>>>>>>>>>>>>>> side
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Write
>>>>>>>>>>> IOps
>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> usually much
>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> like a
>>>>>>>>>>> valid
>>>>>>>>>>>>>>>>> use-case where
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> IOps
>>>>>>>>>>> vs
>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Since
>>>>>>>>>>> this
>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>> supports KVM
>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> value
>>>>>>>>>>> to at
>>>>>>>>>>>>>>>>> least have the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> this
>>>>>>>>>>> later
>>>>>>>>>>>>>>>>> would be a lot of
>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> If
>>>>>>>> a
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> write
>>>>>>>>>>> you
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> always sum
>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Can
>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> explain why you
>>>>>>>>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> The
>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> (1) set
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> maximum rate of
>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> (2)
>>>>>>>>>>> change
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> maximum rate
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> (3)
>>>>>>>>>>> limit
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> disk rate
>>>>>>>>>>>>>>>>>> (total
>>>>>>>>>>>>>>>>>>>>>>> bps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> JIRA
>>>>>>>>>>> ticket:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-
>>>> 1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 1192
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> FS (I
>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> Merge
>>>>>>>>>>>>>>> check
>>>>>>>>>>>>>>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> *
>>>>>>>> Did
>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> check the branch's
>>>>>>>>>>>>>>>>>>>>> RAT
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> *
>>>>>>>> Are
>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>> new dependencies
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> *
>>>>>>>>>>> What
>>>>>>>>>>>>>>>>> automated testing
>>>>>>>>>>>>>>>>>> (unit
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Unit
>>>>>>>>>>> tests
>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> *
>>>>>>>>>>> What
>>>>>>>>>>>>>>>>> testing has been done
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> (1) set
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> bytes rate and
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> (2)
>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>> deploy,
>>>>>>>>>>>>>>> stop,
>>>>>>>>>>>>>>>>> start, reboot,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> (3)
>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>> operations,
>>>>>>>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Attach,
>>>>>>>>>>>>>>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> To
>>>>>>>>>>> review
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> code, you can
>>>>>>>>>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Best
>>>>>>>>>>>>>>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-
>> 2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> -
>>>>>>>>>>> VM
>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> *Mike
>>>>>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> *Senior
>>>>>>>>>>>>>>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> e:
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>> o:
>>>>>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>> Advancing
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> way the world
>>>>>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Mike
>>>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Senior
>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> Advancing
>>>>>>>>>>> the
>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Mike
>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Senior
>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> Advancing the
>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Mike
>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Senior
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Advancing
>>>>>>>> the
>>>>>>>>>>> way
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Senior
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Advancing
>>>>>>>> the
>>>>>>>>>>> way
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> Developer, SolidFire
>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Advancing
>>>> the
>>>>>>>> way
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>> CloudStack
>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>>>> the
>>>>>>>> way
>>>>>>>>>>> the
>>>>>>>>>>>>>>> world
>>>>>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>> CloudStack
>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> the
>>>>>>>> way the
>>>>>>>>>>>>>>> world
>>>>>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>> way
>>>>>>>> the
>>>>>>>>>>> world
>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way
>>>> the
>>>>>>>>>>> world
>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>>>> the
>>>>>>>> world
>>>>>>>>>>> uses
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>>>>>> world
>>>>>>>>>>> uses
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>>>> uses
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>>>> uses
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>>>> uses
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>> uses
>>>>>>>> the
>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>> the
>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *(tm)*****
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*****
>>>>>>>>>>
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>>>>>
>>>>>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>>>>>
>>>>>>>>>> o: 303.746.7302****
>>>>>>>>>>
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *(tm)*****
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ****
>>>>>>>>>>
>>>>>>>>>> ** **
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*****
>>>>>>>>>>
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>>>>>
>>>>>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>>>>>
>>>>>>>>>> o: 303.746.7302****
>>>>>>>>>>
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *(tm)*****
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *(tm)*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *(tm)*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *(tm)*
>>>
>
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Monday, June 10, 2013 4:07 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> That's a good point about the "managed" property being storable in the
> storage_pool_details table. We don't need another column for it in the
> storage_pool table. In the current url field is where this kind of information
> can be passed along and it can be stored in the storage_pool_details table, if
> the plug-in wants to do so.
+1. CreateStoragePoolCmd->details field is the place to pass extra info about a storage pool to storage provider.
>
> Let's decide on how the hypervisor code should operate, though. Should we
> pass in info to the "attach" command to have it create an SR if need be?
> This is sort of how I did it in the patch code I submitted (but I used the
> StoragePoolType.Dynamic field, which we don't have to use).
Master branch doesn't carry a VolumeTO in attach command, which means, there is no place for storage provider to pass down some extra info to hypervisor resource.
While on object_store branch, it has the place to hook up with storage provider.
So there are two options:
1. Use your StoragePoolType.Dynamic as it is, and merge into master branch. After your patch is merged, and also object_store branch is merged, then we can remove StoragePoolType.Dynamic totally.
2. Rebase your patch on object_store branch, merge your patch after object_store. It will take extra time for you, I don't think it's worth the effort though, as I can consolidate your patch with object_store during object_store merge into master.
So I'd prefer option 1, as you only did few line of changes on the mgt server common code(regardless of iops), it should be easy for me to consolidate your patch with object_store branch.
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
That's a good point about the "managed" property being storable in the
storage_pool_details table. We don't need another column for it in the
storage_pool table. In the current url field is where this kind of
information can be passed along and it can be stored in the
storage_pool_details table, if the plug-in wants to do so.
Let's decide on how the hypervisor code should operate, though. Should we
pass in info to the "attach" command to have it create an SR if need be?
This is sort of how I did it in the patch code I submitted (but I used the
StoragePoolType.Dynamic field, which we don't have to use).
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: John Burwell [mailto:jburwell@basho.com]
> Sent: Monday, June 10, 2013 2:44 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Edison,
>
> Please see my response in-line below.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 4:45 PM, Edison Su <Ed...@citrix.com> wrote:
>
> >
> >
> >> -----Original Message-----
> >> From: John Burwell [mailto:jburwell@basho.com]
> >> Sent: Monday, June 10, 2013 1:25 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>
> >> Mike,
> >>
> >> Yes, a vendor may hand a non-managed plugin. For example, on the
> >> secondary storage side, S3 and Swift don't support the concept of being
> >> managed -- an operator is required to create a bucket and point
> CloudStack
> >> at it. Another example would be a generic iSCSI device (e.g. OpenFiler,
> >> FreeNAS) which CloudStack can not directly manage, but can consume a
> LUN
> >> that an operator has created for it.
> >
> > Manage or non-managed storage is handled by DataStoreLifeCyclce.
> > It depends on storage vendor to implement DataStoreLifeCyclce, which
> basically says how to initialize a storage, how to add a storage into a cluster
> etc.
>
> So, implementation of the DataStoreLifeCycle interface indicates that driver
> is capable of managing the device. It does not answer the question of
No, the interface only means there is a way to add a storage into cloudstack, no matter
It's managed or un-managed.
> whether or not the administrator/operator wants CloudStack to manage the
> device. Hence, the need for a flag on the StoragePool to indicate we should
Cloudstack don't need to care about the storage is managed or un-managed, what it cares about
Is how to create/delete something on it, it will call storage plugins to do all the storage related operations.
> be managing it. I am cool with the flag defaulting to true when the driver
> supports it, but we must provide the administrator/operator to ability to opt-
If the storage provider wants to opt-out or wants to do things differently based on input from administrator/operator,
The storage provider can add parameters in createstoragepoolcmd->details, details is a map, which can contain whatever information needed by the storage provider.
Let's say a storage provider which can support both managed/un-managed storage pool, when admin wants a storage pool managed by the storage pool, admin can set "managed" = "true" in the details, then
The storage provider will know, oh, admin wants a managed storage pool, so the DataStoreLifeCyclce->init will create a storage pool dynamically on the storage box. While if the "managed" = "false", then DataStoreLifeCyclce->init will not create a storage pool, instead, it will check the existence of the storage pool, if it's there, then return OK.
I don't think cloudstack mgt server needs to know, the storage pool is managed/unmanaged, it's up to the implementation of storage provider.
> out. Does this interface provide for a set of administrative credentials
> separate from the credentials used to access storage?
Yes, createstoragepoolcmd->details, is the place to add whatever information when creating a storage pool.
>
> Another question comes to mind -- why do we have a Dynamic
> StoragePoolType when this interface exists?
It shouldn't.
Storage provider can return a VolumeTO which can contain whatever information about the volume.
At the resource side, it may check VolumeTO, if the volume is not a VDI, VolumeTO can contain a field called "isManaged", which can be set to true by Solidfire's driver, while by default it's false.
If isManaged is true, then the resource code can create a SR and create a VDI on the fly.
>
> >
> > Take S3 as an example, if the S3 storage provider provides a
> DataStoreLifeCyclce which can create a bucket, then when admin adding a S3
> storage through API, mgt server will first call DataStoreLifeCyclce, which will
> create a bucket, then stored in the image_store_details table, then later on,
> all the referenced to this image store, will contain this bucket information.
>
> From an S3 perspective, creating a bucket from CloudStack would not a be
> security best practice. The general guidance would be for the bucket to
> created and associated to a dedicated CloudStack access/secret key pair to
> properly isolate data. Given this best practice and that only one bucket per
> S3 instance would be necessary for CloudStack operation, I think most
> administrators/operators would want to opt-out CloudStack managing S3
> buckets.
>
> >
> > Take an storage box from some vendor as an example, admin wants to
> create an ISCSI server on a storage box, which can be used by a
> cluster(multiple volumes on a giant LUN ), as long as the storage provider can
> provide a DataStoreLifeCyclce which can program the storage box, and create
> an ISCSI server, then stored the information in storage_pool table, then all
> the referenced to the storage pool will use the dynamically created ISCSI LUN.
> >
> > Take another storage box from some vendor as an example, admin wants
> to add the whole storage box into cloudstack, which can be used as per LUN
> per ISCSI, so in this case, DataStoreLifeCyclce won't create an ISCSI server
> during the initialize method call. Instead, for every createAsync, it will create
> an ISCSI LUN, stored the LUN information in volume table and volume_detail
> table. Whenever the LUN is used by hypervisor, the hypervisor needs to take
> extra care about the volume.
> >
> >
> >
> >
> >>
> >> From an operational perspective, an administrator may elect to say, "Hey,
> I
> >> know you can manage that device for me, but it is has a wider use than
> >> CloudStack can manage. Let me handle it, and just provide the
> information
> >> CloudStack needs." In this scenario, the operator is accepting a greater
> >> management burden in exchange for control. I don't think many will elect
> to
> >> manage these devices themselves, but this ability will be extremely
> >> important for the edge cases that require it. Speaking as someone who
> has
> >> run cloud operations, I can testify to this need -- nothing aggravates you
> >> more than a system insisting on managing a device or service without the
> >> opportunity to opt-out
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 3:19 PM, Mike Tutkowski
> <mi...@solidfire.com>
> >> wrote:
> >>
> >>> So, would a storage vendor write a non-CloudStack-managed plug-in and
> >> maybe
> >>> just implement the snapshot feature or something?
> >>>
> >>> In this case, they'd have to set the hypervisor up ahead of time with a
> >>> large portion of their storage, but they'd still get the vendor-snapshot
> >>> ability.
> >>>
> >>> Just trying to undertand this more.
> >>>
> >>> Thanks
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>
> >>>> Edison,
> >>>>
> >>>> TL;DR Always give operators/system administrators the ability to
> manage
> >>>> things themselves, if they like.
> >>>>
> >>>> As more manageable drivers emerge, I think it will be import to allow
> >>>> operators to opt-out of CloudStack device management. Therefore, a
> >> driver
> >>>> declares whether or not it can support device management (many, in
> fact
> >>>> most, won't). For those drivers that do support management, allow
> >>>> operators to opt-in/out of CloudStack management on a per device
> basis.
> >> To
> >>>> support this notion, we need a flag on the storage_pool table and
> >>>> associated classes to indicate whether or not the operator wants
> >> CloudStack
> >>>> to manage the device.
> >>>>
> >>>> Does this explanation make more sense?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
> >>>>
> >>>>> I don't understand why we need to add a "managed" in the UI and in
> >>>> "storage_pool" table?
> >>>>> AFAIK, what you guys trying to do is to enable
> >>>> attachVolume/DettachVolume if the volume is created on Solidfire,
> right?
> >>>>> If the volume is created on Solidfire already, by createAsync, then
> why
> >>>> we can't just send out all the information about the volume(it's
> volumeTO
> >>>> which will be returned by each storage driver) to hypervisor, then at
> the
> >>>> hypervisor side, check the volumeTO, if the volumeTO is managed,
> then
> >> doing
> >>>> whatever necessary operations(create SR, create VDI etc), then attach
> >> the
> >>>> VDI to volume.
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>> Sent: Monday, June 10, 2013 11:10 AM
> >>>>>> To: John Burwell
> >>>>>> Cc: Edison Su; dev@cloudstack.apache.org
> >>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>
> >>>>>> I believe we're on the same page, John.
> >>>>>>
> >>>>>> I would add a new parameter to the createStoragePool API
> command
> >> (let's
> >>>>>> call it "managed" and it can be true or false). If you pass in "true",
> >>>> but
> >>>>>> the driver does not support being managed by CloudStack, an
> >> exception is
> >>>>>> thrown by the driver. If you pass in "false", but the driver does not
> >>>>>> support being managed outside of CloudStack, an exception is
> thrown
> >> by
> >>>> the
> >>>>>> driver. It is possible one could write a driver that can be managed
> both
> >>>>>> inside or outside of CloudStack (the SolidFire driver requires being
> >>>>>> managed only by CloudStack).
> >>>>>>
> >>>>>> This managed value would end up in the storage_pool table in a new
> >>>> column
> >>>>>> ("managed").
> >>>>>>
> >>>>>> An example usage: After the storage framework calls on the driver's
> >>>>>> createAsync method, it will check to see if the driver is managed. If
> >>>> the
> >>>>>> driver is managed, the storage framework will invoke the proper
> (new)
> >>>>>> command of the hypervisor in use to set up, say, the SR for the SAN
> >>>> volume.
> >>>>>>
> >>>>>> There may be other cases when the isManaged() method is invoked
> on
> >> the
> >>>>>> driver and - if true is returned - the storage framework sends a
> >>>> particular
> >>>>>> command to the hypervisor in use.
> >>>>>>
> >>>>>> ********************
> >>>>>>
> >>>>>> As far as reworking the default storage plug-in (which sends
> commands
> >> to
> >>>>>> hypervisors) for 4.2, I would check with Edison on his schedule.
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell
> <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> I want to make sure we are on the same page regarding isManaged.
> I
> >> see
> >>>>>>> the addition of the following operations:
> >>>>>>>
> >>>>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates
> >> whether
> >>>>>> or
> >>>>>>> not the driver support/implements the management functions
> >>>>>>> * DataStore#isManaged() : boolean -- indicates whether or not a
> >> device
> >>>> is
> >>>>>>> managed by CloudStack
> >>>>>>>
> >>>>>>> Only DataStores associated with DataStoreDrivers that support
> >>>>>> management
> >>>>>>> would be able to have the management flag enabled. This
> behavior
> >>>> allows
> >>>>>>> operators to declare their intention to allow CloudStack to manage
> a
> >>>>>>> device. This behavior would need to be exposed as flag on the
> HTTP
> >>>> API.
> >>>>>>> Additionally, this flag would only be settable when a device is
> >>>>>>> defined/created in the system (i.e. once a device is managed by
> >>>> CloudStack,
> >>>>>>> you can't unmanaged it without dissociating it from CloudStack). In
> >>>> the
> >>>>>>> future, we may consider relaxing that rule, but, in the short term, it
> >>>>>>> greatly decreases the alternative flows.
> >>>>>>>
> >>>>>>> In terms of removing the dependencies from Storage to Hypervisor,
> I
> >>>>>> would
> >>>>>>> really like to see this accomplished in 4.2. It will be much more
> >>>>>>> difficult to address this issue post 4.2 because we will have more
> >>>> plugins
> >>>>>>> to address. I also see this issue as a critical architectural issues
> >>>> that
> >>>>>>> for the management server. I recommend we identify the
> >> dependencies
> >>>>>> that
> >>>>>>> need to be inverted, and see if we can divide the work across a few
> >>>> more
> >>>>>>> resources.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> >>>>>> <mi...@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Sounds good, Edison :)
> >>>>>>>
> >>>>>>> I can implement John's isManaged() logic and resubmit my code for
> >>>> review.
> >>>>>>>
> >>>>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
> >>>> requires
> >>>>>>> some refactoring of the storage framework, then I could do that.
> >>>>>>>
> >>>>>>> Also, the default storage plug-in does have hypervisor logic in it in
> >>>> 4.2.
> >>>>>>> I recommend we leave this as is for 4.2 and it can be refactored in
> 4.3
> >>>>>>> with the above work I mentioned.
> >>>>>>>
> >>>>>>> Are you OK with that, guys?
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
> >>>>>>>> *To:* dev@cloudstack.apache.org
> >>>>>>>> *Cc:* Edison Su; John Burwell
> >>>>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> Another thing we might want to consider for 4.2 is that the way
> the
> >>>> code
> >>>>>>>> is currently implemented in master, I don't think it's even possible
> >>>> to
> >>>>>>>> write a plug-in that connects storage directly to a VM because the
> >>>>>>>> CloudStack storage framework will call the "attach" method of the
> >>>>>>>> hypervisor in use and this method will fail because it assumes
> there
> >>>> is
> >>>>>>>> (talking Xen here) an SR existent and in this situation there isn't
> >>>> one.*
> >>>>>>>> ***
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> My thinking is that we should ignore the direct-attach-of-volume-
> to-
> >> VM
> >>>>>>>> use case for 4.2. I think it will require too much of a rewrite at
> >>>> this
> >>>>>>>> point in development (fewer than three weeks left until code
> >>>>>> freeze).****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> If the plug-in implements an isManaged() method (as John
> >> suggested),
> >>>>>> the
> >>>>>>>> storage framework can invoke the necessary commands on the
> >>>>>> hypervisor in
> >>>>>>>> use.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> For example, the createAsync method is called. The storage plug-
> in
> >>>>>>>> creates a SAN volume and updates the CS DB. The storage
> >> framework
> >>>>>> then asks
> >>>>>>>> the driver if it's managed. If it is, the storage framework executes
> a
> >>>>>>>> command against the hypervisor in use to have it create, say, an
> SR
> >>>> (with
> >>>>>> a
> >>>>>>>> single VDI that takes up the entire SR). Now, when the storage
> >>>>>> framework
> >>>>>>>> executes the "attach" command against the hypervisor, this
> >> command
> >>>> will
> >>>>>>>> work, as well (because that logic assumes the prior existence of
> an
> >>>> SR and
> >>>>>>>> it will, in fact, be present).****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:****
> >>>>>>>>
> >>>>>>>> Hi Edison and John,****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> I wanted to point out something I thought was relevant to this
> >>>>>>>> conversation.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> My plug-in supports both XenServer and ESX(i).****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> These two hypervisors handle their respective hypervisor data
> >>>> structures
> >>>>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> In XenServer, the SR that resides on the SAN LUN is simply
> >> "forgotten"
> >>>>>>>> when you issue a remove-SR command to XenServer.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> When you later want to access the contents of that data, it is still
> >>>>>>>> safely stored on the LUN and can be accessed from XenServer
> (from
> >> the
> >>>>>> same
> >>>>>>>> or different cluster (what XenServer calls a resource pool)) by
> >>>> creating a
> >>>>>>>> new SR that is based on the same IQN and LUN.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> XenServer SRs exist at the cluster level.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
> >>>>>>>> destroyed when you issue a remove-datastore command to
> ESX(i).
> >> In
> >>>>>> other
> >>>>>>>> words, the contents of the SAN LUN are destroyed.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> ESX(i) datastores exist at the datacenter level.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> The XenServer behavior (not destroying the contents of the LUN
> >> when
> >>>>>> the
> >>>>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
> >>>> contents
> >>>>>>>> of the LUN when the datastore is removed) is not desirable.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> I can get around the ESX(i) behavior by removing the referenced
> IQN
> >>>>>> from
> >>>>>>>> each host in the cluster when the CS storage framework notifies
> my
> >>>> plug-
> >>>>>> in
> >>>>>>>> of a detach-volume event. When the CS volume is later attached
> >> again
> >>>>>>>> (either to the same or a different ESX(i) cluster), I can have each
> >>>> host in
> >>>>>>>> the relevant ESX(i) cluster reference the IQN and the datastore
> will
> >>>> be
> >>>>>>>> back again.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> This sequence of events is critical for my plug-in to support both
> >>>>>>>> environments:****
> >>>>>>>>
> >>>>>>>> *XenServer flow (events for Driver class):*****
> >>>>>>>>
> >>>>>>>> createAsync (called right before first attach) = creates SAN
> >>>> volume****
> >>>>>>>>
> >>>>>>>> preAttachVolume = creates SR and creates VDI or just introduces
> >> SR****
> >>>>>>>>
> >>>>>>>> (CS volume gets attached and later detached)****
> >>>>>>>>
> >>>>>>>> postDetachVolume = removes SR (this does not delete data on
> SAN
> >>>>>> volume)**
> >>>>>>>> **
> >>>>>>>>
> >>>>>>>> deleteAsync (can be called when volume is not attached) =
> deletes
> >> the
> >>>>>> SAN
> >>>>>>>> volume****
> >>>>>>>>
> >>>>>>>> *VMware flow (events for Driver class):*****
> >>>>>>>>
> >>>>>>>> createAsync (called right before first attach) = creates SAN
> volume,
> >>>>>>>> creates datastore, and creates VMDK file****
> >>>>>>>>
> >>>>>>>> preAttachVolume = adds iSCSI target to each host in cluster****
> >>>>>>>>
> >>>>>>>> (CS volume gets attached and later detached)****
> >>>>>>>>
> >>>>>>>> postDetachVolume = removes iSCSI target from each host in
> >> cluster****
> >>>>>>>>
> >>>>>>>> deleteAsync (can be called when volume is not attached) =
> deletes
> >>>>>>>> datastore and deletes the SAN volume****
> >>>>>>>>
> >>>>>>>> ************************
> >>>>>>>>
> >>>>>>>> So, the way I have it written up here, the driver is talking to the
> >>>>>>>> hypervisor. As John has pointed out, this may not be ideal.****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> Whatever we decide on, it needs to be able to support this kind of
> >>>> flow.
> >>>>>>>> :)****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> We could go John's route, where the driver says if it's managed or
> >>>> not.**
> >>>>>>>> **
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> If it is, then the storage framework could send the necessary
> >>>> messages to
> >>>>>>>> the hypervisor in question at the right times (just migrate the
> >>>> hypervisor
> >>>>>>>> logic above from the driver to the right parts of the storage
> >>>> framework).
> >>>>>>>> ****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> This is fine, except that it does not resolve Edison's use case of
> >>>>>>>> desiring to have the storage LUN potentially connected directly to
> a
> >>>> VM.*
> >>>>>>>> ***
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> I'm not sure we should design around that use case, though. Is it
> >>>> common
> >>>>>>>> enough? Has anyone asked for it? Is it even desirable? How are
> >> usage
> >>>>>>>> statistics gathered for such volumes? It's nice to offer the
> >>>> flexibility,
> >>>>>>>> but is it a good idea in this case?****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> Thanks!****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> >>>> wrote:**
> >>>>>>>> **
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>> Sent: Friday, June 07, 2013 4:08 PM
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>
> >>>>>>>>> Hi John,
> >>>>>>>>>
> >>>>>>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
> >>>>>>>> volume
> >>>>>>>>> to use it. Same story for VMware: ESX(i) must layer a datastore
> on
> >>>> top
> >>>>>>>> of
> >>>>>>>>> the iSCSI volume to use it. I look at it like they are layering a
> >>>>>>>> clustered
> >>>>>>>>> file system on the SAN volume so the hypervisors can share
> access
> >> to
> >>>>>> the
> >>>>>>>>> contents of the volume.
> >>>>>>>>>
> >>>>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called,
> the
> >>>>>>>> SR or
> >>>>>>>>> datastore has already been created (usually manually by an
> admin).
> >>>>>>>>>
> >>>>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
> >>>>>>>>> environment where each CloudStack volume that a user creates
> is
> >>>>>> mapped
> >>>>>>>>> to a
> >>>>>>>>> single SAN volume (via the SR).
> >>>>>>>>>
> >>>>>>>>> The question comes down to who should allocate the SR
> >> dynamically.
> >>>>>>>>>
> >>>>>>>>> We could have the storage framework ask the storage plug-in if
> it is
> >>>>>>>>> managed. If it is, then the storage framework could send a
> >> message to
> >>>>>>>> the
> >>>>>>>>> hypervisor in question to create (let's talk Xen here) the SR
> ahead
> >>>> of
> >>>>>>>>> time. Then, when the storage framework next sends the attach-
> >> volume
> >>>>>>>>> command
> >>>>>>>>> to the hypervisor, it should work without changes to that
> >>>> attach-volume
> >>>>>>>>> logic (because - from the point of view of the attach logic - the SR
> >>>> is
> >>>>>>>>> already existent, as expected).
> >>>>>>>>>
> >>>>>>>>> Now, as Edison has pointed out, this limits the power of the
> storage
> >>>>>>>>> plug-in. A storage plug-in in this model cannot directly attach
> >>>> storage
> >>>>>>>> to
> >>>>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We
> >> need
> >>>>>> to
> >>>>>>>>> make a call on that.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Yes, I think we need to address this usage case, and won't limit
> the
> >>>>>>>> power of the storage plug-in.
> >>>>>>>> John, what's your idea to fit this usage case into your abstraction?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> If we want to give more power to the storage vendor, we would
> >> have to
> >>>>>>>>> have
> >>>>>>>>> the storage framework call into the storage plug-in to send the
> >>>>>>>> appropriate
> >>>>>>>>> attach commands to the hypervisor.
> >>>>>>>>>
> >>>>>>>>> Let's discuss and come to a consensus (or at least agree on a
> path)
> >>>> as
> >>>>>>>> soon
> >>>>>>>>> as we can. :)
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell
> <jb...@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> My understanding is the Xen expects an SR structure on all
> ISCSI
> >>>>>>>> devices
> >>>>>>>>>> -- at least that is how I read the code in your patch. Is my
> >>>>>>>> understanding
> >>>>>>>>>> correct? If so, the Xen plugin should be able to query the
> storage
> >>>>>>>> device
> >>>>>>>>>> to determine the presence of the SR structure and create it if it
> >>>>>>>> does not
> >>>>>>>>>> exist. Am I missing something in the implementation that
> makes
> >> that
> >>>>>>>> type
> >>>>>>>>>> of implementation impossible?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Yeah, if a storage vendor wanted to attach a volume directly
> to a
> >>>>>>>> VM in
> >>>>>>>>>> 4.2
> >>>>>>>>>>> today, it would probably fail because the attach-volume logic
> >>>>>>>> assumes
> >>>>>>>>> the
> >>>>>>>>>>> existence of the necessary hypervisor data structure (ex. SR
> on
> >>>>>>>> Xen).
> >>>>>>>>>>>
> >>>>>>>>>>> If we wanted to enable such an attach, we could do it the way
> >>>>>> Edison
> >>>>>>>>>>> suggests.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su
> <Ed...@citrix.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>> From: Mike Tutkowski
> [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
> >>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> As we only have three weeks until feature freeze, we
> should
> >>>>>> come
> >>>>>>>> to
> >>>>>>>>> a
> >>>>>>>>>>>>> consensus on this design point as soon as possible.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Right now, if the storage framework asks my driver it is is
> >>>>>>>> managed, it
> >>>>>>>>>>>>> will say 'yes.' This means the framework will tell the driver
> to
> >>>>>>>>>> perform
> >>>>>>>>>>>>> its management activities. This then means the driver will
> call
> >>>>>>>> into
> >>>>>>>>>> the
> >>>>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to
> >> perform
> >>>>>> the
> >>>>>>>>>>>> activity
> >>>>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The driver doesn't know which hypervisor it's talking to, it
> just
> >>>>>>>>>> sends a
> >>>>>>>>>>>>> message to the host to perform the necessary pre-attach
> >> work.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Could we just expose a method like
> >>>>>> "attachVolume/dettachVolume" on
> >>>>>>>>> the
> >>>>>>>>>>>> PrimaryDataStoreDriver driver?
> >>>>>>>>>>>> For most of cases, the implementation of each driver would
> >> just
> >>>>>>>> send
> >>>>>>>>>>>> attachvolumecommand/dettachvolumecmd to
> hypervisor(we
> >> can
> >>>>>> put
> >>>>>>>>> the
> >>>>>>>>>>>> implementation in a base class, so that can be shared by all
> of
> >>>>>>>> this
> >>>>>>>>>>>> drivers), and in each hypervisor resource code, which may
> just
> >>>> call
> >>>>>>>>>>>> hypervisor's api to attach the volume to VM, while for
> certain
> >>>>>>>> storage,
> >>>>>>>>>>>> like SolidFire, may need to create a SR first, and create
> volume
> >>>>>>>> on it,
> >>>>>>>>>>>> then call hypervisor's API to attach volume to VM.
> >>>>>>>>>>>> While for some other storage vendor, may want to bypass
> >>>>>> hypervisor
> >>>>>>>>>> during
> >>>>>>>>>>>> attaching volume, so inside driver's attachvolume
> >> implementation,
> >>>>>>>> the
> >>>>>>>>>>>> driver can do some magic, such as, directly talk to an agent
> >>>>>>>> inside VM
> >>>>>>>>>>>> instance, then create a disk inside VM.
> >>>>>>>>>>>>
> >>>>>>>>>>>> How do you guys think?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su
> >> <Ed...@citrix.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>>> From: Mike Tutkowski
> >> [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> How's about this:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The driver can implement an isManaged() method. The
> >>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
> >>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>> (which is responsible for calling into the hypervisor to
> >> attach
> >>>>>>>> the
> >>>>>>>>>>>> disk)
> >>>>>>>>>>>>>>> can call into the hypervisor to create the necessary
> >> hypervisor
> >>>>>>>> data
> >>>>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The problem here is that storage vendor may work
> >> differently
> >>>>>> with
> >>>>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN,
> while
> >>>>>>>> maybe
> >>>>>>>>>>>> other
> >>>>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the
> >> LUN
> >>>>>>>>> directly
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> VM instance, see the discuss(
> >>>>>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>>>>>>>>>>>> %3E
> >>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>> So I would let storage provider to implement attach disk
> to
> >> VM,
> >>>>>>>>>>>> instead of
> >>>>>>>>>>>>>> implemented by cloudstack itself.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If that's what you're going for, that works for me. By the
> >> way,
> >>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>> default storage plug-in (which handles the default
> storage
> >>>>>>>> behavior
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code
> that
> >>>>>>>> talks to
> >>>>>>>>>>>>>>> hypervisors. You might want to contact him and inform
> him
> >> of
> >>>>>>>> your
> >>>>>>>>>>>>>> concerns
> >>>>>>>>>>>>>>> or that logic (as is) will make it to production.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Please let me know if what I wrote in above (for my
> >> solution)
> >>>>>>>> is OK
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Please see my responses in-line below.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I
> build
> >>>>>>>> up
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>> file, but this is the gist of what I've done:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> * During a volume-attach operation, after
> >>>>>> VolumeManagerImpl
> >>>>>>>>> tells
> >>>>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a
> volume, I
> >>>>>> have
> >>>>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask
> the
> >>>>>> driver if
> >>>>>>>> it
> >>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver
> >> perform
> >>>>>>>>>>>> whatever
> >>>>>>>>>>>>>>>>> activity is required. In my case, this includes sending a
> >>>>>>>>>>>> message to
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> host where the VM is running to have, say XenServer,
> >> add a
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the
> IQN
> >> of
> >>>>>> the
> >>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the
> space
> >> it
> >>>>>>>> can
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> storage repository). After this, the normal attach-
> volume
> >>>>>>>>>>>> message is
> >>>>>>>>>>>>>> sent
> >>>>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> There should be **no** code from a storage driver to
> a
> >>>>>>>> hypervisor.
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>>> apologize for the repetition, but we simply can not
> have
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> specific code in the storage layer. The circular
> >> dependencies
> >>>>>>>>>>>> between
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> two layers are not sustainable in the long term. Either
> the
> >>>>>>>>>>>>>> VirtualManager
> >>>>>>>>>>>>>>>> or Xen hypervisor plugin needs to be
> refactored/modified
> >> to
> >>>>>>>>>>>>> coordinate
> >>>>>>>>>>>>>>>> volume creation and then populating the SR. Ideally,
> we
> >> can
> >>>>>>>>>>>>>> generalize the
> >>>>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
> >>>>>> hypervisor
> >>>>>>>>>>>> plugin
> >>>>>>>>>>>>>>>> would only implement callbacks to perform the attach
> >> action
> >>>>>> and
> >>>>>>>>>>>> create
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver
> should
> >>>>>>>> only be
> >>>>>>>>>>>>>>>> allocating space and providing information about
> contents
> >>>>>> (e.g.
> >>>>>>>>>>>> space
> >>>>>>>>>>>>>>>> available, space consumed, streams to a URI, file
> handle
> >> for a
> >>>>>>>> URI,
> >>>>>>>>>>>>>> etc)
> >>>>>>>>>>>>>>>> and capabilities.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> * The reverse is performed for a detach-volume
> >> command.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in
> my
> >>>>>>>> driver.
> >>>>>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>>>> default driver simply does a "return false;". We could
> >> add a
> >>>>>>>> new
> >>>>>>>>>>>>>>>> parameter
> >>>>>>>>>>>>>>>>> to the createStoragePool API command, if we want,
> to
> >>>>>> remove
> >>>>>>>>> the
> >>>>>>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>>>>>> return values in the drivers (although my driver will
> >>>> probably
> >>>>>>>>>>>> just
> >>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>> this parameter and always return true since it
> wouldn't
> >> make
> >>>>>>>>>>>> sense
> >>>>>>>>>>>>>> for it
> >>>>>>>>>>>>>>>>> to ever return false). We'd need another column in
> the
> >>>>>>>>>>>> storage_pool
> >>>>>>>>>>>>>>> table
> >>>>>>>>>>>>>>>>> to store this value.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yes, I think we should have a parameter added to the
> >>>>>>>>>>>>> createStoragePool
> >>>>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
> >>>>>>>> configured
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> management when their underlying drivers support it.
> To
> >>>>>>>> simplify
> >>>>>>>>>>>>>> things,
> >>>>>>>>>>>>>>>> this flag should only be mutable when the DataStore is
> >>>>>>>> created. It
> >>>>>>>>>>>>>> would be
> >>>>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to
> >> unmanaged
> >>>>>> after
> >>>>>>>>>>>>>> creation.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
> >>>>>> working
> >>>>>>>>>>>> GUI
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>> to review (in that document), if you'd like to check it
> >> out.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed
> >> together to
> >>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor
> >> throttled
> >>>>>> IOPs
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the
> potential
> >> for
> >>>>>>>>>>>>>> confusion,
> >>>>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these
> features
> >> will
> >>>>>>>>>>>> help
> >>>>>>>>>>>>>>>>>>> minimize any potential confusion.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will
> work
> >>>>>>>> itself
> >>>>>>>>>>>> if
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
> >>>> hypervisor
> >>>>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google
> >> doc I
> >>>>>> sent
> >>>>>>>>> to
> >>>>>>>>>>>>> you
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> other day to see how the GUI looks for
> provisioned
> >>>>>> storage
> >>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-
> mail
> >>>> list
> >>>>>>>>>>>> and we
> >>>>>>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add
> >> Disk
> >>>>>>>>> Offering
> >>>>>>>>>>>>>> dialog.
> >>>>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS,
> so
> >> they
> >>>>>>>>> should
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say,
> >> only
> >>>>>>>> use
> >>>>>>>>>>>> Max
> >>>>>>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for
> >> storage
> >>>>>>>> that
> >>>>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size
> field,
> >>>>>>>> the
> >>>>>>>>>>>>>> admin can
> >>>>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max,
> and
> >>>>>> Burst
> >>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's
> >> feature
> >>>>>> or
> >>>>>>>>>>>> mine,
> >>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> decided by the admin).
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two
> >> features
> >>>>>>>>>>>> expressing
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express
> the
> >>>>>> IOPS
> >>>>>>>> for
> >>>>>>>>>>>>>> QoS in
> >>>>>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
> >>>>>>>> read/write
> >>>>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
> >>>>>>>>>>>>> Storage/Hypervisor
> >>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the
> >> behavior
> >>>>>> of
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see
> >> the
> >>>>>>>>>>>> following
> >>>>>>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>>>>>> outstanding:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as
> >> discrete
> >>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are
> expressed
> >> as
> >>>>>> total
> >>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS
> >> attached
> >>>>>> to
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for
> throttled
> >> and
> >>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>> that will permit providers to discriminate these
> >>>>>>>> guaranteed
> >>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
> >>>>>>>> provisioned
> >>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would
> >> neatly
> >>>>>>>> solve
> >>>>>>>>>>>>>> items 1
> >>>>>>>>>>>>>>>>>>> and 2.
> >>>>>>>>>>>>>>>>>>>>> Since the two facilities would not be permitted
> to
> >>>>>> operate
> >>>>>>>>>>>>>> together,
> >>>>>>>>>>>>>>>>>>> they
> >>>>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I
> think
> >> item
> >>>>>> 3
> >>>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage
> records.
> >> As
> >>>>>>>> for
> >>>>>>>>>>>>>> Item 4,
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>>>>>> not familiar with how these two enhancements
> are
> >>>>>>>>> depicted in
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
> >>>>>> enhancements
> >>>>>>>>> for
> >>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >>>>>>>>> <us...@gmail.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I was busy working on other issues
> (CLOUDSTACK-
> >>>>>>>>> 2780/2729,
> >>>>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-
> >> 2823 ,
> >>>>>>>>>>>>> CLOUDSTACK-
> >>>>>>>>>>>>>>> 2875 ) this
> >>>>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes
> >> tomorrow,
> >>>>>> and
> >>>>>>>>>>>> ask
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> second
> >>>>>>>>>>>>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> >>>>>> <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going
> with
> >> this,
> >>>>>>>> John,
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager
> >> determining
> >>>>>>>>> whether or
> >>>>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
> >> StoragePoolType,
> >>>>>> but an
> >>>>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
> >> allocate
> >>>>>>>>>>>> resources
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an
> >> isManaged()
> >>>>>>>>> method
> >>>>>>>>>>>>>>> added to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it
> returns
> >> true,
> >>>>>>>>>>>> then the
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or
> >> whatever
> >>>>>> name)
> >>>>>>>>>>>>> method
> >>>>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface,
> >> as
> >>>>>> well)
> >>>>>>>>> and
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to
> >> create,
> >>>>>> say
> >>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() :
> boolean
> >> on
> >>>>>>>>>>>>>>> DataStoreDriver.
> >>>>>>>>>>>>>>>>>>> Since
> >>>>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to
> primary
> >>>>>> storage,
> >>>>>>>>> I
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage
> (Storage
> >> is a
> >>>>>>>>> straw
> >>>>>>>>>>>>> man
> >>>>>>>>>>>>>>> term
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is
> a
> >>>>>> mutable
> >>>>>>>>>>>>> property
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the
> underlying
> >>>>>>>>>>>>> DataStoreDriver
> >>>>>>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
> >>>>>>>>> manageability
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> devices.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR,
> >> the
> >>>>>> Xen
> >>>>>>>>>>>> plugin
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> responsible for composing
> DataStore/Volume
> >>>>>> methods
> >>>>>>>>> to
> >>>>>>>>>>>>>>> create any
> >>>>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR.
> There
> >>>>>>>> should
> >>>>>>>>>>>> be no
> >>>>>>>>>>>>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As
> I
> >> said
> >>>>>>>>>>>> earlier,
> >>>>>>>>>>>>>> such
> >>>>>>>>>>>>>>>>>>>>>>> circular
> >>>>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
> >>>>>> unmaintainable
> >>>>>>>>> mess.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page
> >> with
> >>>>>> you.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a
> >> Storage
> >>>>>> layer
> >>>>>>>>> that
> >>>>>>>>>>>>>>>> supports n
> >>>>>>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors
> of
> >> m
> >>>>>>>>>>>> hypervisors.
> >>>>>>>>>>>>>>>> Such
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
> >>>>>> untestable
> >>>>>>>>>>>>> beast.
> >>>>>>>>>>>>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven
> to
> >>>>>> evolve
> >>>>>>>>> the
> >>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager
> >> determining
> >>>>>>>>> whether
> >>>>>>>>>>>>> or
> >>>>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
> >> StoragePoolType,
> >>>>>> but
> >>>>>>>>> an
> >>>>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
> >> allocate
> >>>>>>>>>>>> resources
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device
> >> driver
> >>>>>>>> needs
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> indicate
> >>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
> >>>>>> Finally, I
> >>>>>>>>>>>> think
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to
> elect
> >> to
> >>>>>>>> have
> >>>>>>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver
> is
> >>>>>> capable
> >>>>>>>>>>>>> managing
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
> >>>>>>>>> unmanaged).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume
> should
> >> be
> >>>>>> done
> >>>>>>>>> in
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed
> by
> >> the
> >>>>>>>>> Storage
> >>>>>>>>>>>>>>> layer to
> >>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> >>>>>> create a
> >>>>>>>>>>>> file,
> >>>>>>>>>>>>>> etc).
> >>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the
> >> Xen
> >>>>>>>>>>>>>> hypervisor, as
> >>>>>>>>>>>>>>>>>>> such,
> >>>>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation.
> >> From
> >>>>>> my
> >>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>> nothing
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
> >>>>>> content.
> >>>>>>>>> From
> >>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides
> >> the
> >>>>>> means
> >>>>>>>>> to
> >>>>>>>>>>>>>> query
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the content
> by
> >>>>>> higher-
> >>>>>>>>> level
> >>>>>>>>>>>>>>> layers
> >>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be
> a
> >>>>>>>>> composition
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on
> >> the
> >>>>>>>>> Volume
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> protocol
> >>>>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike
> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> >>>>>>>>> implemented
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>> patch,
> >>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code
> alone.
> >> In
> >>>>>>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and
> send it
> >> to
> >>>>>> the
> >>>>>>>>>>>>>>> hypervisor.
> >>>>>>>>>>>>>>>> Before
> >>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if
> >> we're
> >>>>>>>>> dealing
> >>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>> (or
> >>>>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if
> so
> >> -
> >>>>>>>> send a
> >>>>>>>>>>>>>> "Create
> >>>>>>>>>>>>>>>>>>> SR"
> >>>>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns
> OK,
> >> we
> >>>>>>>>> would
> >>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>> proceed
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same
> >> and
> >>>>>> we just
> >>>>>>>>>>>> add
> >>>>>>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called
> for
> >> this
> >>>>>>>>>>>> particular
> >>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer
> today
> >> that
> >>>>>>>>> outlines
> >>>>>>>>>>>>> how
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will
> >> probably
> >>>>>> be
> >>>>>>>> of
> >>>>>>>>>>>> use
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> you, as
> >>>>>>>>>>>>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you
> (you
> >>>>>> should
> >>>>>>>>> have
> >>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John
> >> Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I
> some
> >>>>>>>>> thoughts,
> >>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against
> the
> >>>>>>>> solidfire,
> >>>>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to
> collect
> >> them
> >>>>>> in a
> >>>>>>>>>>>>>>> coherent
> >>>>>>>>>>>>>>>>>>> form
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> >>>>>> introduces
> >>>>>>>>> the
> >>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> >>>>>> introduces
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages
> >> the
> >>>>>> plug-in
> >>>>>>>>> was
> >>>>>>>>>>>>>>>>>>> incomplete,
> >>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a
> managed
> >> iSCSI
> >>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> >>>>>> spending on
> >>>>>>>>>>>> this.
> >>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>> Do
> >>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
> >>>>>> accomplish
> >>>>>>>>>>>>> what
> >>>>>>>>>>>>>>> you're
> >>>>>>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John
> >> Burwell
> >>>>>> <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> >>>>>> introduces
> >>>>>>>>> the
> >>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> >>>>>> management
> >>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>>> (i.e.
> >>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an
> operator
> >> want
> >>>>>> it
> >>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should
> >> end up
> >>>>>> with
> >>>>>>>>> a
> >>>>>>>>>>>>>>> semantic
> >>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to
> be
> >>>>>> managed.
> >>>>>>>>> I
> >>>>>>>>>>>>> also
> >>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage
> >> types
> >>>>>> from
> >>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike
> >> Tutkowski
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in
> red.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM,
> John
> >>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the
> >> following
> >>>>>>>>> questions
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> me
> >>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
> iSCSI
> >>>>>> LUNs
> >>>>>>>>> and
> >>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a
> >> single
> >>>>>> LUN
> >>>>>>>>> (LUN
> >>>>>>>>>>>>> 0).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between
> SAN
> >>>>>> Volumes
> >>>>>>>>> and
> >>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack
> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> >>>>>> management
> >>>>>>>>>>>>> server or
> >>>>>>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-
> in,
> >> a
> >>>>>> SAN
> >>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>> (same
> >>>>>>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server
> >> (via
> >>>>>> the
> >>>>>>>>> plug-in)
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> >>>>>> hypervisor.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the
> SolidFire
> >> plug-
> >>>>>> in,
> >>>>>>>>> but
> >>>>>>>>>>>>>> still
> >>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do
> >> this
> >>>>>> already
> >>>>>>>>>>>>> today
> >>>>>>>>>>>>>>>> (prior
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN
> >> volume
> >>>>>> and -
> >>>>>>>>> in
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>> case -
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN
> volume.
> >>>>>> While
> >>>>>>>>> you
> >>>>>>>>>>>>> can do
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce
> >> storage
> >>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> >>>>>> management
> >>>>>>>>> server
> >>>>>>>>>>>>> or
> >>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used,
> the
> >>>>>> SAN
> >>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the
> >> plug-
> >>>>>> in).
> >>>>>>>>> There
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping
> between
> >> a
> >>>>>> SAN
> >>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>> and a
> >>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage
> >> vendor
> >>>>>> that
> >>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>> true
> >>>>>>>>>>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these
> >> pieces
> >>>>>> are
> >>>>>>>>>>>> related
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure
> >> that
> >>>>>>>>> multiple
> >>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the
> underlying
> >>>>>> storage."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we
> need
> >> to
> >>>>>> map a
> >>>>>>>>>>>>> single
> >>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We
> >> cannot
> >>>>>> have
> >>>>>>>>>>>>> multiple
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and
> still
> >>>>>>>>> guarantee
> >>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single
> >> SAN
> >>>>>>>>> volume
> >>>>>>>>>>>>> house
> >>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do
> >> that
> >>>>>> today
> >>>>>>>>>>>>> without
> >>>>>>>>>>>>>>> any of
> >>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM,
> >> Mike
> >>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate
> a
> >> SAN
> >>>>>>>>> volume
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will
> be
> >>>>>> created."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on
> this.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM,
> >> John
> >>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part
> which
> >>>>>>>>> concerns me
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into
> >> storage
> >>>>>> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM,
> Mike
> >>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the
> iSCSI
> >> type
> >>>>>> is
> >>>>>>>>> today
> >>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer
> (when
> >> you
> >>>>>> have
> >>>>>>>>> not
> >>>>>>>>>>>>>>> PreSetup an
> >>>>>>>>>>>>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >>>>>> volume
> >>>>>>>>> from
> >>>>>>>>>>>>>>> VMware, it
> >>>>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >>>>>> volume
> >>>>>>>>> from
> >>>>>>>>>>>>> KVM,
> >>>>>>>>>>>>>>> it uses
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and
> Edison's
> >>>>>> thinking
> >>>>>>>>>>>>> here
> >>>>>>>>>>>>>>> was to
> >>>>>>>>>>>>>>>>>>> make a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this
> >> dynamic
> >>>>>> ability
> >>>>>>>>>>>>> Edison
> >>>>>>>>>>>>>>> added
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should
> be
> >>>>>> more
> >>>>>>>>>>>>> specificy,
> >>>>>>>>>>>>>>> though:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27
> PM,
> >>>>>> Mike
> >>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> >>>>>> shouldn't
> >>>>>>>>> know
> >>>>>>>>>>>>> or
> >>>>>>>>>>>>>>> care
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be
> able
> >> to
> >>>>>> answer
> >>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here
> is
> >> that
> >>>>>> the
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so
> >> that
> >>>>>> the
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to
> removing
> >> that
> >>>>>>>>> constraint
> >>>>>>>>>>>>>>> and just
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just
> >> need a
> >>>>>> way
> >>>>>>>>> for
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new
> requirement
> >> and
> >>>>>>>>> perform
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24
> PM,
> >>>>>> John
> >>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM,
> >> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack
> >> may
> >>>>>> be
> >>>>>>>>> able to
> >>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it
> >> unmanaged by
> >>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> (e.g. the
> >>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> >>>>>> operator has
> >>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> >>>>>> reasoning
> >>>>>>>>>>>>> make
> >>>>>>>>>>>>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> >>>>>> creating a
> >>>>>>>>> SAN
> >>>>>>>>>>>>>>> volume via
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> >>>>>> hypervisor)
> >>>>>>>>> would
> >>>>>>>>>>>>>>> work unless
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the
> form
> >> of,
> >>>>>> say,
> >>>>>>>>> an
> >>>>>>>>>>>>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that,
> >> independent
> >>>>>> of
> >>>>>>>>>>>>>>> CloudStack, an
> >>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to
> >> CloudStack,
> >>>>>> and
> >>>>>>>>>>>>> exposes it
> >>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of
> that
> >> LUN
> >>>>>> over
> >>>>>>>>> to
> >>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate
> >> anymore
> >>>>>> LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands
> >> prior
> >>>>>> to my
> >>>>>>>>>>>>>>> changes, we
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does
> not
> >> have
> >>>>>> the
> >>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we
> should
> >>>>>> maybe
> >>>>>>>>> have
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume
> >> needs
> >>>>>>>>> support
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> >>>>>> specifically
> >>>>>>>>> to
> >>>>>>>>>>>>> create
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent
> code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> >>>>>> management
> >>>>>>>>> plugin
> >>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> >>>>>> determine
> >>>>>>>>>>>>> available
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow
> >> the
> >>>>>>>>>>>>> interrogation of
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> >>>>>> device
> >>>>>>>>> itself
> >>>>>>>>>>>>>>> shouldn't
> >>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -
> -
> >>>>>> simply be
> >>>>>>>>> able
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> answer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> 1:01
> >> PM,
> >>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as
> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers
> the
> >>>>>> SolidFire
> >>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once
> >> this is
> >>>>>>>>> done, a
> >>>>>>>>>>>>> new
> >>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a
> Disk
> >>>>>> Offering
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> references
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> >>>>>> Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates
> a
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> volume. This
> >>>>>>>>>>>>>>>>>>> leads
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches
> >> the
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> volume to a
> >>>>>>>>>>>>>>>>>>> VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> >>>>>> framework
> >>>>>>>>> calling
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system
> (in
> >> my
> >>>>>> case, a
> >>>>>>>>>>>>> SAN).
> >>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> >>>>>> applicable
> >>>>>>>>> data
> >>>>>>>>>>>>>>> (like the
> >>>>>>>>>>>>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> >>>>>> invoked if
> >>>>>>>>> the
> >>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After
> the
> >>>>>> attach,
> >>>>>>>>> the
> >>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a
> detach
> >> disk)
> >>>>>> and
> >>>>>>>>> the
> >>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> >>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach
> >> logic
> >>>>>> is run
> >>>>>>>>> and
> >>>>>>>>>>>>>>> detects
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs
> "assistance"
> >> in
> >>>>>> the
> >>>>>>>>> form
> >>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> >> 12:54
> >>>>>> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we
> are in
> >>>>>> sync on
> >>>>>>>>>>>>>>> terminology,
> >>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to
> the
> >>>>>> physical
> >>>>>>>>>>>>>>> allocation on
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say
> 'volume',
> >> I
> >>>>>> try to
> >>>>>>>>>>>>> mean
> >>>>>>>>>>>>>>> 'SAN
> >>>>>>>>>>>>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the
> 'volume'
> >> the
> >>>>>> end
> >>>>>>>>> user
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>>> make in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack
> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> >> 12:50
> >>>>>> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here
> may
> >>>>>> very
> >>>>>>>>> well
> >>>>>>>>>>>>> make
> >>>>>>>>>>>>>>> sense, but
> >>>>>>>>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should
> >> draw
> >>>>>> Edison
> >>>>>>>>> in on
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to
> suggest
> >> the
> >>>>>>>>>>>>> approach I
> >>>>>>>>>>>>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013
> at
> >> 12:42
> >>>>>> PM,
> >>>>>>>>>>>>> John
> >>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> >>>>>> combining
> >>>>>>>>> two
> >>>>>>>>>>>>>>> distinct
> >>>>>>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and
> >> storage
> >>>>>>>>> protocols.
> >>>>>>>>>>>>> In
> >>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one
> allows
> >> the
> >>>>>>>>> system to
> >>>>>>>>>>>>>>>> create/delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the
> other
> >>>>>> requires
> >>>>>>>>> the
> >>>>>>>>>>>>>>> volume to
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the
> >> CloudStack
> >>>>>>>>> context.
> >>>>>>>>>>>>> To
> >>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology,
> volume,
> >> in
> >>>>>> these
> >>>>>>>>>>>>>>> definitions,
> >>>>>>>>>>>>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the
> >> device,
> >>>>>>>>> correct?
> >>>>>>>>>>>>>>> Minimally,
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> >>>>>> storage
> >>>>>>>>> device
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> move bits
> >>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete
> bits,
> >> etc.
> >>>>>>>>>>>>> Optionally, a
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
> >> CloudStack.
> >>>>>>>>> Therefore,
> >>>>>>>>>>>>>>> we can have
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we
> store a
> >> Xen
> >>>>>> SR,
> >>>>>>>>> and
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>>> can have a
> >>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> >>>>>> CloudStack is
> >>>>>>>>>>>>> capable
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally,
> >> while
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>> may
> >>>>>>>>>>>>>>> be able
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose
> to
> >>>>>> leave it
> >>>>>>>>>>>>>>> unmanaged by
> >>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many
> >> services,
> >>>>>> and
> >>>>>>>>> the
> >>>>>>>>>>>>>>> operator has
> >>>>>>>>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> >>>>>> CloudStack).
> >>>>>>>>> Does
> >>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my
> thoughts
> >>>>>> above
> >>>>>>>>> are
> >>>>>>>>>>>>>>> reasonable, it
> >>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the
> management
> >>>>>>>>> concerns
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with
> an
> >>>>>>>>> attached
> >>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
> >> CloudStack,
> >>>>>> and
> >>>>>>>>>>>>> establish
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a
> device
> >>>>>> separate
> >>>>>>>>> that
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> associated
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in
> place,
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> drivers can
> >>>>>>>>>>>>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when
> >> they
> >>>>>>>>> encounter
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
> >> operations
> >>>>>>>>> exposed
> >>>>>>>>>>>>> by
> >>>>>>>>>>>>>>> the driver to
> >>>>>>>>>>>>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these
> >> thoughts/concepts
> >>>>>> make
> >>>>>>>>>>>>> sense,
> >>>>>>>>>>>>>>> then we can
> >>>>>>>>>>>>>>>>>>> sit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed
> design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> 5:25
> >> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the
> >> difference
> >>>>>>>>> between
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> current
> >>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The
> admin
> >>>>>> has to
> >>>>>>>>> go in
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At
> this
> >>>>>> point in
> >>>>>>>>> time,
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it
> is
> >> pre-
> >>>>>>>>> allocated).
> >>>>>>>>>>>>>>> Future
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR
> >> that
> >>>>>> was
> >>>>>>>>> created
> >>>>>>>>>>>>>>> behind the
> >>>>>>>>>>>>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The
> >> admin
> >>>>>> has
> >>>>>>>>> to
> >>>>>>>>>>>>> go in
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will
> >> create
> >>>>>> and
> >>>>>>>>> delete
> >>>>>>>>>>>>>>> volumes on
> >>>>>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> >>>>>> enabled via
> >>>>>>>>> the
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a
> CloudStack
> >>>>>> volume
> >>>>>>>>> that
> >>>>>>>>>>>>> was
> >>>>>>>>>>>>>>> created,
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a
> >> new
> >>>>>>>>> volume.
> >>>>>>>>>>>>> After
> >>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>>>> done,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> >>>>>> question is
> >>>>>>>>> called.
> >>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point
> because
> >> the
> >>>>>>>>> volume
> >>>>>>>>>>>>> was
> >>>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> >>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> >> at
> >>>>>> 3:21
> >>>>>>>>> PM,
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new
> >> terms,
> >>>>>> so I
> >>>>>>>>> should
> >>>>>>>>>>>>>>> probably
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage:
> >> Storage
> >>>>>> that is
> >>>>>>>>>>>>> pre-
> >>>>>>>>>>>>>>> allocated
> >>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a
> SAN),
> >> then
> >>>>>> a
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> >>>>>> XenServer
> >>>>>>>>> SR),
> >>>>>>>>>>>>> then
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> >>>>>> CloudStack.
> >>>>>>>>> Disks
> >>>>>>>>>>>>> (VDI)
> >>>>>>>>>>>>>>> are later
> >>>>>>>>>>>>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> >>>>>> needed. In
> >>>>>>>>>>>>> these
> >>>>>>>>>>>>>>> cases, the
> >>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >>>>>> structure is
> >>>>>>>>>>>>> already
> >>>>>>>>>>>>>>> in place
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> hypervisor
> >> data
> >>>>>>>>> structure
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> the VM in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> >>>>>> Storage
> >>>>>>>>> that is
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent
> >> storage,
> >>>>>> this
> >>>>>>>>> could
> >>>>>>>>>>>>> be a
> >>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>> (not
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> >>>>>> hypervisor
> >>>>>>>>> data
> >>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>> must be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is
> >> performed
> >>>>>>>>> because
> >>>>>>>>>>>>> these
> >>>>>>>>>>>>>>> types of
> >>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a
> >> hypervisor
> >>>>>> data
> >>>>>>>>>>>>>>> structure by an
> >>>>>>>>>>>>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an
> SR
> >> on
> >>>>>>>>> XenServer
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within
> the
> >> SR
> >>>>>> to the
> >>>>>>>>> VM
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> >> at
> >>>>>> 3:13
> >>>>>>>>> PM,
> >>>>>>>>>>>>>>> John Burwell
> >>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >>>>>>>>> implementation of
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen
> ISCSI
> >>>>>> which
> >>>>>>>>> why I
> >>>>>>>>>>>>> ask
> >>>>>>>>>>>>>>> about the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the
> question --
> >>>>>> what is
> >>>>>>>>> the
> >>>>>>>>>>>>>>> definition
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> at
> >> 5:10
> >>>>>> PM,
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I
> know,
> >> the
> >>>>>> iSCSI
> >>>>>>>>> type
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> uniquely
> >>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> >>>>>> Primary
> >>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>> directly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you
> to
> >>>>>> skip the
> >>>>>>>>>>>>> step of
> >>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> repository
> >>>>>> based
> >>>>>>>>> on
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think
> >> this is
> >>>>>> only
> >>>>>>>>>>>>>>> supported for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors,
> you
> >>>>>> must
> >>>>>>>>> first go
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really
> >> know
> >>>>>> what
> >>>>>>>>> RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>>>> at
> >>>>>>>>> 2:13
> >>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading
> >> through
> >>>>>> the
> >>>>>>>>> code,
> >>>>>>>>>>>>> what
> >>>>>>>>>>>>>>> is the
> >>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> types?
> >>>>>> Why
> >>>>>>>>> isn't
> >>>>>>>>>>>>> RBD
> >>>>>>>>>>>>>>> considered
> >>>>>>>>>>>>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> 2013,
> >> at
> >>>>>> 3:46
> >>>>>>>>> PM,
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new
> type
> >> of
> >>>>>>>>> storage is
> >>>>>>>>>>>>>>> defined in
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called
> >> Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static
> >> enum
> >>>>>>>>>>>>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Filesystem(false),
> >>>>>> //
> >>>>>>>>> local
> >>>>>>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> NetworkFilesystem(true),
> >>>>>>>>>>>>> //
> >>>>>>>>>>>>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IscsiLUN(true),
> >> //
> >>>>>>>>> shared
> >>>>>>>>>>>>> LUN,
> >>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true),
> //
> >> for
> >>>>>> e.g.,
> >>>>>>>>> ZFS
> >>>>>>>>>>>>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false),
> //
> >> for
> >>>>>> iso
> >>>>>>>>> image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false),
> //
> >>>>>>>>> XenServer
> >>>>>>>>>>>>> local
> >>>>>>>>>>>>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true),
> //
> >>>>>>>>> VMware
> >>>>>>>>>>>>> VMFS
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> PreSetup(true), //
> >>>>>> for
> >>>>>>>>>>>>>>> XenServer, Storage
> >>>>>>>>>>>>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false),
> //
> >>>>>>>>> XenServer
> >>>>>>>>>>>>> local
> >>>>>>>>>>>>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Dynamic(true);
> >> //
> >>>>>>>>> dynamic,
> >>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean
> >> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> StoragePoolType(boolean
> >>>>>>>>>>>>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared
> =
> >>>>>> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public
> boolean
> >>>>>>>>> isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return
> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >>>>>> 2013 at
> >>>>>>>>> 1:41
> >>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> example,
> >>>>>> let's say
> >>>>>>>>>>>>> another
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> >>>>>> leverage its
> >>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> Service
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>> storage, as
> >>>>>>>>> well.
> >>>>>>>>>>>>>>> They would
> >>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as
> I've
> >>>>>> made
> >>>>>>>>> the
> >>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support
> >> their
> >>>>>> plug-
> >>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun
> >> 3,
> >>>>>> 2013
> >>>>>>>>> at
> >>>>>>>>>>>>> 1:39
> >>>>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry
> to
> >>>>>> imply
> >>>>>>>>> the
> >>>>>>>>>>>>>>> XenServer code
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> XenServer
> >>>>>>>>> attach
> >>>>>>>>>>>>> logic is
> >>>>>>>>>>>>>>> now aware
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and
> >> SolidFire is
> >>>>>> an
> >>>>>>>>>>>>>>> implementation of
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is
> >> new
> >>>>>> to 4.2
> >>>>>>>>>>>>> with
> >>>>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison
> >> created
> >>>>>> a
> >>>>>>>>> new
> >>>>>>>>>>>>>>> framework that
> >>>>>>>>>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> volumes
> >>>>>>>>> dynamically.
> >>>>>>>>>>>>>>> However, when I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April,
> we
> >>>>>> realized
> >>>>>>>>> that
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>> was not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> >>>>>> CloudStack
> >>>>>>>>> could
> >>>>>>>>>>>>> do
> >>>>>>>>>>>>>>> with these
> >>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed
> to
> >>>>>>>>> recognize
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>> new type of
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> hypervisor
> >> data
> >>>>>>>>> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun 3,
> >>>>>> 2013
> >>>>>>>>> at
> >>>>>>>>>>>>> 1:28
> >>>>>>>>>>>>>>> PM, John
> >>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >> generally
> >>>>>> odd
> >>>>>>>>> to me
> >>>>>>>>>>>>>>> that any
> >>>>>>>>>>>>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> understand or
> >>>>>> care
> >>>>>>>>>>>>> about
> >>>>>>>>>>>>>>> details. I
> >>>>>>>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose
> a
> >> set
> >>>>>> of
> >>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>> that can be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> implementations to
> >>>>>>>>>>>>>>> allocate
> >>>>>>>>>>>>>>>> space/create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >> invert
> >>>>>> this
> >>>>>>>>>>>>>>> dependency, we are
> >>>>>>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> problem
> >> that
> >>>>>> will
> >>>>>>>>> make
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> enhance.
> >> Am
> >>>>>> I
> >>>>>>>>>>>>>>> understanding that the
> >>>>>>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located
> in
> >> the
> >>>>>>>>>>>>>>> CitrixResourceBase
> >>>>>>>>>>>>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun 3,
> >>>>>>>>> 2013 at
> >>>>>>>>>>>>> 3:12
> >>>>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> delve
> >> into
> >>>>>> this
> >>>>>>>>> in a
> >>>>>>>>>>>>> bit
> >>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior
> to
> >> 4.2
> >>>>>> and
> >>>>>>>>> aside
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>> one setup
> >>>>>>>>>>>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first
> >> create
> >>>>>> a
> >>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set
> up
> >> a
> >>>>>> data
> >>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>> to make
> >>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> repository on
> >>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>> or a
> >>>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> >> this
> >>>>>>>>> storage
> >>>>>>>>>>>>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >> Edison's
> >>>>>> new
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>> framework,
> >>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> you
> >> can
> >>>>>> easily
> >>>>>>>>>>>>> create
> >>>>>>>>>>>>>>> a 1:1
> >>>>>>>>>>>>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> volume
> >> and
> >>>>>> the
> >>>>>>>>> VM's
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>> (necessary
> >>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
> can
> >> now
> >>>>>> write
> >>>>>>>>> a
> >>>>>>>>>>>>> plug-
> >>>>>>>>>>>>>>> in that is
> >>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> volumes
> >> as
> >>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> problem
> >>>>>> that
> >>>>>>>>> the
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> deleting
> >> the
> >>>>>>>>>>>>> hypervisor-
> >>>>>>>>>>>>>>> specific data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> >> being
> >>>>>> the
> >>>>>>>>> case,
> >>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>> enhancing
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> worked
> >> out
> >>>>>> and
> >>>>>>>>>>>>>>> submitted. I've got
> >>>>>>>>>>>>>>>>>>> ESX(i)
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we
> >> extend
> >>>>>> the
> >>>>>>>>> 4.2
> >>>>>>>>>>>>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does
> that
> >>>>>> help a
> >>>>>>>>> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Mon,
> >> Jun
> >>>>>> 3,
> >>>>>>>>> 2013 at
> >>>>>>>>>>>>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> storage
> >>>>>>>>> plug-in -
> >>>>>>>>>>>>> by
> >>>>>>>>>>>>>>> itself - is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> issue is
> >>>>>> with
> >>>>>>>>> the
> >>>>>>>>>>>>>>> volume-attach
> >>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> framework
> >>>>>> calls
> >>>>>>>>> into
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> plug-in to
> >>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> >> the
> >>>>>> time
> >>>>>>>>>>>>> comes to
> >>>>>>>>>>>>>>> attach the
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has
> to
> >> be
> >>>>>> smart
> >>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>> to recognize
> >>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (where
> >> the
> >>>>>>>>> volume
> >>>>>>>>>>>>> has
> >>>>>>>>>>>>>>> just been
> >>>>>>>>>>>>>>>>>>> created)
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> repository
> >>>>>> (for
> >>>>>>>>>>>>>>> XenServer) or a
> >>>>>>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> volume
> >> that
> >>>>>> was
> >>>>>>>>> just
> >>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >> been
> >>>>>>>>> spending
> >>>>>>>>>>>>> most
> >>>>>>>>>>>>>>> of my time
> >>>>>>>>>>>>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >> agent
> >>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does
> >> that
> >>>>>> clear it
> >>>>>>>>> up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Mon,
> >>>>>> Jun 3,
> >>>>>>>>> 2013
> >>>>>>>>>>>>> at
> >>>>>>>>>>>>>>> 12:48 PM,
> >>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >> you
> >>>>>> explain
> >>>>>>>>>>>>> why
> >>>>>>>>>>>>>>> the the storage
> >>>>>>>>>>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Jun
> >> 3,
> >>>>>> 2013,
> >>>>>>>>> at
> >>>>>>>>>>>>> 1:21
> >>>>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Yes,
> >>>>>>>>> ultimately I
> >>>>>>>>>>>>>>> would like to
> >>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> supports.
> >>>>>> I
> >>>>>>>>> think
> >>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>> just out of
> >>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Right
> >>>>>> now this
> >>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> regards
> >>>>>> to 4.2
> >>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>> freeze, I
> >>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Also,
> >> just
> >>>>>> to
> >>>>>>>>> be
> >>>>>>>>>>>>> clear,
> >>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan
> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> support
> >>>>>>>>> Compute
> >>>>>>>>>>>>>>> Offerings post
> >>>>>>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >> Mon,
> >>>>>> Jun 3,
> >>>>>>>>>>>>> 2013
> >>>>>>>>>>>>>>> at 11:14 AM,
> >>>>>>>>>>>>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is
> >> there
> >>>>>> any
> >>>>>>>>> plan
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> supporting
> >>>>>>>>>>>>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> ---
> >>>>>> Original
> >>>>>>>>>>>>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> From:
> >>>>>> "Mike
> >>>>>>>>>>>>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> To:
> >>>>>>>>>>>>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Sent:
> >>>>>>>>> Monday,
> >>>>>>>>>>>>> June
> >>>>>>>>>>>>>>> 3, 2013
> >>>>>>>>>>>>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Subject:
> >>>>>> Re:
> >>>>>>>>>>>>> [MERGE]
> >>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> agree
> >>>>>> on
> >>>>>>>>>>>>> merging
> >>>>>>>>>>>>>>> Wei's feature
> >>>>>>>>>>>>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> his
> >>>>>> feature
> >>>>>>>>> is
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> KVM only,
> >>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> KVM in
> >>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> Mon,
> >>>>>> Jun
> >>>>>>>>> 3,
> >>>>>>>>>>>>> 2013
> >>>>>>>>>>>>>>> at 8:55 AM,
> >>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> For
> >>>>>> the
> >>>>>>>>> billing,
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>>> no one works
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>>> billing
> >>>>>>>>> by
> >>>>>>>>>>>>>>> themselves. They
> >>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> disk_offering of
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>> VMs and
> >>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> to
> >> tell
> >>>>>> user
> >>>>>>>>> the
> >>>>>>>>>>>>>>> exact
> >>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> rate
> >>>>>>>>> limitation
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> nics as
> >>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> is a
> >>>>>> part of
> >>>>>>>>> I/O
> >>>>>>>>>>>>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Sorry
> >>>>>> my
> >>>>>>>>>>>>>>> misunstand the second
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Agree
> >>>>>> with
> >>>>>>>>>>>>> what
> >>>>>>>>>>>>>>> you said about
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> 2013/6/3
> >>>>>>>>> John
> >>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>> Jun 3,
> >>>>>>>>> 2013,
> >>>>>>>>>>>>> at
> >>>>>>>>>>>>>>> 2:13 AM, Wei
> >>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> Hi
> >>>>>> John,
> >>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> I
> >>>>>> hope
> >>>>>>>>> Mike's
> >>>>>>>>>>>>>>> aswer helps you.
> >>>>>>>>>>>>>>>>>>> I am
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> (1) I
> >>>>>>>>> think
> >>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>> depend
> >>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>> limitation.
> >>>>>>>>>>>>> Please
> >>>>>>>>>>>>>>> review
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> get
> >>>>>> the
> >>>>>>>>> IO
> >>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>> virtual
> >>>>>>>>>>>>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> Going by
> >>>>>>>>> the
> >>>>>>>>>>>>> AWS
> >>>>>>>>>>>>>>> model,
> >>>>>>>>>>>>>>>> customers
> >>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> provisioned
> >>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>> as well as,
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://aws.amazon.com/ebs/).
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> option to
> >>>>>>>>>>>>> employ
> >>>>>>>>>>>>>>> similar cost
> >>>>>>>>>>>>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> billing
> >>>>>>>>> model
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the current
> >>>>>>>>>>>>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> (2)
> >>>>>> Do
> >>>>>>>>> you
> >>>>>>>>>>>>> mean
> >>>>>>>>>>>>>>> IOPS runtime
> >>>>>>>>>>>>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>> limitation for
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>>>> virtual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>> does
> >>>>>>>>>>>>>>> not support
> >>>>>>>>>>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>> (computer
> >>>>>>>>>>>>>>> offering or disk
> >>>>>>>>>>>>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>>>> meant at
> >>>>>>>>> the
> >>>>>>>>>>>>>>> Java interface
> >>>>>>>>>>>>>>>>>>> level.
> >>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> we
> >>>>>> more
> >>>>>>>>>>>>>>> generalize allocation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> describe
> >>>>>>>>> the
> >>>>>>>>>>>>>>> service guarantees
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> (3)
> >>>>>> It is a
> >>>>>>>>>>>>> good
> >>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>> Maybe it
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>> disk_io_throttling as Mike
> >>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>> type, I
> >>>>>>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >> will
> >>>>>>>>> expand
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> my thoughts
> >>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> touch
> >>>>>>>>> points
> >>>>>>>>>>>>>>> between these two
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> will
> >>>>>> need
> >>>>>>>>> to be
> >>>>>>>>>>>>>>> merged before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> coordination
> >>>>>>>>>>>>>>> between the
> >>>>>>>>>>>>>>>> branches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> disk_io_throttling)
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >> -
> >>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>> 2013/6/3
> >>>>>>>>>>>>> John
> >>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> The
> >>>>>>>>> things I
> >>>>>>>>>>>>>>> want to
> >>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> 1)
> >>>>>> Is
> >>>>>>>>> there
> >>>>>>>>>>>>>>> value in
> >>>>>>>>>>>>>>>> capturing
> >>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> data
> >>>>>>>>> model
> >>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> 2)
> >>>>>>>>> Should
> >>>>>>>>>>>>> there
> >>>>>>>>>>>>>>> be a common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>> provisioning
> >>>>>>>>>>>>> at
> >>>>>>>>>>>>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> 3)
> >>>>>> How
> >>>>>>>>> are
> >>>>>>>>>>>>>>> conflicting
> >>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> storage device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> is
> >>>>>> lead
> >>>>>>>>> to
> >>>>>>>>>>>>>>> believe (and
> >>>>>>>>>>>>>>>> billed)
> >>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> storage
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> could
> >>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>> consistent
> >>>>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>>>> between a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> time,
> >>>>>>>>> but a
> >>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>> modification
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> -
> >>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> On
> >>>>>> Jun
> >>>>>>>>> 2,
> >>>>>>>>>>>>> 2013,
> >>>>>>>>>>>>>>> at 8:38 PM,
> >>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> Hi
> >>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> I
> >>>>>>>>> believe
> >>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>> feature deals
> >>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number
> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> the
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> My
> >>>>>>>>> feature
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>> focused on
> >>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> I
> >>>>>> hope
> >>>>>>>>> that
> >>>>>>>>>>>>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> On
> >>>>>> Sun,
> >>>>>>>>> Jun
> >>>>>>>>>>>>> 2,
> >>>>>>>>>>>>>>> 2013 at 6:35
> >>>>>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> My
> >>>>>>>>>>>>> opinion is
> >>>>>>>>>>>>>>> that no
> >>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> issues
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>>> been resolved
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> total
> >>>>>>>>> Ops
> >>>>>>>>>>>>> vs
> >>>>>>>>>>>>>>> discrete
> >>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> re-
> >>>>>>>>> reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido,
> >>>>>>>>> I
> >>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>> think this
> >>>>>>>>>>>>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Also,
> >>>>>>>>> how
> >>>>>>>>>>>>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> also
> >>>>>>>>>>>>> involves
> >>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> scenario
> >>>>>>>>>>>>>>> where provisioned
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> -
> >>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> On
> >>>>>>>>> Jun 1,
> >>>>>>>>>>>>> 2013,
> >>>>>>>>>>>>>>> at 6:47 AM,
> >>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Sure. I
> >>>>>>>>> will
> >>>>>>>>>>>>>>> change it next
> >>>>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> -
> >>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> 2013/6/1
> >>>>>>>>>>>>>>> Wido den Hollander
> >>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> On
> >>>>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> Exactly. I
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>>> pushed the
> >>>>>>>>>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> If
> >>>>>>>>> anyone
> >>>>>>>>>>>>>>> object thems for
> >>>>>>>>>>>>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> For
> >>>>>>>>> the
> >>>>>>>>>>>>> sake
> >>>>>>>>>>>>>>> of clarity I
> >>>>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> the
> >>>>>>>>> total
> >>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>> to R/W IOps
> >>>>>>>>>>>>>>>>>>> asap
> >>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> only
> >>>>>>>>> total
> >>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> You
> >>>>>>>>> laid
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> groundwork for
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> however
> >>>>>>>>>>>>>>> prevent that we
> >>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> -
> >>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> 2013/5/31
> >>>>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> On
> >>>>>>>>>>>>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> +1 --
> >>>>>>>>> this
> >>>>>>>>>>>>>>> enhancement must
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> don't
> >>>>>>>>> see
> >>>>>>>>>>>>> how
> >>>>>>>>>>>>>>> it could be
> >>>>>>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> split
> >>>>>>>>> total
> >>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>> into read
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> total
> >>>>>>>>>>>>>>> unless/until we
> >>>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> What
> >>>>>>>>> Wei
> >>>>>>>>>>>>>>> meant was merging
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> 4.2
> >>>>>>>>> branch
> >>>>>>>>>>>>>>> and add Read /
> >>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> will be
> >>>>>>>>>>>>>>> released with Read
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> This is
> >>>>>>>>> to
> >>>>>>>>>>>>>>> make the May 31st
> >>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> (see
> >>>>>>>>> other
> >>>>>>>>>>>>>>> threads) then it
> >>>>>>>>>>>>>>>>>>> won't
> >>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> I
> >>>>>> also
> >>>>>>>>>>>>>>> completely agree that
> >>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> disk
> >>>>>>>>> I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> -
> >>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> On
> >>>>>>>>> May 31,
> >>>>>>>>>>>>>>> 2013, at 9:51 AM,
> >>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> On
> >>>>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Hi
> >>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> Thanks.
> >>>>>>>>>>>>> Good
> >>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> I
> >>>>>>>>> thought
> >>>>>>>>>>>>>>> about at the
> >>>>>>>>>>>>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> difference
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>> read and write
> >>>>>>>>>>>>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> care
> >>>>>>>>> the
> >>>>>>>>>>>>>>> difference of sent
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> That
> >>>>>>>>>>>>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Disk
> >>>>>>>>> I/O is
> >>>>>>>>>>>>>>> much more
> >>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> Implementing
> >>>>>>>>>>>>>>> it will be some
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> few
> >>>>>>>>> days.
> >>>>>>>>>>>>> For
> >>>>>>>>>>>>>>> the deadline
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> that ,
> >>>>>>>>> if
> >>>>>>>>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> It
> >>>>>>>>> think it's
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>> feature we
> >>>>>>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> window
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>>> have to make sure
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> it
> >>>>>> in
> >>>>>>>>> 4.3,
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>> will confuse
> >>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> -
> >>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> 2013/5/31
> >>>>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> On
> >>>>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> I
> >>>>>>>>> would
> >>>>>>>>>>>>> like to
> >>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> If
> >>>>>>>>> nobody
> >>>>>>>>>>>>>>> object, I will
> >>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> The
> >>>>>>>>>>>>> purpose
> >>>>>>>>>>>>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Virtual
> >>>>>>>>>>>>>>> machines are running
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> share
> >>>>>>>>>>>>> strage).
> >>>>>>>>>>>>>>> Because of
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> iops),
> >>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> one
> >>>>>>>>> VM
> >>>>>>>>>>>>> has
> >>>>>>>>>>>>>>> large disk
> >>>>>>>>>>>>>>>>>>> operation,
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> other
> >>>>>>>>> VMs
> >>>>>>>>>>>>>>> running on the
> >>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> It
> >>>>>> is
> >>>>>>>>>>>>> neccesary
> >>>>>>>>>>>>>>> to set the
> >>>>>>>>>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> Looking at
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> code I see
> >>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Qemu
> >>>>>>>>> and
> >>>>>>>>>>>>>>> libvirt support
> >>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> IOps
> >>>>>>>>>>>>> which
> >>>>>>>>>>>>>>> could benefit a
> >>>>>>>>>>>>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> It's
> >>>>>>>>> also
> >>>>>>>>>>>>>>> strange, in the
> >>>>>>>>>>>>>>>>>>> polling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> IOps,
> >>>>>>>>> but
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> the throttling
> >>>>>>>>>>>>>>>>>>> side
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Write
> >>>>>>>>> IOps
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>>> usually much
> >>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> like a
> >>>>>>>>> valid
> >>>>>>>>>>>>>>> use-case where
> >>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> IOps
> >>>>>>>>> vs
> >>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Since
> >>>>>>>>> this
> >>>>>>>>>>>>> only
> >>>>>>>>>>>>>>> supports KVM
> >>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> value
> >>>>>>>>> to at
> >>>>>>>>>>>>>>> least have the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> this
> >>>>>>>>> later
> >>>>>>>>>>>>>>> would be a lot of
> >>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> If
> >>>>>> a
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> doesn't
> >>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> write
> >>>>>>>>> you
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>>> always sum
> >>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Can
> >>>>>>>>> you
> >>>>>>>>>>>>>>> explain why you
> >>>>>>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> The
> >>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> (1) set
> >>>>>>>>> the
> >>>>>>>>>>>>>>> maximum rate of
> >>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> (2)
> >>>>>>>>> change
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> maximum rate
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> (3)
> >>>>>>>>> limit
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> disk rate
> >>>>>>>>>>>>>>>> (total
> >>>>>>>>>>>>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> JIRA
> >>>>>>>>> ticket:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-
> >> 1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>>>> 1192
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> FS (I
> >>>>>>>>> will
> >>>>>>>>>>>>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> Merge
> >>>>>>>>>>>>> check
> >>>>>>>>>>>>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> *
> >>>>>> Did
> >>>>>>>>> you
> >>>>>>>>>>>>>>> check the branch's
> >>>>>>>>>>>>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> *
> >>>>>> Are
> >>>>>>>>>>>>> there
> >>>>>>>>>>>>>>> new dependencies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> *
> >>>>>>>>> What
> >>>>>>>>>>>>>>> automated testing
> >>>>>>>>>>>>>>>> (unit
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Unit
> >>>>>>>>> tests
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> *
> >>>>>>>>> What
> >>>>>>>>>>>>>>> testing has been done
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> (1) set
> >>>>>>>>> the
> >>>>>>>>>>>>>>> bytes rate and
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> (2)
> >>>>>>>>> VM
> >>>>>>>>>>>>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>> deploy,
> >>>>>>>>>>>>> stop,
> >>>>>>>>>>>>>>> start, reboot,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> (3)
> >>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>> operations,
> >>>>>>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Attach,
> >>>>>>>>>>>>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> To
> >>>>>>>>> review
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> code, you can
> >>>>>>>>>>>>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Best
> >>>>>>>>>>>>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> [2]
> >>>>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> >>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-
> 2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> -
> >>>>>>>>> VM
> >>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> *Mike
> >>>>>>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> *Senior
> >>>>>>>>>>>>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> e:
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >> o:
> >>>>>>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>> Advancing
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> way the world
> >>>>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Mike
> >>>>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Senior
> >>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> Advancing
> >>>>>>>>> the
> >>>>>>>>>>>>> way
> >>>>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Mike
> >>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Senior
> >>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> Advancing the
> >>>>>>>>>>>>> way
> >>>>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Mike
> >>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Advancing
> >>>>>> the
> >>>>>>>>> way
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Advancing
> >>>>>> the
> >>>>>>>>> way
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>> CloudStack
> >>>>>>>>>>>>>>> Developer, SolidFire
> >>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> >> the
> >>>>>> way
> >>>>>>>>> the
> >>>>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >> the
> >>>>>> way
> >>>>>>>>> the
> >>>>>>>>>>>>> world
> >>>>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >>>>>> way the
> >>>>>>>>>>>>> world
> >>>>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>>>> the
> >>>>>>>>> world
> >>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> >> the
> >>>>>>>>> world
> >>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >> the
> >>>>>> world
> >>>>>>>>> uses
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> >>>>>> world
> >>>>>>>>> uses
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>>>> uses
> >>>>>>>>> the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>>>> uses
> >>>>>>>>> the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>>>> uses
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >> uses
> >>>>>> the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> the
> >>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *(tm)*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*****
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*****
> >>>>>>>>
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>>>
> >>>>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>>>
> >>>>>>>> o: 303.746.7302****
> >>>>>>>>
> >>>>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *(tm)*****
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ****
> >>>>>>>>
> >>>>>>>> ** **
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*****
> >>>>>>>>
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>>>
> >>>>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>>>
> >>>>>>>> o: 303.746.7302****
> >>>>>>>>
> >>>>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *(tm)*****
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *(tm)*
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *(tm)*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *(tm)*
> >
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I see the unmanaged() method in the life cycle object now (didn't know this
was there before). I also notice it is not being invoked anywhere.
I can make use of this method to help the storage framework determine if it
should send a message to the hypervisor in use to create an SR (or
datastore for VMware).
By the way, if I set the StoragePoolType of my plug-in to Iscsi, exceptions
are thrown. I think CloudStack is trying to handle my storage in a way it
shouldn't when that value is used.
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Edison,
Please see my response in-line below.
Thanks,
-John
On Jun 10, 2013, at 4:45 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: John Burwell [mailto:jburwell@basho.com]
>> Sent: Monday, June 10, 2013 1:25 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>> Mike,
>>
>> Yes, a vendor may hand a non-managed plugin. For example, on the
>> secondary storage side, S3 and Swift don't support the concept of being
>> managed -- an operator is required to create a bucket and point CloudStack
>> at it. Another example would be a generic iSCSI device (e.g. OpenFiler,
>> FreeNAS) which CloudStack can not directly manage, but can consume a LUN
>> that an operator has created for it.
>
> Manage or non-managed storage is handled by DataStoreLifeCyclce.
> It depends on storage vendor to implement DataStoreLifeCyclce, which basically says how to initialize a storage, how to add a storage into a cluster etc.
So, implementation of the DataStoreLifeCycle interface indicates that driver is capable of managing the device. It does not answer the question of whether or not the administrator/operator wants CloudStack to manage the device. Hence, the need for a flag on the StoragePool to indicate we should be managing it. I am cool with the flag defaulting to true when the driver supports it, but we must provide the administrator/operator to ability to opt-out. Does this interface provide for a set of administrative credentials separate from the credentials used to access storage?
Another question comes to mind -- why do we have a Dynamic StoragePoolType when this interface exists?
>
> Take S3 as an example, if the S3 storage provider provides a DataStoreLifeCyclce which can create a bucket, then when admin adding a S3 storage through API, mgt server will first call DataStoreLifeCyclce, which will create a bucket, then stored in the image_store_details table, then later on, all the referenced to this image store, will contain this bucket information.
From an S3 perspective, creating a bucket from CloudStack would not a be security best practice. The general guidance would be for the bucket to created and associated to a dedicated CloudStack access/secret key pair to properly isolate data. Given this best practice and that only one bucket per S3 instance would be necessary for CloudStack operation, I think most administrators/operators would want to opt-out CloudStack managing S3 buckets.
>
> Take an storage box from some vendor as an example, admin wants to create an ISCSI server on a storage box, which can be used by a cluster(multiple volumes on a giant LUN ), as long as the storage provider can provide a DataStoreLifeCyclce which can program the storage box, and create an ISCSI server, then stored the information in storage_pool table, then all the referenced to the storage pool will use the dynamically created ISCSI LUN.
>
> Take another storage box from some vendor as an example, admin wants to add the whole storage box into cloudstack, which can be used as per LUN per ISCSI, so in this case, DataStoreLifeCyclce won't create an ISCSI server during the initialize method call. Instead, for every createAsync, it will create an ISCSI LUN, stored the LUN information in volume table and volume_detail table. Whenever the LUN is used by hypervisor, the hypervisor needs to take extra care about the volume.
>
>
>
>
>>
>> From an operational perspective, an administrator may elect to say, "Hey, I
>> know you can manage that device for me, but it is has a wider use than
>> CloudStack can manage. Let me handle it, and just provide the information
>> CloudStack needs." In this scenario, the operator is accepting a greater
>> management burden in exchange for control. I don't think many will elect to
>> manage these devices themselves, but this ability will be extremely
>> important for the edge cases that require it. Speaking as someone who has
>> run cloud operations, I can testify to this need -- nothing aggravates you
>> more than a system insisting on managing a device or service without the
>> opportunity to opt-out
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 3:19 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> So, would a storage vendor write a non-CloudStack-managed plug-in and
>> maybe
>>> just implement the snapshot feature or something?
>>>
>>> In this case, they'd have to set the hypervisor up ahead of time with a
>>> large portion of their storage, but they'd still get the vendor-snapshot
>>> ability.
>>>
>>> Just trying to undertand this more.
>>>
>>> Thanks
>>>
>>>
>>> On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>
>>>> Edison,
>>>>
>>>> TL;DR Always give operators/system administrators the ability to manage
>>>> things themselves, if they like.
>>>>
>>>> As more manageable drivers emerge, I think it will be import to allow
>>>> operators to opt-out of CloudStack device management. Therefore, a
>> driver
>>>> declares whether or not it can support device management (many, in fact
>>>> most, won't). For those drivers that do support management, allow
>>>> operators to opt-in/out of CloudStack management on a per device basis.
>> To
>>>> support this notion, we need a flag on the storage_pool table and
>>>> associated classes to indicate whether or not the operator wants
>> CloudStack
>>>> to manage the device.
>>>>
>>>> Does this explanation make more sense?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>
>>>>> I don't understand why we need to add a "managed" in the UI and in
>>>> "storage_pool" table?
>>>>> AFAIK, what you guys trying to do is to enable
>>>> attachVolume/DettachVolume if the volume is created on Solidfire, right?
>>>>> If the volume is created on Solidfire already, by createAsync, then why
>>>> we can't just send out all the information about the volume(it's volumeTO
>>>> which will be returned by each storage driver) to hypervisor, then at the
>>>> hypervisor side, check the volumeTO, if the volumeTO is managed, then
>> doing
>>>> whatever necessary operations(create SR, create VDI etc), then attach
>> the
>>>> VDI to volume.
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>> Sent: Monday, June 10, 2013 11:10 AM
>>>>>> To: John Burwell
>>>>>> Cc: Edison Su; dev@cloudstack.apache.org
>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>
>>>>>> I believe we're on the same page, John.
>>>>>>
>>>>>> I would add a new parameter to the createStoragePool API command
>> (let's
>>>>>> call it "managed" and it can be true or false). If you pass in "true",
>>>> but
>>>>>> the driver does not support being managed by CloudStack, an
>> exception is
>>>>>> thrown by the driver. If you pass in "false", but the driver does not
>>>>>> support being managed outside of CloudStack, an exception is thrown
>> by
>>>> the
>>>>>> driver. It is possible one could write a driver that can be managed both
>>>>>> inside or outside of CloudStack (the SolidFire driver requires being
>>>>>> managed only by CloudStack).
>>>>>>
>>>>>> This managed value would end up in the storage_pool table in a new
>>>> column
>>>>>> ("managed").
>>>>>>
>>>>>> An example usage: After the storage framework calls on the driver's
>>>>>> createAsync method, it will check to see if the driver is managed. If
>>>> the
>>>>>> driver is managed, the storage framework will invoke the proper (new)
>>>>>> command of the hypervisor in use to set up, say, the SR for the SAN
>>>> volume.
>>>>>>
>>>>>> There may be other cases when the isManaged() method is invoked on
>> the
>>>>>> driver and - if true is returned - the storage framework sends a
>>>> particular
>>>>>> command to the hypervisor in use.
>>>>>>
>>>>>> ********************
>>>>>>
>>>>>> As far as reworking the default storage plug-in (which sends commands
>> to
>>>>>> hypervisors) for 4.2, I would check with Edison on his schedule.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> I want to make sure we are on the same page regarding isManaged. I
>> see
>>>>>>> the addition of the following operations:
>>>>>>>
>>>>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates
>> whether
>>>>>> or
>>>>>>> not the driver support/implements the management functions
>>>>>>> * DataStore#isManaged() : boolean -- indicates whether or not a
>> device
>>>> is
>>>>>>> managed by CloudStack
>>>>>>>
>>>>>>> Only DataStores associated with DataStoreDrivers that support
>>>>>> management
>>>>>>> would be able to have the management flag enabled. This behavior
>>>> allows
>>>>>>> operators to declare their intention to allow CloudStack to manage a
>>>>>>> device. This behavior would need to be exposed as flag on the HTTP
>>>> API.
>>>>>>> Additionally, this flag would only be settable when a device is
>>>>>>> defined/created in the system (i.e. once a device is managed by
>>>> CloudStack,
>>>>>>> you can't unmanaged it without dissociating it from CloudStack). In
>>>> the
>>>>>>> future, we may consider relaxing that rule, but, in the short term, it
>>>>>>> greatly decreases the alternative flows.
>>>>>>>
>>>>>>> In terms of removing the dependencies from Storage to Hypervisor, I
>>>>>> would
>>>>>>> really like to see this accomplished in 4.2. It will be much more
>>>>>>> difficult to address this issue post 4.2 because we will have more
>>>> plugins
>>>>>>> to address. I also see this issue as a critical architectural issues
>>>> that
>>>>>>> for the management server. I recommend we identify the
>> dependencies
>>>>>> that
>>>>>>> need to be inverted, and see if we can divide the work across a few
>>>> more
>>>>>>> resources.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
>>>>>> <mi...@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Sounds good, Edison :)
>>>>>>>
>>>>>>> I can implement John's isManaged() logic and resubmit my code for
>>>> review.
>>>>>>>
>>>>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
>>>> requires
>>>>>>> some refactoring of the storage framework, then I could do that.
>>>>>>>
>>>>>>> Also, the default storage plug-in does have hypervisor logic in it in
>>>> 4.2.
>>>>>>> I recommend we leave this as is for 4.2 and it can be refactored in 4.3
>>>>>>> with the above work I mentioned.
>>>>>>>
>>>>>>> Are you OK with that, guys?
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
>>>> wrote:
>>>>>>>
>>>>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
>>>>>>>> *To:* dev@cloudstack.apache.org
>>>>>>>> *Cc:* Edison Su; John Burwell
>>>>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Another thing we might want to consider for 4.2 is that the way the
>>>> code
>>>>>>>> is currently implemented in master, I don't think it's even possible
>>>> to
>>>>>>>> write a plug-in that connects storage directly to a VM because the
>>>>>>>> CloudStack storage framework will call the "attach" method of the
>>>>>>>> hypervisor in use and this method will fail because it assumes there
>>>> is
>>>>>>>> (talking Xen here) an SR existent and in this situation there isn't
>>>> one.*
>>>>>>>> ***
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> My thinking is that we should ignore the direct-attach-of-volume-to-
>> VM
>>>>>>>> use case for 4.2. I think it will require too much of a rewrite at
>>>> this
>>>>>>>> point in development (fewer than three weeks left until code
>>>>>> freeze).****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> If the plug-in implements an isManaged() method (as John
>> suggested),
>>>>>> the
>>>>>>>> storage framework can invoke the necessary commands on the
>>>>>> hypervisor in
>>>>>>>> use.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> For example, the createAsync method is called. The storage plug-in
>>>>>>>> creates a SAN volume and updates the CS DB. The storage
>> framework
>>>>>> then asks
>>>>>>>> the driver if it's managed. If it is, the storage framework executes a
>>>>>>>> command against the hypervisor in use to have it create, say, an SR
>>>> (with
>>>>>> a
>>>>>>>> single VDI that takes up the entire SR). Now, when the storage
>>>>>> framework
>>>>>>>> executes the "attach" command against the hypervisor, this
>> command
>>>> will
>>>>>>>> work, as well (because that logic assumes the prior existence of an
>>>> SR and
>>>>>>>> it will, in fact, be present).****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:****
>>>>>>>>
>>>>>>>> Hi Edison and John,****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> I wanted to point out something I thought was relevant to this
>>>>>>>> conversation.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> My plug-in supports both XenServer and ESX(i).****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> These two hypervisors handle their respective hypervisor data
>>>> structures
>>>>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> In XenServer, the SR that resides on the SAN LUN is simply
>> "forgotten"
>>>>>>>> when you issue a remove-SR command to XenServer.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> When you later want to access the contents of that data, it is still
>>>>>>>> safely stored on the LUN and can be accessed from XenServer (from
>> the
>>>>>> same
>>>>>>>> or different cluster (what XenServer calls a resource pool)) by
>>>> creating a
>>>>>>>> new SR that is based on the same IQN and LUN.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> XenServer SRs exist at the cluster level.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
>>>>>>>> destroyed when you issue a remove-datastore command to ESX(i).
>> In
>>>>>> other
>>>>>>>> words, the contents of the SAN LUN are destroyed.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> ESX(i) datastores exist at the datacenter level.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> The XenServer behavior (not destroying the contents of the LUN
>> when
>>>>>> the
>>>>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
>>>> contents
>>>>>>>> of the LUN when the datastore is removed) is not desirable.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> I can get around the ESX(i) behavior by removing the referenced IQN
>>>>>> from
>>>>>>>> each host in the cluster when the CS storage framework notifies my
>>>> plug-
>>>>>> in
>>>>>>>> of a detach-volume event. When the CS volume is later attached
>> again
>>>>>>>> (either to the same or a different ESX(i) cluster), I can have each
>>>> host in
>>>>>>>> the relevant ESX(i) cluster reference the IQN and the datastore will
>>>> be
>>>>>>>> back again.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> This sequence of events is critical for my plug-in to support both
>>>>>>>> environments:****
>>>>>>>>
>>>>>>>> *XenServer flow (events for Driver class):*****
>>>>>>>>
>>>>>>>> createAsync (called right before first attach) = creates SAN
>>>> volume****
>>>>>>>>
>>>>>>>> preAttachVolume = creates SR and creates VDI or just introduces
>> SR****
>>>>>>>>
>>>>>>>> (CS volume gets attached and later detached)****
>>>>>>>>
>>>>>>>> postDetachVolume = removes SR (this does not delete data on SAN
>>>>>> volume)**
>>>>>>>> **
>>>>>>>>
>>>>>>>> deleteAsync (can be called when volume is not attached) = deletes
>> the
>>>>>> SAN
>>>>>>>> volume****
>>>>>>>>
>>>>>>>> *VMware flow (events for Driver class):*****
>>>>>>>>
>>>>>>>> createAsync (called right before first attach) = creates SAN volume,
>>>>>>>> creates datastore, and creates VMDK file****
>>>>>>>>
>>>>>>>> preAttachVolume = adds iSCSI target to each host in cluster****
>>>>>>>>
>>>>>>>> (CS volume gets attached and later detached)****
>>>>>>>>
>>>>>>>> postDetachVolume = removes iSCSI target from each host in
>> cluster****
>>>>>>>>
>>>>>>>> deleteAsync (can be called when volume is not attached) = deletes
>>>>>>>> datastore and deletes the SAN volume****
>>>>>>>>
>>>>>>>> ************************
>>>>>>>>
>>>>>>>> So, the way I have it written up here, the driver is talking to the
>>>>>>>> hypervisor. As John has pointed out, this may not be ideal.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Whatever we decide on, it needs to be able to support this kind of
>>>> flow.
>>>>>>>> :)****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> We could go John's route, where the driver says if it's managed or
>>>> not.**
>>>>>>>> **
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> If it is, then the storage framework could send the necessary
>>>> messages to
>>>>>>>> the hypervisor in question at the right times (just migrate the
>>>> hypervisor
>>>>>>>> logic above from the driver to the right parts of the storage
>>>> framework).
>>>>>>>> ****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> This is fine, except that it does not resolve Edison's use case of
>>>>>>>> desiring to have the storage LUN potentially connected directly to a
>>>> VM.*
>>>>>>>> ***
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> I'm not sure we should design around that use case, though. Is it
>>>> common
>>>>>>>> enough? Has anyone asked for it? Is it even desirable? How are
>> usage
>>>>>>>> statistics gathered for such volumes? It's nice to offer the
>>>> flexibility,
>>>>>>>> but is it a good idea in this case?****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Thanks!****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
>>>> wrote:**
>>>>>>>> **
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>> Sent: Friday, June 07, 2013 4:08 PM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>
>>>>>>>>> Hi John,
>>>>>>>>>
>>>>>>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
>>>>>>>> volume
>>>>>>>>> to use it. Same story for VMware: ESX(i) must layer a datastore on
>>>> top
>>>>>>>> of
>>>>>>>>> the iSCSI volume to use it. I look at it like they are layering a
>>>>>>>> clustered
>>>>>>>>> file system on the SAN volume so the hypervisors can share access
>> to
>>>>>> the
>>>>>>>>> contents of the volume.
>>>>>>>>>
>>>>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called, the
>>>>>>>> SR or
>>>>>>>>> datastore has already been created (usually manually by an admin).
>>>>>>>>>
>>>>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
>>>>>>>>> environment where each CloudStack volume that a user creates is
>>>>>> mapped
>>>>>>>>> to a
>>>>>>>>> single SAN volume (via the SR).
>>>>>>>>>
>>>>>>>>> The question comes down to who should allocate the SR
>> dynamically.
>>>>>>>>>
>>>>>>>>> We could have the storage framework ask the storage plug-in if it is
>>>>>>>>> managed. If it is, then the storage framework could send a
>> message to
>>>>>>>> the
>>>>>>>>> hypervisor in question to create (let's talk Xen here) the SR ahead
>>>> of
>>>>>>>>> time. Then, when the storage framework next sends the attach-
>> volume
>>>>>>>>> command
>>>>>>>>> to the hypervisor, it should work without changes to that
>>>> attach-volume
>>>>>>>>> logic (because - from the point of view of the attach logic - the SR
>>>> is
>>>>>>>>> already existent, as expected).
>>>>>>>>>
>>>>>>>>> Now, as Edison has pointed out, this limits the power of the storage
>>>>>>>>> plug-in. A storage plug-in in this model cannot directly attach
>>>> storage
>>>>>>>> to
>>>>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We
>> need
>>>>>> to
>>>>>>>>> make a call on that.
>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, I think we need to address this usage case, and won't limit the
>>>>>>>> power of the storage plug-in.
>>>>>>>> John, what's your idea to fit this usage case into your abstraction?
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> If we want to give more power to the storage vendor, we would
>> have to
>>>>>>>>> have
>>>>>>>>> the storage framework call into the storage plug-in to send the
>>>>>>>> appropriate
>>>>>>>>> attach commands to the hypervisor.
>>>>>>>>>
>>>>>>>>> Let's discuss and come to a consensus (or at least agree on a path)
>>>> as
>>>>>>>> soon
>>>>>>>>> as we can. :)
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> My understanding is the Xen expects an SR structure on all ISCSI
>>>>>>>> devices
>>>>>>>>>> -- at least that is how I read the code in your patch. Is my
>>>>>>>> understanding
>>>>>>>>>> correct? If so, the Xen plugin should be able to query the storage
>>>>>>>> device
>>>>>>>>>> to determine the presence of the SR structure and create it if it
>>>>>>>> does not
>>>>>>>>>> exist. Am I missing something in the implementation that makes
>> that
>>>>>>>> type
>>>>>>>>>> of implementation impossible?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yeah, if a storage vendor wanted to attach a volume directly to a
>>>>>>>> VM in
>>>>>>>>>> 4.2
>>>>>>>>>>> today, it would probably fail because the attach-volume logic
>>>>>>>> assumes
>>>>>>>>> the
>>>>>>>>>>> existence of the necessary hypervisor data structure (ex. SR on
>>>>>>>> Xen).
>>>>>>>>>>>
>>>>>>>>>>> If we wanted to enable such an attach, we could do it the way
>>>>>> Edison
>>>>>>>>>>> suggests.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>
>>>>>>>>>>>>> As we only have three weeks until feature freeze, we should
>>>>>> come
>>>>>>>> to
>>>>>>>>> a
>>>>>>>>>>>>> consensus on this design point as soon as possible.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Right now, if the storage framework asks my driver it is is
>>>>>>>> managed, it
>>>>>>>>>>>>> will say 'yes.' This means the framework will tell the driver to
>>>>>>>>>> perform
>>>>>>>>>>>>> its management activities. This then means the driver will call
>>>>>>>> into
>>>>>>>>>> the
>>>>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to
>> perform
>>>>>> the
>>>>>>>>>>>> activity
>>>>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The driver doesn't know which hypervisor it's talking to, it just
>>>>>>>>>> sends a
>>>>>>>>>>>>> message to the host to perform the necessary pre-attach
>> work.
>>>>>>>>>>>>
>>>>>>>>>>>> Could we just expose a method like
>>>>>> "attachVolume/dettachVolume" on
>>>>>>>>> the
>>>>>>>>>>>> PrimaryDataStoreDriver driver?
>>>>>>>>>>>> For most of cases, the implementation of each driver would
>> just
>>>>>>>> send
>>>>>>>>>>>> attachvolumecommand/dettachvolumecmd to hypervisor(we
>> can
>>>>>> put
>>>>>>>>> the
>>>>>>>>>>>> implementation in a base class, so that can be shared by all of
>>>>>>>> this
>>>>>>>>>>>> drivers), and in each hypervisor resource code, which may just
>>>> call
>>>>>>>>>>>> hypervisor's api to attach the volume to VM, while for certain
>>>>>>>> storage,
>>>>>>>>>>>> like SolidFire, may need to create a SR first, and create volume
>>>>>>>> on it,
>>>>>>>>>>>> then call hypervisor's API to attach volume to VM.
>>>>>>>>>>>> While for some other storage vendor, may want to bypass
>>>>>> hypervisor
>>>>>>>>>> during
>>>>>>>>>>>> attaching volume, so inside driver's attachvolume
>> implementation,
>>>>>>>> the
>>>>>>>>>>>> driver can do some magic, such as, directly talk to an agent
>>>>>>>> inside VM
>>>>>>>>>>>> instance, then create a disk inside VM.
>>>>>>>>>>>>
>>>>>>>>>>>> How do you guys think?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su
>> <Ed...@citrix.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Mike Tutkowski
>> [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> How's about this:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The driver can implement an isManaged() method. The
>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>> (which is responsible for calling into the hypervisor to
>> attach
>>>>>>>> the
>>>>>>>>>>>> disk)
>>>>>>>>>>>>>>> can call into the hypervisor to create the necessary
>> hypervisor
>>>>>>>> data
>>>>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The problem here is that storage vendor may work
>> differently
>>>>>> with
>>>>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while
>>>>>>>> maybe
>>>>>>>>>>>> other
>>>>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the
>> LUN
>>>>>>>>> directly
>>>>>>>>>>>> to
>>>>>>>>>>>>>> VM instance, see the discuss(
>>>>>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>>>>>>>>>>>>> %3E
>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>> So I would let storage provider to implement attach disk to
>> VM,
>>>>>>>>>>>> instead of
>>>>>>>>>>>>>> implemented by cloudstack itself.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If that's what you're going for, that works for me. By the
>> way,
>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>> default storage plug-in (which handles the default storage
>>>>>>>> behavior
>>>>>>>>>>>> in
>>>>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that
>>>>>>>> talks to
>>>>>>>>>>>>>>> hypervisors. You might want to contact him and inform him
>> of
>>>>>>>> your
>>>>>>>>>>>>>> concerns
>>>>>>>>>>>>>>> or that logic (as is) will make it to production.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please let me know if what I wrote in above (for my
>> solution)
>>>>>>>> is OK
>>>>>>>>>>>> with
>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please see my responses in-line below.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I build
>>>>>>>> up
>>>>>>>>>>>> a
>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>> file, but this is the gist of what I've done:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * During a volume-attach operation, after
>>>>>> VolumeManagerImpl
>>>>>>>>> tells
>>>>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a volume, I
>>>>>> have
>>>>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
>>>>>> driver if
>>>>>>>> it
>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver
>> perform
>>>>>>>>>>>> whatever
>>>>>>>>>>>>>>>>> activity is required. In my case, this includes sending a
>>>>>>>>>>>> message to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> host where the VM is running to have, say XenServer,
>> add a
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the IQN
>> of
>>>>>> the
>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space
>> it
>>>>>>>> can
>>>>>>>>>>>> on
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> storage repository). After this, the normal attach-volume
>>>>>>>>>>>> message is
>>>>>>>>>>>>>> sent
>>>>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There should be **no** code from a storage driver to a
>>>>>>>> hypervisor.
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> apologize for the repetition, but we simply can not have
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> specific code in the storage layer. The circular
>> dependencies
>>>>>>>>>>>> between
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> two layers are not sustainable in the long term. Either the
>>>>>>>>>>>>>> VirtualManager
>>>>>>>>>>>>>>>> or Xen hypervisor plugin needs to be refactored/modified
>> to
>>>>>>>>>>>>> coordinate
>>>>>>>>>>>>>>>> volume creation and then populating the SR. Ideally, we
>> can
>>>>>>>>>>>>>> generalize the
>>>>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
>>>>>> hypervisor
>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>> would only implement callbacks to perform the attach
>> action
>>>>>> and
>>>>>>>>>>>> create
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver should
>>>>>>>> only be
>>>>>>>>>>>>>>>> allocating space and providing information about contents
>>>>>> (e.g.
>>>>>>>>>>>> space
>>>>>>>>>>>>>>>> available, space consumed, streams to a URI, file handle
>> for a
>>>>>>>> URI,
>>>>>>>>>>>>>> etc)
>>>>>>>>>>>>>>>> and capabilities.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * The reverse is performed for a detach-volume
>> command.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in my
>>>>>>>> driver.
>>>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>> default driver simply does a "return false;". We could
>> add a
>>>>>>>> new
>>>>>>>>>>>>>>>> parameter
>>>>>>>>>>>>>>>>> to the createStoragePool API command, if we want, to
>>>>>> remove
>>>>>>>>> the
>>>>>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>>>> return values in the drivers (although my driver will
>>>> probably
>>>>>>>>>>>> just
>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>> this parameter and always return true since it wouldn't
>> make
>>>>>>>>>>>> sense
>>>>>>>>>>>>>> for it
>>>>>>>>>>>>>>>>> to ever return false). We'd need another column in the
>>>>>>>>>>>> storage_pool
>>>>>>>>>>>>>>> table
>>>>>>>>>>>>>>>>> to store this value.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes, I think we should have a parameter added to the
>>>>>>>>>>>>> createStoragePool
>>>>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
>>>>>>>> configured
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> management when their underlying drivers support it. To
>>>>>>>> simplify
>>>>>>>>>>>>>> things,
>>>>>>>>>>>>>>>> this flag should only be mutable when the DataStore is
>>>>>>>> created. It
>>>>>>>>>>>>>> would be
>>>>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to
>> unmanaged
>>>>>> after
>>>>>>>>>>>>>> creation.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
>>>>>> working
>>>>>>>>>>>> GUI
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>> to review (in that document), if you'd like to check it
>> out.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed
>> together to
>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor
>> throttled
>>>>>> IOPs
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the potential
>> for
>>>>>>>>>>>>>> confusion,
>>>>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these features
>> will
>>>>>>>>>>>> help
>>>>>>>>>>>>>>>>>>> minimize any potential confusion.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will work
>>>>>>>> itself
>>>>>>>>>>>> if
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
>>>> hypervisor
>>>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google
>> doc I
>>>>>> sent
>>>>>>>>> to
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> other day to see how the GUI looks for provisioned
>>>>>> storage
>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-mail
>>>> list
>>>>>>>>>>>> and we
>>>>>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add
>> Disk
>>>>>>>>> Offering
>>>>>>>>>>>>>> dialog.
>>>>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS, so
>> they
>>>>>>>>> should
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say,
>> only
>>>>>>>> use
>>>>>>>>>>>> Max
>>>>>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for
>> storage
>>>>>>>> that
>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
>>>>>>>> the
>>>>>>>>>>>>>> admin can
>>>>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and
>>>>>> Burst
>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's
>> feature
>>>>>> or
>>>>>>>>>>>> mine,
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> decided by the admin).
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two
>> features
>>>>>>>>>>>> expressing
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> speed
>>>>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express the
>>>>>> IOPS
>>>>>>>> for
>>>>>>>>>>>>>> QoS in
>>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
>>>>>>>> read/write
>>>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
>>>>>>>>>>>>> Storage/Hypervisor
>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the
>> behavior
>>>>>> of
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see
>> the
>>>>>>>>>>>> following
>>>>>>>>>>>>>>>> issues
>>>>>>>>>>>>>>>>>>>>> outstanding:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as
>> discrete
>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are expressed
>> as
>>>>>> total
>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS
>> attached
>>>>>> to
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for throttled
>> and
>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>> that will permit providers to discriminate these
>>>>>>>> guaranteed
>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
>>>>>>>> provisioned
>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
>>>>>>>> hypervisor
>>>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would
>> neatly
>>>>>>>> solve
>>>>>>>>>>>>>> items 1
>>>>>>>>>>>>>>>>>>> and 2.
>>>>>>>>>>>>>>>>>>>>> Since the two facilities would not be permitted to
>>>>>> operate
>>>>>>>>>>>>>> together,
>>>>>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I think
>> item
>>>>>> 3
>>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>> resolved
>>>>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage records.
>> As
>>>>>>>> for
>>>>>>>>>>>>>> Item 4,
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>>>>>> not familiar with how these two enhancements are
>>>>>>>>> depicted in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
>>>>>> enhancements
>>>>>>>>> for
>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>>>>>>>>> <us...@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> John and Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
>>>>>>>>> 2780/2729,
>>>>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-
>> 2823 ,
>>>>>>>>>>>>> CLOUDSTACK-
>>>>>>>>>>>>>>> 2875 ) this
>>>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes
>> tomorrow,
>>>>>> and
>>>>>>>>>>>> ask
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> second
>>>>>>>>>>>>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
>>>>>> <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going with
>> this,
>>>>>>>> John,
>>>>>>>>>>>> and
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager
>> determining
>>>>>>>>> whether or
>>>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
>> StoragePoolType,
>>>>>> but an
>>>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
>> allocate
>>>>>>>>>>>> resources
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an
>> isManaged()
>>>>>>>>> method
>>>>>>>>>>>>>>> added to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns
>> true,
>>>>>>>>>>>> then the
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or
>> whatever
>>>>>> name)
>>>>>>>>>>>>> method
>>>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface,
>> as
>>>>>> well)
>>>>>>>>> and
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to
>> create,
>>>>>> say
>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>> SR?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean
>> on
>>>>>>>>>>>>>>> DataStoreDriver.
>>>>>>>>>>>>>>>>>>> Since
>>>>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary
>>>>>> storage,
>>>>>>>>> I
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage
>> is a
>>>>>>>>> straw
>>>>>>>>>>>>> man
>>>>>>>>>>>>>>> term
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> something other than volume) methods to
>>>>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a
>>>>>> mutable
>>>>>>>>>>>>> property
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> DataStore
>>>>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the underlying
>>>>>>>>>>>>> DataStoreDriver
>>>>>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
>>>>>>>>> manageability
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> devices.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR,
>> the
>>>>>> Xen
>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> responsible for composing DataStore/Volume
>>>>>> methods
>>>>>>>>> to
>>>>>>>>>>>>>>> create any
>>>>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR. There
>>>>>>>> should
>>>>>>>>>>>> be no
>>>>>>>>>>>>>>>>>>>>>>> dependencies
>>>>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I
>> said
>>>>>>>>>>>> earlier,
>>>>>>>>>>>>>> such
>>>>>>>>>>>>>>>>>>>>>>> circular
>>>>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
>>>>>> unmaintainable
>>>>>>>>> mess.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page
>> with
>>>>>> you.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>>>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a
>> Storage
>>>>>> layer
>>>>>>>>> that
>>>>>>>>>>>>>>>> supports n
>>>>>>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors of
>> m
>>>>>>>>>>>> hypervisors.
>>>>>>>>>>>>>>>> Such
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
>>>>>> untestable
>>>>>>>>>>>>> beast.
>>>>>>>>>>>>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven to
>>>>>> evolve
>>>>>>>>> the
>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager
>> determining
>>>>>>>>> whether
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
>> StoragePoolType,
>>>>>> but
>>>>>>>>> an
>>>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
>> allocate
>>>>>>>>>>>> resources
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device
>> driver
>>>>>>>> needs
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> indicate
>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
>>>>>> Finally, I
>>>>>>>>>>>> think
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to elect
>> to
>>>>>>>> have
>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
>>>>>> capable
>>>>>>>>>>>>> managing
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
>>>>>>>>> unmanaged).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume should
>> be
>>>>>> done
>>>>>>>>> in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed by
>> the
>>>>>>>>> Storage
>>>>>>>>>>>>>>> layer to
>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
>>>>>> create a
>>>>>>>>>>>> file,
>>>>>>>>>>>>>> etc).
>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the
>> Xen
>>>>>>>>>>>>>> hypervisor, as
>>>>>>>>>>>>>>>>>>> such,
>>>>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation.
>> From
>>>>>> my
>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>> nothing
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
>>>>>> content.
>>>>>>>>> From
>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides
>> the
>>>>>> means
>>>>>>>>> to
>>>>>>>>>>>>>> query
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the content by
>>>>>> higher-
>>>>>>>>> level
>>>>>>>>>>>>>>> layers
>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
>>>>>>>>> composition
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on
>> the
>>>>>>>>> Volume
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> protocol
>>>>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
>>>>>>>>> implemented
>>>>>>>>>>>> in
>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>> patch,
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone.
>> In
>>>>>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it
>> to
>>>>>> the
>>>>>>>>>>>>>>> hypervisor.
>>>>>>>>>>>>>>>> Before
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if
>> we're
>>>>>>>>> dealing
>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>> (or
>>>>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so
>> -
>>>>>>>> send a
>>>>>>>>>>>>>> "Create
>>>>>>>>>>>>>>>>>>> SR"
>>>>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK,
>> we
>>>>>>>>> would
>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>> proceed
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same
>> and
>>>>>> we just
>>>>>>>>>>>> add
>>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called for
>> this
>>>>>>>>>>>> particular
>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer today
>> that
>>>>>>>>> outlines
>>>>>>>>>>>>> how
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will
>> probably
>>>>>> be
>>>>>>>> of
>>>>>>>>>>>> use
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> you, as
>>>>>>>>>>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you (you
>>>>>> should
>>>>>>>>> have
>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John
>> Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
>>>>>>>>> thoughts,
>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
>>>>>>>> solidfire,
>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect
>> them
>>>>>> in a
>>>>>>>>>>>>>>> coherent
>>>>>>>>>>>>>>>>>>> form
>>>>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
>>>>>> introduces
>>>>>>>>> the
>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
>>>>>> introduces
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages
>> the
>>>>>> plug-in
>>>>>>>>> was
>>>>>>>>>>>>>>>>>>> incomplete,
>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed
>> iSCSI
>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
>>>>>> spending on
>>>>>>>>>>>> this.
>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>> Do
>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
>>>>>> accomplish
>>>>>>>>>>>>> what
>>>>>>>>>>>>>>> you're
>>>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John
>> Burwell
>>>>>> <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
>>>>>> introduces
>>>>>>>>> the
>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
>>>>>> management
>>>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>>>> (i.e.
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator
>> want
>>>>>> it
>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should
>> end up
>>>>>> with
>>>>>>>>> a
>>>>>>>>>>>>>>> semantic
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
>>>>>> managed.
>>>>>>>>> I
>>>>>>>>>>>>> also
>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage
>> types
>>>>>> from
>>>>>>>>> the
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike
>> Tutkowski
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the
>> following
>>>>>>>>> questions
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
>>>>>> LUNs
>>>>>>>>> and
>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>> volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a
>> single
>>>>>> LUN
>>>>>>>>> (LUN
>>>>>>>>>>>>> 0).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
>>>>>> Volumes
>>>>>>>>> and
>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
>>>>>> management
>>>>>>>>>>>>> server or
>>>>>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in,
>> a
>>>>>> SAN
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>> (same
>>>>>>>>>>>>>>>> as a
>>>>>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server
>> (via
>>>>>> the
>>>>>>>>> plug-in)
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> first
>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
>>>>>> hypervisor.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire
>> plug-
>>>>>> in,
>>>>>>>>> but
>>>>>>>>>>>>>> still
>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do
>> this
>>>>>> already
>>>>>>>>>>>>> today
>>>>>>>>>>>>>>>> (prior
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN
>> volume
>>>>>> and -
>>>>>>>>> in
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> case -
>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
>>>>>> While
>>>>>>>>> you
>>>>>>>>>>>>> can do
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce
>> storage
>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
>>>>>> management
>>>>>>>>> server
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
>>>>>> SAN
>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the
>> plug-
>>>>>> in).
>>>>>>>>> There
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between
>> a
>>>>>> SAN
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>> and a
>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage
>> vendor
>>>>>> that
>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>> true
>>>>>>>>>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these
>> pieces
>>>>>> are
>>>>>>>>>>>> related
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure
>> that
>>>>>>>>> multiple
>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
>>>>>> storage."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need
>> to
>>>>>> map a
>>>>>>>>>>>>> single
>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We
>> cannot
>>>>>> have
>>>>>>>>>>>>> multiple
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
>>>>>>>>> guarantee
>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single
>> SAN
>>>>>>>>> volume
>>>>>>>>>>>>> house
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do
>> that
>>>>>> today
>>>>>>>>>>>>> without
>>>>>>>>>>>>>>> any of
>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM,
>> Mike
>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a
>> SAN
>>>>>>>>> volume
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
>>>>>> created."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM,
>> John
>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
>>>>>>>>> concerns me
>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into
>> storage
>>>>>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI
>> type
>>>>>> is
>>>>>>>>> today
>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>> (as
>>>>>>>>>>>>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when
>> you
>>>>>> have
>>>>>>>>> not
>>>>>>>>>>>>>>> PreSetup an
>>>>>>>>>>>>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>>>>>> volume
>>>>>>>>> from
>>>>>>>>>>>>>>> VMware, it
>>>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>>>>>> volume
>>>>>>>>> from
>>>>>>>>>>>>> KVM,
>>>>>>>>>>>>>>> it uses
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
>>>>>> thinking
>>>>>>>>>>>>> here
>>>>>>>>>>>>>>> was to
>>>>>>>>>>>>>>>>>>> make a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this
>> dynamic
>>>>>> ability
>>>>>>>>>>>>> Edison
>>>>>>>>>>>>>>> added
>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
>>>>>> more
>>>>>>>>>>>>> specificy,
>>>>>>>>>>>>>>> though:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
>>>>>> Mike
>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
>>>>>> shouldn't
>>>>>>>>> know
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> care
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able
>> to
>>>>>> answer
>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is
>> that
>>>>>> the
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so
>> that
>>>>>> the
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>> knows
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing
>> that
>>>>>>>>> constraint
>>>>>>>>>>>>>>> and just
>>>>>>>>>>>>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just
>> need a
>>>>>> way
>>>>>>>>> for
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement
>> and
>>>>>>>>> perform
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
>>>>>> John
>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM,
>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack
>> may
>>>>>> be
>>>>>>>>> able to
>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it
>> unmanaged by
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> (e.g. the
>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
>>>>>> operator has
>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
>>>>>> reasoning
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
>>>>>> creating a
>>>>>>>>> SAN
>>>>>>>>>>>>>>> volume via
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
>>>>>> hypervisor)
>>>>>>>>> would
>>>>>>>>>>>>>>> work unless
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form
>> of,
>>>>>> say,
>>>>>>>>> an
>>>>>>>>>>>>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that,
>> independent
>>>>>> of
>>>>>>>>>>>>>>> CloudStack, an
>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to
>> CloudStack,
>>>>>> and
>>>>>>>>>>>>> exposes it
>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that
>> LUN
>>>>>> over
>>>>>>>>> to
>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate
>> anymore
>>>>>> LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands
>> prior
>>>>>> to my
>>>>>>>>>>>>>>> changes, we
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not
>> have
>>>>>> the
>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
>>>>>> maybe
>>>>>>>>> have
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume
>> needs
>>>>>>>>> support
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
>>>>>> specifically
>>>>>>>>> to
>>>>>>>>>>>>> create
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
>>>>>> management
>>>>>>>>> plugin
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
>>>>>> determine
>>>>>>>>>>>>> available
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow
>> the
>>>>>>>>>>>>> interrogation of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
>>>>>> device
>>>>>>>>> itself
>>>>>>>>>>>>>>> shouldn't
>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
>>>>>> simply be
>>>>>>>>> able
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> answer
>>>>>>>>>>>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01
>> PM,
>>>>>> Mike
>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
>>>>>> SolidFire
>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once
>> this is
>>>>>>>>> done, a
>>>>>>>>>>>>> new
>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
>>>>>> Offering
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
>>>>>> Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> volume. This
>>>>>>>>>>>>>>>>>>> leads
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches
>> the
>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> volume to a
>>>>>>>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
>>>>>> framework
>>>>>>>>> calling
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in
>> my
>>>>>> case, a
>>>>>>>>>>>>> SAN).
>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
>>>>>> applicable
>>>>>>>>> data
>>>>>>>>>>>>>>> (like the
>>>>>>>>>>>>>>>>>>> IQN of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
>>>>>> invoked if
>>>>>>>>> the
>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
>>>>>> attach,
>>>>>>>>> the
>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach
>> disk)
>>>>>> and
>>>>>>>>> the
>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach
>> logic
>>>>>> is run
>>>>>>>>> and
>>>>>>>>>>>>>>> detects
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance"
>> in
>>>>>> the
>>>>>>>>> form
>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>> 12:54
>>>>>> PM,
>>>>>>>>> Mike
>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
>>>>>> sync on
>>>>>>>>>>>>>>> terminology,
>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
>>>>>> physical
>>>>>>>>>>>>>>> allocation on
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume',
>> I
>>>>>> try to
>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>> 'SAN
>>>>>>>>>>>>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume'
>> the
>>>>>> end
>>>>>>>>> user
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> make in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>> 12:50
>>>>>> PM,
>>>>>>>>> Mike
>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
>>>>>> very
>>>>>>>>> well
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>> sense, but
>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should
>> draw
>>>>>> Edison
>>>>>>>>> in on
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> conversation
>>>>>>>>>>>>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest
>> the
>>>>>>>>>>>>> approach I
>>>>>>>>>>>>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
>> 12:42
>>>>>> PM,
>>>>>>>>>>>>> John
>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
>>>>>> combining
>>>>>>>>> two
>>>>>>>>>>>>>>> distinct
>>>>>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and
>> storage
>>>>>>>>> protocols.
>>>>>>>>>>>>> In
>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>> cases, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows
>> the
>>>>>>>>> system to
>>>>>>>>>>>>>>>> create/delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
>>>>>> requires
>>>>>>>>> the
>>>>>>>>>>>>>>> volume to
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the
>> CloudStack
>>>>>>>>> context.
>>>>>>>>>>>>> To
>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume,
>> in
>>>>>> these
>>>>>>>>>>>>>>> definitions,
>>>>>>>>>>>>>>>>>>> refers to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the
>> device,
>>>>>>>>> correct?
>>>>>>>>>>>>>>> Minimally,
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
>>>>>> storage
>>>>>>>>> device
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> move bits
>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits,
>> etc.
>>>>>>>>>>>>> Optionally, a
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
>> CloudStack.
>>>>>>>>> Therefore,
>>>>>>>>>>>>>>> we can have
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a
>> Xen
>>>>>> SR,
>>>>>>>>> and
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> can have a
>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
>>>>>> CloudStack is
>>>>>>>>>>>>> capable
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> allocating
>>>>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally,
>> while
>>>>>>>>> CloudStack
>>>>>>>>>>>>> may
>>>>>>>>>>>>>>> be able
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
>>>>>> leave it
>>>>>>>>>>>>>>> unmanaged by
>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many
>> services,
>>>>>> and
>>>>>>>>> the
>>>>>>>>>>>>>>> operator has
>>>>>>>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
>>>>>> CloudStack).
>>>>>>>>> Does
>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
>>>>>> above
>>>>>>>>> are
>>>>>>>>>>>>>>> reasonable, it
>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
>>>>>>>>> concerns
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
>>>>>>>>> attached
>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
>> CloudStack,
>>>>>> and
>>>>>>>>>>>>> establish
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
>>>>>> separate
>>>>>>>>> that
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> associated
>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> drivers can
>>>>>>>>>>>>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when
>> they
>>>>>>>>> encounter
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
>> operations
>>>>>>>>> exposed
>>>>>>>>>>>>> by
>>>>>>>>>>>>>>> the driver to
>>>>>>>>>>>>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these
>> thoughts/concepts
>>>>>> make
>>>>>>>>>>>>> sense,
>>>>>>>>>>>>>>> then we can
>>>>>>>>>>>>>>>>>>> sit
>>>>>>>>>>>>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25
>> PM,
>>>>>>>>> Mike
>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the
>> difference
>>>>>>>>> between
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
>>>>>> has to
>>>>>>>>> go in
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
>>>>>> point in
>>>>>>>>> time,
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is
>> pre-
>>>>>>>>> allocated).
>>>>>>>>>>>>>>> Future
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR
>> that
>>>>>> was
>>>>>>>>> created
>>>>>>>>>>>>>>> behind the
>>>>>>>>>>>>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The
>> admin
>>>>>> has
>>>>>>>>> to
>>>>>>>>>>>>> go in
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will
>> create
>>>>>> and
>>>>>>>>> delete
>>>>>>>>>>>>>>> volumes on
>>>>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
>>>>>> enabled via
>>>>>>>>> the
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
>>>>>> volume
>>>>>>>>> that
>>>>>>>>>>>>> was
>>>>>>>>>>>>>>> created,
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a
>> new
>>>>>>>>> volume.
>>>>>>>>>>>>> After
>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>> done,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
>>>>>> question is
>>>>>>>>> called.
>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because
>> the
>>>>>>>>> volume
>>>>>>>>>>>>> was
>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> at
>>>>>> 3:21
>>>>>>>>> PM,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new
>> terms,
>>>>>> so I
>>>>>>>>> should
>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage:
>> Storage
>>>>>> that is
>>>>>>>>>>>>> pre-
>>>>>>>>>>>>>>> allocated
>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN),
>> then
>>>>>> a
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
>>>>>> XenServer
>>>>>>>>> SR),
>>>>>>>>>>>>> then
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
>>>>>> CloudStack.
>>>>>>>>> Disks
>>>>>>>>>>>>> (VDI)
>>>>>>>>>>>>>>> are later
>>>>>>>>>>>>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
>>>>>> needed. In
>>>>>>>>>>>>> these
>>>>>>>>>>>>>>> cases, the
>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>>>>>> structure is
>>>>>>>>>>>>> already
>>>>>>>>>>>>>>> in place
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor
>> data
>>>>>>>>> structure
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> the VM in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
>>>>>> Storage
>>>>>>>>> that is
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent
>> storage,
>>>>>> this
>>>>>>>>> could
>>>>>>>>>>>>> be a
>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>> (not
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
>>>>>> hypervisor
>>>>>>>>> data
>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>> must be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is
>> performed
>>>>>>>>> because
>>>>>>>>>>>>> these
>>>>>>>>>>>>>>> types of
>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a
>> hypervisor
>>>>>> data
>>>>>>>>>>>>>>> structure by an
>>>>>>>>>>>>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR
>> on
>>>>>>>>> XenServer
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the
>> SR
>>>>>> to the
>>>>>>>>> VM
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> at
>>>>>> 3:13
>>>>>>>>> PM,
>>>>>>>>>>>>>>> John Burwell
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>>>>>>>>> implementation of
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
>>>>>> which
>>>>>>>>> why I
>>>>>>>>>>>>> ask
>>>>>>>>>>>>>>> about the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
>>>>>> what is
>>>>>>>>> the
>>>>>>>>>>>>>>> definition
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>> 5:10
>>>>>> PM,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know,
>> the
>>>>>> iSCSI
>>>>>>>>> type
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> uniquely
>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
>>>>>> Primary
>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>> directly
>>>>>>>>>>>>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
>>>>>> skip the
>>>>>>>>>>>>> step of
>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> repository
>>>>>> based
>>>>>>>>> on
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>> target as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think
>> this is
>>>>>> only
>>>>>>>>>>>>>>> supported for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
>>>>>> must
>>>>>>>>> first go
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really
>> know
>>>>>> what
>>>>>>>>> RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013
>>>>>> at
>>>>>>>>> 2:13
>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading
>> through
>>>>>> the
>>>>>>>>> code,
>>>>>>>>>>>>> what
>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
>>>>>> Why
>>>>>>>>> isn't
>>>>>>>>>>>>> RBD
>>>>>>>>>>>>>>> considered
>>>>>>>>>>>>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
>> at
>>>>>> 3:46
>>>>>>>>> PM,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type
>> of
>>>>>>>>> storage is
>>>>>>>>>>>>>>> defined in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called
>> Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static
>> enum
>>>>>>>>>>>>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Filesystem(false),
>>>>>> //
>>>>>>>>> local
>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> NetworkFilesystem(true),
>>>>>>>>>>>>> //
>>>>>>>>>>>>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true),
>> //
>>>>>>>>> shared
>>>>>>>>>>>>> LUN,
>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), //
>> for
>>>>>> e.g.,
>>>>>>>>> ZFS
>>>>>>>>>>>>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), //
>> for
>>>>>> iso
>>>>>>>>> image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
>>>>>>>>> XenServer
>>>>>>>>>>>>> local
>>>>>>>>>>>>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
>>>>>>>>> VMware
>>>>>>>>>>>>> VMFS
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> PreSetup(true), //
>>>>>> for
>>>>>>>>>>>>>>> XenServer, Storage
>>>>>>>>>>>>>>>>>>> Pool
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
>>>>>>>>> XenServer
>>>>>>>>>>>>> local
>>>>>>>>>>>>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true);
>> //
>>>>>>>>> dynamic,
>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean
>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> StoragePoolType(boolean
>>>>>>>>>>>>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
>>>>>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
>>>>>>>>> isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>>>> 2013 at
>>>>>>>>> 1:41
>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
>>>>>> let's say
>>>>>>>>>>>>> another
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> company
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
>>>>>> leverage its
>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> Service
>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>> storage, as
>>>>>>>>> well.
>>>>>>>>>>>>>>> They would
>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
>>>>>> made
>>>>>>>>> the
>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>> changes to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support
>> their
>>>>>> plug-
>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
>> 3,
>>>>>> 2013
>>>>>>>>> at
>>>>>>>>>>>>> 1:39
>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
>>>>>> imply
>>>>>>>>> the
>>>>>>>>>>>>>>> XenServer code
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> XenServer
>>>>>>>>> attach
>>>>>>>>>>>>> logic is
>>>>>>>>>>>>>>> now aware
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and
>> SolidFire is
>>>>>> an
>>>>>>>>>>>>>>> implementation of
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is
>> new
>>>>>> to 4.2
>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison
>> created
>>>>>> a
>>>>>>>>> new
>>>>>>>>>>>>>>> framework that
>>>>>>>>>>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
>>>>>>>>> dynamically.
>>>>>>>>>>>>>>> However, when I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
>>>>>> realized
>>>>>>>>> that
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> was not
>>>>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
>>>>>> CloudStack
>>>>>>>>> could
>>>>>>>>>>>>> do
>>>>>>>>>>>>>>> with these
>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
>>>>>>>>> recognize
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> new type of
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> data
>>>>>>>>> structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun 3,
>>>>>> 2013
>>>>>>>>> at
>>>>>>>>>>>>> 1:28
>>>>>>>>>>>>>>> PM, John
>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> generally
>>>>>> odd
>>>>>>>>> to me
>>>>>>>>>>>>>>> that any
>>>>>>>>>>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> understand or
>>>>>> care
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>> details. I
>>>>>>>>>>>>>>>>>>> expect
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a
>> set
>>>>>> of
>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>> that can be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> implementations to
>>>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>>>> space/create
>>>>>>>>>>>>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> invert
>>>>>> this
>>>>>>>>>>>>>>> dependency, we are
>>>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem
>> that
>>>>>> will
>>>>>>>>> make
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance.
>> Am
>>>>>> I
>>>>>>>>>>>>>>> understanding that the
>>>>>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in
>> the
>>>>>>>>>>>>>>> CitrixResourceBase
>>>>>>>>>>>>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun 3,
>>>>>>>>> 2013 at
>>>>>>>>>>>>> 3:12
>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve
>> into
>>>>>> this
>>>>>>>>> in a
>>>>>>>>>>>>> bit
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to
>> 4.2
>>>>>> and
>>>>>>>>> aside
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>> one setup
>>>>>>>>>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first
>> create
>>>>>> a
>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up
>> a
>>>>>> data
>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>> to make
>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> repository on
>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>> or a
>>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
>> this
>>>>>>>>> storage
>>>>>>>>>>>>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>> Edison's
>>>>>> new
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>> framework,
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you
>> can
>>>>>> easily
>>>>>>>>>>>>> create
>>>>>>>>>>>>>>> a 1:1
>>>>>>>>>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> and
>>>>>> the
>>>>>>>>> VM's
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>> (necessary
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can
>> now
>>>>>> write
>>>>>>>>> a
>>>>>>>>>>>>> plug-
>>>>>>>>>>>>>>> in that is
>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>> as
>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> problem
>>>>>> that
>>>>>>>>> the
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting
>> the
>>>>>>>>>>>>> hypervisor-
>>>>>>>>>>>>>>> specific data
>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>> being
>>>>>> the
>>>>>>>>> case,
>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>> enhancing
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked
>> out
>>>>>> and
>>>>>>>>>>>>>>> submitted. I've got
>>>>>>>>>>>>>>>>>>> ESX(i)
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we
>> extend
>>>>>> the
>>>>>>>>> 4.2
>>>>>>>>>>>>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
>>>>>> help a
>>>>>>>>> bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun
>>>>>> 3,
>>>>>>>>> 2013 at
>>>>>>>>>>>>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> storage
>>>>>>>>> plug-in -
>>>>>>>>>>>>> by
>>>>>>>>>>>>>>> itself - is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> issue is
>>>>>> with
>>>>>>>>> the
>>>>>>>>>>>>>>> volume-attach
>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> framework
>>>>>> calls
>>>>>>>>> into
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> plug-in to
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>> the
>>>>>> time
>>>>>>>>>>>>> comes to
>>>>>>>>>>>>>>> attach the
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to
>> be
>>>>>> smart
>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>> to recognize
>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where
>> the
>>>>>>>>> volume
>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> just been
>>>>>>>>>>>>>>>>>>> created)
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> repository
>>>>>> (for
>>>>>>>>>>>>>>> XenServer) or a
>>>>>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> that
>>>>>> was
>>>>>>>>> just
>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>> been
>>>>>>>>> spending
>>>>>>>>>>>>> most
>>>>>>>>>>>>>>> of my time
>>>>>>>>>>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>> agent
>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does
>> that
>>>>>> clear it
>>>>>>>>> up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>>>> Jun 3,
>>>>>>>>> 2013
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> 12:48 PM,
>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> you
>>>>>> explain
>>>>>>>>>>>>> why
>>>>>>>>>>>>>>> the the storage
>>>>>>>>>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
>> 3,
>>>>>> 2013,
>>>>>>>>> at
>>>>>>>>>>>>> 1:21
>>>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
>>>>>>>>> ultimately I
>>>>>>>>>>>>>>> would like to
>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> supports.
>>>>>> I
>>>>>>>>> think
>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>> just out of
>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
>>>>>> now this
>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> regards
>>>>>> to 4.2
>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>> freeze, I
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
>> just
>>>>>> to
>>>>>>>>> be
>>>>>>>>>>>>> clear,
>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> support
>>>>>>>>> Compute
>>>>>>>>>>>>>>> Offerings post
>>>>>>>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Mon,
>>>>>> Jun 3,
>>>>>>>>>>>>> 2013
>>>>>>>>>>>>>>> at 11:14 AM,
>>>>>>>>>>>>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is
>> there
>>>>>> any
>>>>>>>>> plan
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> supporting
>>>>>>>>>>>>>>>>>>> KVM in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
>>>>>> Original
>>>>>>>>>>>>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> From:
>>>>>> "Mike
>>>>>>>>>>>>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>>>>>>>>>>>>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Sent:
>>>>>>>>> Monday,
>>>>>>>>>>>>> June
>>>>>>>>>>>>>>> 3, 2013
>>>>>>>>>>>>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Subject:
>>>>>> Re:
>>>>>>>>>>>>> [MERGE]
>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> agree
>>>>>> on
>>>>>>>>>>>>> merging
>>>>>>>>>>>>>>> Wei's feature
>>>>>>>>>>>>>>>>>>>>> first,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> his
>>>>>> feature
>>>>>>>>> is
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> KVM only,
>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> KVM in
>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Mon,
>>>>>> Jun
>>>>>>>>> 3,
>>>>>>>>>>>>> 2013
>>>>>>>>>>>>>>> at 8:55 AM,
>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> For
>>>>>> the
>>>>>>>>> billing,
>>>>>>>>>>>>> as
>>>>>>>>>>>>>>> no one works
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> the
>>>>>> billing
>>>>>>>>> by
>>>>>>>>>>>>>>> themselves. They
>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> disk_offering of
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> VMs and
>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> tell
>>>>>> user
>>>>>>>>> the
>>>>>>>>>>>>>>> exact
>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> rate
>>>>>>>>> limitation
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> nics as
>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
>>>>>> part of
>>>>>>>>> I/O
>>>>>>>>>>>>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Sorry
>>>>>> my
>>>>>>>>>>>>>>> misunstand the second
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Agree
>>>>>> with
>>>>>>>>>>>>> what
>>>>>>>>>>>>>>> you said about
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> 2013/6/3
>>>>>>>>> John
>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>> Jun 3,
>>>>>>>>> 2013,
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> 2:13 AM, Wei
>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Hi
>>>>>> John,
>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> I
>>>>>> hope
>>>>>>>>> Mike's
>>>>>>>>>>>>>>> aswer helps you.
>>>>>>>>>>>>>>>>>>> I am
>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (1) I
>>>>>>>>> think
>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>> depend
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> limitation.
>>>>>>>>>>>>> Please
>>>>>>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> get
>>>>>> the
>>>>>>>>> IO
>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> virtual
>>>>>>>>>>>>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Going by
>>>>>>>>> the
>>>>>>>>>>>>> AWS
>>>>>>>>>>>>>>> model,
>>>>>>>>>>>>>>>> customers
>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> provisioned
>>>>>>>>>>>>> IOPS,
>>>>>>>>>>>>>>> as well as,
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/).
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> option to
>>>>>>>>>>>>> employ
>>>>>>>>>>>>>>> similar cost
>>>>>>>>>>>>>>>>>>>>> models.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> billing
>>>>>>>>> model
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the current
>>>>>>>>>>>>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (2)
>>>>>> Do
>>>>>>>>> you
>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>> IOPS runtime
>>>>>>>>>>>>>>>>>>>>> change?
>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> limitation for
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>> virtual
>>>>>>>>>>>>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> CloudStack
>>>>>>>>>>>>> does
>>>>>>>>>>>>>>> not support
>>>>>>>>>>>>>>>>>>>>> changing
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> (computer
>>>>>>>>>>>>>>> offering or disk
>>>>>>>>>>>>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>> meant at
>>>>>>>>> the
>>>>>>>>>>>>>>> Java interface
>>>>>>>>>>>>>>>>>>> level.
>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> we
>>>>>> more
>>>>>>>>>>>>>>> generalize allocation
>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> describe
>>>>>>>>> the
>>>>>>>>>>>>>>> service guarantees
>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (3)
>>>>>> It is a
>>>>>>>>>>>>> good
>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>> Maybe it
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> disk_io_throttling as Mike
>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> type, I
>>>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> will
>>>>>>>>> expand
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> my thoughts
>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> touch
>>>>>>>>> points
>>>>>>>>>>>>>>> between these two
>>>>>>>>>>>>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> will
>>>>>> need
>>>>>>>>> to be
>>>>>>>>>>>>>>> merged before
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> coordination
>>>>>>>>>>>>>>> between the
>>>>>>>>>>>>>>>> branches
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> disk_io_throttling)
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> -
>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> 2013/6/3
>>>>>>>>>>>>> John
>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> The
>>>>>>>>> things I
>>>>>>>>>>>>>>> want to
>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 1)
>>>>>> Is
>>>>>>>>> there
>>>>>>>>>>>>>>> value in
>>>>>>>>>>>>>>>> capturing
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> data
>>>>>>>>> model
>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 2)
>>>>>>>>> Should
>>>>>>>>>>>>> there
>>>>>>>>>>>>>>> be a common
>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> provisioning
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 3)
>>>>>> How
>>>>>>>>> are
>>>>>>>>>>>>>>> conflicting
>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> hypervisor
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> storage device
>>>>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> is
>>>>>> lead
>>>>>>>>> to
>>>>>>>>>>>>>>> believe (and
>>>>>>>>>>>>>>>> billed)
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> storage
>>>>>>>>>>>>> device
>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> could
>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> consistent
>>>>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>>>>> between a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> time,
>>>>>>>>> but a
>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>> modification
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> -
>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>> Jun
>>>>>>>>> 2,
>>>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>> at 8:38 PM,
>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Hi
>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> I
>>>>>>>>> believe
>>>>>>>>>>>>> Wei's
>>>>>>>>>>>>>>> feature deals
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> the
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> My
>>>>>>>>> feature
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> focused on
>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> I
>>>>>> hope
>>>>>>>>> that
>>>>>>>>>>>>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>> Sun,
>>>>>>>>> Jun
>>>>>>>>>>>>> 2,
>>>>>>>>>>>>>>> 2013 at 6:35
>>>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> My
>>>>>>>>>>>>> opinion is
>>>>>>>>>>>>>>> that no
>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> issues
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> been resolved
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> total
>>>>>>>>> Ops
>>>>>>>>>>>>> vs
>>>>>>>>>>>>>>> discrete
>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
>>>>>>>>> reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido,
>>>>>>>>> I
>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>> think this
>>>>>>>>>>>>>>>>>>> criteria
>>>>>>>>>>>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Also,
>>>>>>>>> how
>>>>>>>>>>>>>>> does this work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> also
>>>>>>>>>>>>> involves
>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> scenario
>>>>>>>>>>>>>>> where provisioned
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> -
>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> On
>>>>>>>>> Jun 1,
>>>>>>>>>>>>> 2013,
>>>>>>>>>>>>>>> at 6:47 AM,
>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Sure. I
>>>>>>>>> will
>>>>>>>>>>>>>>> change it next
>>>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> -
>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> 2013/6/1
>>>>>>>>>>>>>>> Wido den Hollander
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> On
>>>>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> Exactly. I
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> pushed the
>>>>>>>>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> If
>>>>>>>>> anyone
>>>>>>>>>>>>>>> object thems for
>>>>>>>>>>>>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> For
>>>>>>>>> the
>>>>>>>>>>>>> sake
>>>>>>>>>>>>>>> of clarity I
>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> the
>>>>>>>>> total
>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>> to R/W IOps
>>>>>>>>>>>>>>>>>>> asap
>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> only
>>>>>>>>> total
>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> You
>>>>>>>>> laid
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> groundwork for
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> however
>>>>>>>>>>>>>>> prevent that we
>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> -
>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> 2013/5/31
>>>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> On
>>>>>>>>>>>>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> +1 --
>>>>>>>>> this
>>>>>>>>>>>>>>> enhancement must
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> don't
>>>>>>>>> see
>>>>>>>>>>>>> how
>>>>>>>>>>>>>>> it could be
>>>>>>>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> split
>>>>>>>>> total
>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>> into read
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> total
>>>>>>>>>>>>>>> unless/until we
>>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> What
>>>>>>>>> Wei
>>>>>>>>>>>>>>> meant was merging
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> 4.2
>>>>>>>>> branch
>>>>>>>>>>>>>>> and add Read /
>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> will be
>>>>>>>>>>>>>>> released with Read
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> This is
>>>>>>>>> to
>>>>>>>>>>>>>>> make the May 31st
>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> (see
>>>>>>>>> other
>>>>>>>>>>>>>>> threads) then it
>>>>>>>>>>>>>>>>>>> won't
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> I
>>>>>> also
>>>>>>>>>>>>>>> completely agree that
>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> disk
>>>>>>>>> I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> -
>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> On
>>>>>>>>> May 31,
>>>>>>>>>>>>>>> 2013, at 9:51 AM,
>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> On
>>>>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Hi
>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> Thanks.
>>>>>>>>>>>>> Good
>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> I
>>>>>>>>> thought
>>>>>>>>>>>>>>> about at the
>>>>>>>>>>>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> difference
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> read and write
>>>>>>>>>>>>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> care
>>>>>>>>> the
>>>>>>>>>>>>>>> difference of sent
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> That
>>>>>>>>>>>>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Disk
>>>>>>>>> I/O is
>>>>>>>>>>>>>>> much more
>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> Implementing
>>>>>>>>>>>>>>> it will be some
>>>>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> few
>>>>>>>>> days.
>>>>>>>>>>>>> For
>>>>>>>>>>>>>>> the deadline
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> that ,
>>>>>>>>> if
>>>>>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> It
>>>>>>>>> think it's
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> feature we
>>>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> window
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> have to make sure
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> it
>>>>>> in
>>>>>>>>> 4.3,
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> will confuse
>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> -
>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> 2013/5/31
>>>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> On
>>>>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> I
>>>>>>>>> would
>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> If
>>>>>>>>> nobody
>>>>>>>>>>>>>>> object, I will
>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> The
>>>>>>>>>>>>> purpose
>>>>>>>>>>>>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Virtual
>>>>>>>>>>>>>>> machines are running
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> share
>>>>>>>>>>>>> strage).
>>>>>>>>>>>>>>> Because of
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> iops),
>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> one
>>>>>>>>> VM
>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> large disk
>>>>>>>>>>>>>>>>>>> operation,
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> other
>>>>>>>>> VMs
>>>>>>>>>>>>>>> running on the
>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> It
>>>>>> is
>>>>>>>>>>>>> neccesary
>>>>>>>>>>>>>>> to set the
>>>>>>>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> Looking at
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> code I see
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Qemu
>>>>>>>>> and
>>>>>>>>>>>>>>> libvirt support
>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> IOps
>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> could benefit a
>>>>>>>>>>>>>>>>>>> lot of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> It's
>>>>>>>>> also
>>>>>>>>>>>>>>> strange, in the
>>>>>>>>>>>>>>>>>>> polling
>>>>>>>>>>>>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> IOps,
>>>>>>>>> but
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> the throttling
>>>>>>>>>>>>>>>>>>> side
>>>>>>>>>>>>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Write
>>>>>>>>> IOps
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>> usually much
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> like a
>>>>>>>>> valid
>>>>>>>>>>>>>>> use-case where
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> IOps
>>>>>>>>> vs
>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Since
>>>>>>>>> this
>>>>>>>>>>>>> only
>>>>>>>>>>>>>>> supports KVM
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> value
>>>>>>>>> to at
>>>>>>>>>>>>>>> least have the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> this
>>>>>>>>> later
>>>>>>>>>>>>>>> would be a lot of
>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> If
>>>>>> a
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> write
>>>>>>>>> you
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> always sum
>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Can
>>>>>>>>> you
>>>>>>>>>>>>>>> explain why you
>>>>>>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> The
>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> (1) set
>>>>>>>>> the
>>>>>>>>>>>>>>> maximum rate of
>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> (2)
>>>>>>>>> change
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> maximum rate
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> (3)
>>>>>>>>> limit
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> disk rate
>>>>>>>>>>>>>>>> (total
>>>>>>>>>>>>>>>>>>>>> bps
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> JIRA
>>>>>>>>> ticket:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-
>> 1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>> 1192
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> FS (I
>>>>>>>>> will
>>>>>>>>>>>>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> Merge
>>>>>>>>>>>>> check
>>>>>>>>>>>>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *
>>>>>> Did
>>>>>>>>> you
>>>>>>>>>>>>>>> check the branch's
>>>>>>>>>>>>>>>>>>> RAT
>>>>>>>>>>>>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *
>>>>>> Are
>>>>>>>>>>>>> there
>>>>>>>>>>>>>>> new dependencies
>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *
>>>>>>>>> What
>>>>>>>>>>>>>>> automated testing
>>>>>>>>>>>>>>>> (unit
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Unit
>>>>>>>>> tests
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *
>>>>>>>>> What
>>>>>>>>>>>>>>> testing has been done
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> (1) set
>>>>>>>>> the
>>>>>>>>>>>>>>> bytes rate and
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> (2)
>>>>>>>>> VM
>>>>>>>>>>>>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>> deploy,
>>>>>>>>>>>>> stop,
>>>>>>>>>>>>>>> start, reboot,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> (3)
>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>> operations,
>>>>>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Attach,
>>>>>>>>>>>>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> To
>>>>>>>>> review
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> code, you can
>>>>>>>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Best
>>>>>>>>>>>>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> [2]
>>>>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> -
>>>>>>>>> VM
>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *Mike
>>>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *Senior
>>>>>>>>>>>>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> e:
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> o:
>>>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> Advancing
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> way the world
>>>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Mike
>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Senior
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Advancing
>>>>>>>>> the
>>>>>>>>>>>>> way
>>>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Mike
>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Senior
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> Advancing the
>>>>>>>>>>>>> way
>>>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Advancing
>>>>>> the
>>>>>>>>> way
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Advancing
>>>>>> the
>>>>>>>>> way
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>> CloudStack
>>>>>>>>>>>>>>> Developer, SolidFire
>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> the
>>>>>> way
>>>>>>>>> the
>>>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> the
>>>>>> way
>>>>>>>>> the
>>>>>>>>>>>>> world
>>>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>>>> way the
>>>>>>>>>>>>> world
>>>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way
>>>>>> the
>>>>>>>>> world
>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>>>>>>> world
>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>>>> world
>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>>>> world
>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>>>> uses
>>>>>>>>> the
>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>>>> uses
>>>>>>>>> the
>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>>>> uses
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>>>> the
>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *(tm)*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *(tm)*****
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*****
>>>>>>>>
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>>>
>>>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>>>
>>>>>>>> o: 303.746.7302****
>>>>>>>>
>>>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *(tm)*****
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*****
>>>>>>>>
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>>>
>>>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>>>
>>>>>>>> o: 303.746.7302****
>>>>>>>>
>>>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *(tm)*****
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *(tm)*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *(tm)*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *(tm)*
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Right...all of that makes sense to me, Edison.
I think John's concern with my patch file (which implements the approach
you outlined in your last paragraph) is that he didn't like the "attach"
command performing logic to create an SR if the SR isn't present.
He wanted a new hypervisor command to create the SR and he wanted this
invoked from the CS MS if the driver (or it could be the lifecycle object)
says it's managed.
We have at least one issue to deal with:
1) Will we allow hypervisor calls in the storage plug-in code?
If 'no', then the default storage plug-in is in violation of this
restriction.
This also means the "attach" hypervisor command needs to create the SR, if
need be (the way my patch is coded up already) or the management server has
to send a command to the hypervisor to create the SR if the driver needs it
(and the driver has to have some way to tell the CS MS to do this).
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: John Burwell [mailto:jburwell@basho.com]
> Sent: Monday, June 10, 2013 1:25 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Mike,
>
> Yes, a vendor may hand a non-managed plugin. For example, on the
> secondary storage side, S3 and Swift don't support the concept of being
> managed -- an operator is required to create a bucket and point CloudStack
> at it. Another example would be a generic iSCSI device (e.g. OpenFiler,
> FreeNAS) which CloudStack can not directly manage, but can consume a LUN
> that an operator has created for it.
Manage or non-managed storage is handled by DataStoreLifeCyclce.
It depends on storage vendor to implement DataStoreLifeCyclce, which basically says how to initialize a storage, how to add a storage into a cluster etc.
Take S3 as an example, if the S3 storage provider provides a DataStoreLifeCyclce which can create a bucket, then when admin adding a S3 storage through API, mgt server will first call DataStoreLifeCyclce, which will create a bucket, then stored in the image_store_details table, then later on, all the referenced to this image store, will contain this bucket information.
Take an storage box from some vendor as an example, admin wants to create an ISCSI server on a storage box, which can be used by a cluster(multiple volumes on a giant LUN ), as long as the storage provider can provide a DataStoreLifeCyclce which can program the storage box, and create an ISCSI server, then stored the information in storage_pool table, then all the referenced to the storage pool will use the dynamically created ISCSI LUN.
Take another storage box from some vendor as an example, admin wants to add the whole storage box into cloudstack, which can be used as per LUN per ISCSI, so in this case, DataStoreLifeCyclce won't create an ISCSI server during the initialize method call. Instead, for every createAsync, it will create an ISCSI LUN, stored the LUN information in volume table and volume_detail table. Whenever the LUN is used by hypervisor, the hypervisor needs to take extra care about the volume.
>
> From an operational perspective, an administrator may elect to say, "Hey, I
> know you can manage that device for me, but it is has a wider use than
> CloudStack can manage. Let me handle it, and just provide the information
> CloudStack needs." In this scenario, the operator is accepting a greater
> management burden in exchange for control. I don't think many will elect to
> manage these devices themselves, but this ability will be extremely
> important for the edge cases that require it. Speaking as someone who has
> run cloud operations, I can testify to this need -- nothing aggravates you
> more than a system insisting on managing a device or service without the
> opportunity to opt-out
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 3:19 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > So, would a storage vendor write a non-CloudStack-managed plug-in and
> maybe
> > just implement the snapshot feature or something?
> >
> > In this case, they'd have to set the hypervisor up ahead of time with a
> > large portion of their storage, but they'd still get the vendor-snapshot
> > ability.
> >
> > Just trying to undertand this more.
> >
> > Thanks
> >
> >
> > On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Edison,
> >>
> >> TL;DR Always give operators/system administrators the ability to manage
> >> things themselves, if they like.
> >>
> >> As more manageable drivers emerge, I think it will be import to allow
> >> operators to opt-out of CloudStack device management. Therefore, a
> driver
> >> declares whether or not it can support device management (many, in fact
> >> most, won't). For those drivers that do support management, allow
> >> operators to opt-in/out of CloudStack management on a per device basis.
> To
> >> support this notion, we need a flag on the storage_pool table and
> >> associated classes to indicate whether or not the operator wants
> CloudStack
> >> to manage the device.
> >>
> >> Does this explanation make more sense?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
> >>
> >>> I don't understand why we need to add a "managed" in the UI and in
> >> "storage_pool" table?
> >>> AFAIK, what you guys trying to do is to enable
> >> attachVolume/DettachVolume if the volume is created on Solidfire, right?
> >>> If the volume is created on Solidfire already, by createAsync, then why
> >> we can't just send out all the information about the volume(it's volumeTO
> >> which will be returned by each storage driver) to hypervisor, then at the
> >> hypervisor side, check the volumeTO, if the volumeTO is managed, then
> doing
> >> whatever necessary operations(create SR, create VDI etc), then attach
> the
> >> VDI to volume.
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>> Sent: Monday, June 10, 2013 11:10 AM
> >>>> To: John Burwell
> >>>> Cc: Edison Su; dev@cloudstack.apache.org
> >>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>
> >>>> I believe we're on the same page, John.
> >>>>
> >>>> I would add a new parameter to the createStoragePool API command
> (let's
> >>>> call it "managed" and it can be true or false). If you pass in "true",
> >> but
> >>>> the driver does not support being managed by CloudStack, an
> exception is
> >>>> thrown by the driver. If you pass in "false", but the driver does not
> >>>> support being managed outside of CloudStack, an exception is thrown
> by
> >> the
> >>>> driver. It is possible one could write a driver that can be managed both
> >>>> inside or outside of CloudStack (the SolidFire driver requires being
> >>>> managed only by CloudStack).
> >>>>
> >>>> This managed value would end up in the storage_pool table in a new
> >> column
> >>>> ("managed").
> >>>>
> >>>> An example usage: After the storage framework calls on the driver's
> >>>> createAsync method, it will check to see if the driver is managed. If
> >> the
> >>>> driver is managed, the storage framework will invoke the proper (new)
> >>>> command of the hypervisor in use to set up, say, the SR for the SAN
> >> volume.
> >>>>
> >>>> There may be other cases when the isManaged() method is invoked on
> the
> >>>> driver and - if true is returned - the storage framework sends a
> >> particular
> >>>> command to the hypervisor in use.
> >>>>
> >>>> ********************
> >>>>
> >>>> As far as reworking the default storage plug-in (which sends commands
> to
> >>>> hypervisors) for 4.2, I would check with Edison on his schedule.
> >>>>
> >>>> Thanks!
> >>>>
> >>>>
> >>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> I want to make sure we are on the same page regarding isManaged. I
> see
> >>>>> the addition of the following operations:
> >>>>>
> >>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates
> whether
> >>>> or
> >>>>> not the driver support/implements the management functions
> >>>>> * DataStore#isManaged() : boolean -- indicates whether or not a
> device
> >> is
> >>>>> managed by CloudStack
> >>>>>
> >>>>> Only DataStores associated with DataStoreDrivers that support
> >>>> management
> >>>>> would be able to have the management flag enabled. This behavior
> >> allows
> >>>>> operators to declare their intention to allow CloudStack to manage a
> >>>>> device. This behavior would need to be exposed as flag on the HTTP
> >> API.
> >>>>> Additionally, this flag would only be settable when a device is
> >>>>> defined/created in the system (i.e. once a device is managed by
> >> CloudStack,
> >>>>> you can't unmanaged it without dissociating it from CloudStack). In
> >> the
> >>>>> future, we may consider relaxing that rule, but, in the short term, it
> >>>>> greatly decreases the alternative flows.
> >>>>>
> >>>>> In terms of removing the dependencies from Storage to Hypervisor, I
> >>>> would
> >>>>> really like to see this accomplished in 4.2. It will be much more
> >>>>> difficult to address this issue post 4.2 because we will have more
> >> plugins
> >>>>> to address. I also see this issue as a critical architectural issues
> >> that
> >>>>> for the management server. I recommend we identify the
> dependencies
> >>>> that
> >>>>> need to be inverted, and see if we can divide the work across a few
> >> more
> >>>>> resources.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> >>>> <mi...@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>> Sounds good, Edison :)
> >>>>>
> >>>>> I can implement John's isManaged() logic and resubmit my code for
> >> review.
> >>>>>
> >>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
> >> requires
> >>>>> some refactoring of the storage framework, then I could do that.
> >>>>>
> >>>>> Also, the default storage plug-in does have hypervisor logic in it in
> >> 4.2.
> >>>>> I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> >>>>> with the above work I mentioned.
> >>>>>
> >>>>> Are you OK with that, guys?
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
> >> wrote:
> >>>>>
> >>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
> >>>>>> *To:* dev@cloudstack.apache.org
> >>>>>> *Cc:* Edison Su; John Burwell
> >>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> Another thing we might want to consider for 4.2 is that the way the
> >> code
> >>>>>> is currently implemented in master, I don't think it's even possible
> >> to
> >>>>>> write a plug-in that connects storage directly to a VM because the
> >>>>>> CloudStack storage framework will call the "attach" method of the
> >>>>>> hypervisor in use and this method will fail because it assumes there
> >> is
> >>>>>> (talking Xen here) an SR existent and in this situation there isn't
> >> one.*
> >>>>>> ***
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> My thinking is that we should ignore the direct-attach-of-volume-to-
> VM
> >>>>>> use case for 4.2. I think it will require too much of a rewrite at
> >> this
> >>>>>> point in development (fewer than three weeks left until code
> >>>> freeze).****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> If the plug-in implements an isManaged() method (as John
> suggested),
> >>>> the
> >>>>>> storage framework can invoke the necessary commands on the
> >>>> hypervisor in
> >>>>>> use.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> For example, the createAsync method is called. The storage plug-in
> >>>>>> creates a SAN volume and updates the CS DB. The storage
> framework
> >>>> then asks
> >>>>>> the driver if it's managed. If it is, the storage framework executes a
> >>>>>> command against the hypervisor in use to have it create, say, an SR
> >> (with
> >>>> a
> >>>>>> single VDI that takes up the entire SR). Now, when the storage
> >>>> framework
> >>>>>> executes the "attach" command against the hypervisor, this
> command
> >> will
> >>>>>> work, as well (because that logic assumes the prior existence of an
> >> SR and
> >>>>>> it will, in fact, be present).****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:****
> >>>>>>
> >>>>>> Hi Edison and John,****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> I wanted to point out something I thought was relevant to this
> >>>>>> conversation.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> My plug-in supports both XenServer and ESX(i).****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> These two hypervisors handle their respective hypervisor data
> >> structures
> >>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> In XenServer, the SR that resides on the SAN LUN is simply
> "forgotten"
> >>>>>> when you issue a remove-SR command to XenServer.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> When you later want to access the contents of that data, it is still
> >>>>>> safely stored on the LUN and can be accessed from XenServer (from
> the
> >>>> same
> >>>>>> or different cluster (what XenServer calls a resource pool)) by
> >> creating a
> >>>>>> new SR that is based on the same IQN and LUN.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> XenServer SRs exist at the cluster level.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
> >>>>>> destroyed when you issue a remove-datastore command to ESX(i).
> In
> >>>> other
> >>>>>> words, the contents of the SAN LUN are destroyed.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> ESX(i) datastores exist at the datacenter level.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> The XenServer behavior (not destroying the contents of the LUN
> when
> >>>> the
> >>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
> >> contents
> >>>>>> of the LUN when the datastore is removed) is not desirable.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> I can get around the ESX(i) behavior by removing the referenced IQN
> >>>> from
> >>>>>> each host in the cluster when the CS storage framework notifies my
> >> plug-
> >>>> in
> >>>>>> of a detach-volume event. When the CS volume is later attached
> again
> >>>>>> (either to the same or a different ESX(i) cluster), I can have each
> >> host in
> >>>>>> the relevant ESX(i) cluster reference the IQN and the datastore will
> >> be
> >>>>>> back again.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> This sequence of events is critical for my plug-in to support both
> >>>>>> environments:****
> >>>>>>
> >>>>>> *XenServer flow (events for Driver class):*****
> >>>>>>
> >>>>>> createAsync (called right before first attach) = creates SAN
> >> volume****
> >>>>>>
> >>>>>> preAttachVolume = creates SR and creates VDI or just introduces
> SR****
> >>>>>>
> >>>>>> (CS volume gets attached and later detached)****
> >>>>>>
> >>>>>> postDetachVolume = removes SR (this does not delete data on SAN
> >>>> volume)**
> >>>>>> **
> >>>>>>
> >>>>>> deleteAsync (can be called when volume is not attached) = deletes
> the
> >>>> SAN
> >>>>>> volume****
> >>>>>>
> >>>>>> *VMware flow (events for Driver class):*****
> >>>>>>
> >>>>>> createAsync (called right before first attach) = creates SAN volume,
> >>>>>> creates datastore, and creates VMDK file****
> >>>>>>
> >>>>>> preAttachVolume = adds iSCSI target to each host in cluster****
> >>>>>>
> >>>>>> (CS volume gets attached and later detached)****
> >>>>>>
> >>>>>> postDetachVolume = removes iSCSI target from each host in
> cluster****
> >>>>>>
> >>>>>> deleteAsync (can be called when volume is not attached) = deletes
> >>>>>> datastore and deletes the SAN volume****
> >>>>>>
> >>>>>> ************************
> >>>>>>
> >>>>>> So, the way I have it written up here, the driver is talking to the
> >>>>>> hypervisor. As John has pointed out, this may not be ideal.****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> Whatever we decide on, it needs to be able to support this kind of
> >> flow.
> >>>>>> :)****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> We could go John's route, where the driver says if it's managed or
> >> not.**
> >>>>>> **
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> If it is, then the storage framework could send the necessary
> >> messages to
> >>>>>> the hypervisor in question at the right times (just migrate the
> >> hypervisor
> >>>>>> logic above from the driver to the right parts of the storage
> >> framework).
> >>>>>> ****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> This is fine, except that it does not resolve Edison's use case of
> >>>>>> desiring to have the storage LUN potentially connected directly to a
> >> VM.*
> >>>>>> ***
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> I'm not sure we should design around that use case, though. Is it
> >> common
> >>>>>> enough? Has anyone asked for it? Is it even desirable? How are
> usage
> >>>>>> statistics gathered for such volumes? It's nice to offer the
> >> flexibility,
> >>>>>> but is it a good idea in this case?****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> Thanks!****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> >> wrote:**
> >>>>>> **
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>> Sent: Friday, June 07, 2013 4:08 PM
> >>>>>>> To: dev@cloudstack.apache.org
> >>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>
> >>>>>>> Hi John,
> >>>>>>>
> >>>>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
> >>>>>> volume
> >>>>>>> to use it. Same story for VMware: ESX(i) must layer a datastore on
> >> top
> >>>>>> of
> >>>>>>> the iSCSI volume to use it. I look at it like they are layering a
> >>>>>> clustered
> >>>>>>> file system on the SAN volume so the hypervisors can share access
> to
> >>>> the
> >>>>>>> contents of the volume.
> >>>>>>>
> >>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called, the
> >>>>>> SR or
> >>>>>>> datastore has already been created (usually manually by an admin).
> >>>>>>>
> >>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
> >>>>>>> environment where each CloudStack volume that a user creates is
> >>>> mapped
> >>>>>>> to a
> >>>>>>> single SAN volume (via the SR).
> >>>>>>>
> >>>>>>> The question comes down to who should allocate the SR
> dynamically.
> >>>>>>>
> >>>>>>> We could have the storage framework ask the storage plug-in if it is
> >>>>>>> managed. If it is, then the storage framework could send a
> message to
> >>>>>> the
> >>>>>>> hypervisor in question to create (let's talk Xen here) the SR ahead
> >> of
> >>>>>>> time. Then, when the storage framework next sends the attach-
> volume
> >>>>>>> command
> >>>>>>> to the hypervisor, it should work without changes to that
> >> attach-volume
> >>>>>>> logic (because - from the point of view of the attach logic - the SR
> >> is
> >>>>>>> already existent, as expected).
> >>>>>>>
> >>>>>>> Now, as Edison has pointed out, this limits the power of the storage
> >>>>>>> plug-in. A storage plug-in in this model cannot directly attach
> >> storage
> >>>>>> to
> >>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We
> need
> >>>> to
> >>>>>>> make a call on that.
> >>>>>>
> >>>>>>
> >>>>>> Yes, I think we need to address this usage case, and won't limit the
> >>>>>> power of the storage plug-in.
> >>>>>> John, what's your idea to fit this usage case into your abstraction?
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> If we want to give more power to the storage vendor, we would
> have to
> >>>>>>> have
> >>>>>>> the storage framework call into the storage plug-in to send the
> >>>>>> appropriate
> >>>>>>> attach commands to the hypervisor.
> >>>>>>>
> >>>>>>> Let's discuss and come to a consensus (or at least agree on a path)
> >> as
> >>>>>> soon
> >>>>>>> as we can. :)
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> My understanding is the Xen expects an SR structure on all ISCSI
> >>>>>> devices
> >>>>>>>> -- at least that is how I read the code in your patch. Is my
> >>>>>> understanding
> >>>>>>>> correct? If so, the Xen plugin should be able to query the storage
> >>>>>> device
> >>>>>>>> to determine the presence of the SR structure and create it if it
> >>>>>> does not
> >>>>>>>> exist. Am I missing something in the implementation that makes
> that
> >>>>>> type
> >>>>>>>> of implementation impossible?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Yeah, if a storage vendor wanted to attach a volume directly to a
> >>>>>> VM in
> >>>>>>>> 4.2
> >>>>>>>>> today, it would probably fail because the attach-volume logic
> >>>>>> assumes
> >>>>>>> the
> >>>>>>>>> existence of the necessary hypervisor data structure (ex. SR on
> >>>>>> Xen).
> >>>>>>>>>
> >>>>>>>>> If we wanted to enable such an attach, we could do it the way
> >>>> Edison
> >>>>>>>>> suggests.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> As we only have three weeks until feature freeze, we should
> >>>> come
> >>>>>> to
> >>>>>>> a
> >>>>>>>>>>> consensus on this design point as soon as possible.
> >>>>>>>>>>>
> >>>>>>>>>>> Right now, if the storage framework asks my driver it is is
> >>>>>> managed, it
> >>>>>>>>>>> will say 'yes.' This means the framework will tell the driver to
> >>>>>>>> perform
> >>>>>>>>>>> its management activities. This then means the driver will call
> >>>>>> into
> >>>>>>>> the
> >>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to
> perform
> >>>> the
> >>>>>>>>>> activity
> >>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
> >>>>>>>>>>>
> >>>>>>>>>>> The driver doesn't know which hypervisor it's talking to, it just
> >>>>>>>> sends a
> >>>>>>>>>>> message to the host to perform the necessary pre-attach
> work.
> >>>>>>>>>>
> >>>>>>>>>> Could we just expose a method like
> >>>> "attachVolume/dettachVolume" on
> >>>>>>> the
> >>>>>>>>>> PrimaryDataStoreDriver driver?
> >>>>>>>>>> For most of cases, the implementation of each driver would
> just
> >>>>>> send
> >>>>>>>>>> attachvolumecommand/dettachvolumecmd to hypervisor(we
> can
> >>>> put
> >>>>>>> the
> >>>>>>>>>> implementation in a base class, so that can be shared by all of
> >>>>>> this
> >>>>>>>>>> drivers), and in each hypervisor resource code, which may just
> >> call
> >>>>>>>>>> hypervisor's api to attach the volume to VM, while for certain
> >>>>>> storage,
> >>>>>>>>>> like SolidFire, may need to create a SR first, and create volume
> >>>>>> on it,
> >>>>>>>>>> then call hypervisor's API to attach volume to VM.
> >>>>>>>>>> While for some other storage vendor, may want to bypass
> >>>> hypervisor
> >>>>>>>> during
> >>>>>>>>>> attaching volume, so inside driver's attachvolume
> implementation,
> >>>>>> the
> >>>>>>>>>> driver can do some magic, such as, directly talk to an agent
> >>>>>> inside VM
> >>>>>>>>>> instance, then create a disk inside VM.
> >>>>>>>>>>
> >>>>>>>>>> How do you guys think?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su
> <Ed...@citrix.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>> From: Mike Tutkowski
> [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> How's about this:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The driver can implement an isManaged() method. The
> >>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
> >>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>> (which is responsible for calling into the hypervisor to
> attach
> >>>>>> the
> >>>>>>>>>> disk)
> >>>>>>>>>>>>> can call into the hypervisor to create the necessary
> hypervisor
> >>>>>> data
> >>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
> >>>>>>>>>>>>
> >>>>>>>>>>>> The problem here is that storage vendor may work
> differently
> >>>> with
> >>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> >>>>>> maybe
> >>>>>>>>>> other
> >>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the
> LUN
> >>>>>>> directly
> >>>>>>>>>> to
> >>>>>>>>>>>> VM instance, see the discuss(
> >>>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>>>>>>>>>>
> >>>>>>>
> >>>>
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>>>>>>>>>> %3E
> >>>>>>>>>>>> ).
> >>>>>>>>>>>> So I would let storage provider to implement attach disk to
> VM,
> >>>>>>>>>> instead of
> >>>>>>>>>>>> implemented by cloudstack itself.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If that's what you're going for, that works for me. By the
> way,
> >>>>>>>>>> Edison's
> >>>>>>>>>>>>> default storage plug-in (which handles the default storage
> >>>>>> behavior
> >>>>>>>>>> in
> >>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> >>>>>> talks to
> >>>>>>>>>>>>> hypervisors. You might want to contact him and inform him
> of
> >>>>>> your
> >>>>>>>>>>>> concerns
> >>>>>>>>>>>>> or that logic (as is) will make it to production.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Please let me know if what I wrote in above (for my
> solution)
> >>>>>> is OK
> >>>>>>>>>> with
> >>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please see my responses in-line below.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I build
> >>>>>> up
> >>>>>>>>>> a
> >>>>>>>>>>>> patch
> >>>>>>>>>>>>>>> file, but this is the gist of what I've done:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * During a volume-attach operation, after
> >>>> VolumeManagerImpl
> >>>>>>> tells
> >>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a volume, I
> >>>> have
> >>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> >>>> driver if
> >>>>>> it
> >>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver
> perform
> >>>>>>>>>> whatever
> >>>>>>>>>>>>>>> activity is required. In my case, this includes sending a
> >>>>>>>>>> message to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> host where the VM is running to have, say XenServer,
> add a
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the IQN
> of
> >>>> the
> >>>>>>>>>> SAN
> >>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space
> it
> >>>>>> can
> >>>>>>>>>> on
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> storage repository). After this, the normal attach-volume
> >>>>>>>>>> message is
> >>>>>>>>>>>> sent
> >>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There should be **no** code from a storage driver to a
> >>>>>> hypervisor.
> >>>>>>>>>> I
> >>>>>>>>>>>>>> apologize for the repetition, but we simply can not have
> >>>>>> hypervisor
> >>>>>>>>>>>>>> specific code in the storage layer. The circular
> dependencies
> >>>>>>>>>> between
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> two layers are not sustainable in the long term. Either the
> >>>>>>>>>>>> VirtualManager
> >>>>>>>>>>>>>> or Xen hypervisor plugin needs to be refactored/modified
> to
> >>>>>>>>>>> coordinate
> >>>>>>>>>>>>>> volume creation and then populating the SR. Ideally, we
> can
> >>>>>>>>>>>> generalize the
> >>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
> >>>> hypervisor
> >>>>>>>>>> plugin
> >>>>>>>>>>>>>> would only implement callbacks to perform the attach
> action
> >>>> and
> >>>>>>>>>> create
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver should
> >>>>>> only be
> >>>>>>>>>>>>>> allocating space and providing information about contents
> >>>> (e.g.
> >>>>>>>>>> space
> >>>>>>>>>>>>>> available, space consumed, streams to a URI, file handle
> for a
> >>>>>> URI,
> >>>>>>>>>>>> etc)
> >>>>>>>>>>>>>> and capabilities.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * The reverse is performed for a detach-volume
> command.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in my
> >>>>>> driver.
> >>>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>> default driver simply does a "return false;". We could
> add a
> >>>>>> new
> >>>>>>>>>>>>>> parameter
> >>>>>>>>>>>>>>> to the createStoragePool API command, if we want, to
> >>>> remove
> >>>>>>> the
> >>>>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>>>> return values in the drivers (although my driver will
> >> probably
> >>>>>>>>>> just
> >>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>> this parameter and always return true since it wouldn't
> make
> >>>>>>>>>> sense
> >>>>>>>>>>>> for it
> >>>>>>>>>>>>>>> to ever return false). We'd need another column in the
> >>>>>>>>>> storage_pool
> >>>>>>>>>>>>> table
> >>>>>>>>>>>>>>> to store this value.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes, I think we should have a parameter added to the
> >>>>>>>>>>> createStoragePool
> >>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
> >>>>>> configured
> >>>>>>>>>> for
> >>>>>>>>>>>>>> management when their underlying drivers support it. To
> >>>>>> simplify
> >>>>>>>>>>>> things,
> >>>>>>>>>>>>>> this flag should only be mutable when the DataStore is
> >>>>>> created. It
> >>>>>>>>>>>> would be
> >>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to
> unmanaged
> >>>> after
> >>>>>>>>>>>> creation.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
> >>>> working
> >>>>>>>>>> GUI
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>> to review (in that document), if you'd like to check it
> out.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed
> together to
> >>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor
> throttled
> >>>> IOPs
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the potential
> for
> >>>>>>>>>>>> confusion,
> >>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these features
> will
> >>>>>>>>>> help
> >>>>>>>>>>>>>>>>> minimize any potential confusion.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will work
> >>>>>> itself
> >>>>>>>>>> if
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
> >> hypervisor
> >>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google
> doc I
> >>>> sent
> >>>>>>> to
> >>>>>>>>>>> you
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> other day to see how the GUI looks for provisioned
> >>>> storage
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-mail
> >> list
> >>>>>>>>>> and we
> >>>>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add
> Disk
> >>>>>>> Offering
> >>>>>>>>>>>> dialog.
> >>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS, so
> they
> >>>>>>> should
> >>>>>>>>>>> be
> >>>>>>>>>>>>> able
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say,
> only
> >>>>>> use
> >>>>>>>>>> Max
> >>>>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for
> storage
> >>>>>> that
> >>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
> >>>>>> the
> >>>>>>>>>>>> admin can
> >>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> >>>> Burst
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's
> feature
> >>>> or
> >>>>>>>>>> mine,
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> decided by the admin).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two
> features
> >>>>>>>>>> expressing
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express the
> >>>> IOPS
> >>>>>> for
> >>>>>>>>>>>> QoS in
> >>>>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
> >>>>>> read/write
> >>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
> >>>>>>>>>>> Storage/Hypervisor
> >>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the
> behavior
> >>>> of
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see
> the
> >>>>>>>>>> following
> >>>>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>>>> outstanding:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as
> discrete
> >>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are expressed
> as
> >>>> total
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS
> attached
> >>>> to
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for throttled
> and
> >>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> that will permit providers to discriminate these
> >>>>>> guaranteed
> >>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
> >>>>>> provisioned
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
> >>>>>> hypervisor
> >>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would
> neatly
> >>>>>> solve
> >>>>>>>>>>>> items 1
> >>>>>>>>>>>>>>>>> and 2.
> >>>>>>>>>>>>>>>>>>> Since the two facilities would not be permitted to
> >>>> operate
> >>>>>>>>>>>> together,
> >>>>>>>>>>>>>>>>> they
> >>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I think
> item
> >>>> 3
> >>>>>>>>>> can be
> >>>>>>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage records.
> As
> >>>>>> for
> >>>>>>>>>>>> Item 4,
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>>>> not familiar with how these two enhancements are
> >>>>>>> depicted in
> >>>>>>>>>>> the
> >>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
> >>>> enhancements
> >>>>>>> for
> >>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >>>>>>> <us...@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> >>>>>>> 2780/2729,
> >>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-
> 2823 ,
> >>>>>>>>>>> CLOUDSTACK-
> >>>>>>>>>>>>> 2875 ) this
> >>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes
> tomorrow,
> >>>> and
> >>>>>>>>>> ask
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> second
> >>>>>>>>>>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> >>>> <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going with
> this,
> >>>>>> John,
> >>>>>>>>>> and
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager
> determining
> >>>>>>> whether or
> >>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
> StoragePoolType,
> >>>> but an
> >>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
> allocate
> >>>>>>>>>> resources
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an
> isManaged()
> >>>>>>> method
> >>>>>>>>>>>>> added to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns
> true,
> >>>>>>>>>> then the
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or
> whatever
> >>>> name)
> >>>>>>>>>>> method
> >>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface,
> as
> >>>> well)
> >>>>>>> and
> >>>>>>>>>>> this
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to
> create,
> >>>> say
> >>>>>>> on
> >>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean
> on
> >>>>>>>>>>>>> DataStoreDriver.
> >>>>>>>>>>>>>>>>> Since
> >>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> >>>> storage,
> >>>>>>> I
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage
> is a
> >>>>>>> straw
> >>>>>>>>>>> man
> >>>>>>>>>>>>> term
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> >>>> mutable
> >>>>>>>>>>> property
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the underlying
> >>>>>>>>>>> DataStoreDriver
> >>>>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
> >>>>>>> manageability
> >>>>>>>>>> of
> >>>>>>>>>>>>>> devices.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR,
> the
> >>>> Xen
> >>>>>>>>>> plugin
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> responsible for composing DataStore/Volume
> >>>> methods
> >>>>>>> to
> >>>>>>>>>>>>> create any
> >>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR. There
> >>>>>> should
> >>>>>>>>>> be no
> >>>>>>>>>>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I
> said
> >>>>>>>>>> earlier,
> >>>>>>>>>>>> such
> >>>>>>>>>>>>>>>>>>>>> circular
> >>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
> >>>> unmaintainable
> >>>>>>> mess.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page
> with
> >>>> you.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a
> Storage
> >>>> layer
> >>>>>>> that
> >>>>>>>>>>>>>> supports n
> >>>>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors of
> m
> >>>>>>>>>> hypervisors.
> >>>>>>>>>>>>>> Such
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
> >>>> untestable
> >>>>>>>>>>> beast.
> >>>>>>>>>>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven to
> >>>> evolve
> >>>>>>> the
> >>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager
> determining
> >>>>>>> whether
> >>>>>>>>>>> or
> >>>>>>>>>>>>> not a
> >>>>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the
> StoragePoolType,
> >>>> but
> >>>>>>> an
> >>>>>>>>>>> actual
> >>>>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to
> allocate
> >>>>>>>>>> resources
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device
> driver
> >>>>>> needs
> >>>>>>>>>> to
> >>>>>>>>>>>>>> indicate
> >>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
> >>>> Finally, I
> >>>>>>>>>> think
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to elect
> to
> >>>>>> have
> >>>>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> >>>> capable
> >>>>>>>>>>> managing
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
> >>>>>>> unmanaged).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume should
> be
> >>>> done
> >>>>>>> in
> >>>>>>>>>>> the
> >>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed by
> the
> >>>>>>> Storage
> >>>>>>>>>>>>> layer to
> >>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> >>>> create a
> >>>>>>>>>> file,
> >>>>>>>>>>>> etc).
> >>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the
> Xen
> >>>>>>>>>>>> hypervisor, as
> >>>>>>>>>>>>>>>>> such,
> >>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation.
> From
> >>>> my
> >>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>> nothing
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
> >>>> content.
> >>>>>>> From
> >>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides
> the
> >>>> means
> >>>>>>> to
> >>>>>>>>>>>> query
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the content by
> >>>> higher-
> >>>>>>> level
> >>>>>>>>>>>>> layers
> >>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> >>>>>>> composition
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on
> the
> >>>>>>> Volume
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> protocol
> >>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> >>>>>>> implemented
> >>>>>>>>>> in
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>> patch,
> >>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone.
> In
> >>>>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it
> to
> >>>> the
> >>>>>>>>>>>>> hypervisor.
> >>>>>>>>>>>>>> Before
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if
> we're
> >>>>>>> dealing
> >>>>>>>>>>> with
> >>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>> (or
> >>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so
> -
> >>>>>> send a
> >>>>>>>>>>>> "Create
> >>>>>>>>>>>>>>>>> SR"
> >>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK,
> we
> >>>>>>> would
> >>>>>>>>>>> then
> >>>>>>>>>>>>>>>>> proceed
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same
> and
> >>>> we just
> >>>>>>>>>> add
> >>>>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called for
> this
> >>>>>>>>>> particular
> >>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike
> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer today
> that
> >>>>>>> outlines
> >>>>>>>>>>> how
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will
> probably
> >>>> be
> >>>>>> of
> >>>>>>>>>> use
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> you, as
> >>>>>>>>>>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you (you
> >>>> should
> >>>>>>> have
> >>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike
> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John
> Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> >>>>>>> thoughts,
> >>>>>>>>>>> but
> >>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> >>>>>> solidfire,
> >>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect
> them
> >>>> in a
> >>>>>>>>>>>>> coherent
> >>>>>>>>>>>>>>>>> form
> >>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike
> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> >>>> introduces
> >>>>>>> the
> >>>>>>>>>>>> notion
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> >>>> introduces
> >>>>>>>>>> the
> >>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages
> the
> >>>> plug-in
> >>>>>>> was
> >>>>>>>>>>>>>>>>> incomplete,
> >>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed
> iSCSI
> >>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> >>>> spending on
> >>>>>>>>>> this.
> >>>>>>>>>>>> :)
> >>>>>>>>>>>>> Do
> >>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
> >>>> accomplish
> >>>>>>>>>>> what
> >>>>>>>>>>>>> you're
> >>>>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John
> Burwell
> >>>> <
> >>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> >>>> introduces
> >>>>>>> the
> >>>>>>>>>>>> notion
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> >>>> management
> >>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>> (i.e.
> >>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator
> want
> >>>> it
> >>>>>>>>>>> managed
> >>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should
> end up
> >>>> with
> >>>>>>> a
> >>>>>>>>>>>>> semantic
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> >>>> managed.
> >>>>>>> I
> >>>>>>>>>>> also
> >>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage
> types
> >>>> from
> >>>>>>> the
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike
> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the
> following
> >>>>>>> questions
> >>>>>>>>>>> for
> >>>>>>>>>>>>> me
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> >>>> LUNs
> >>>>>>> and
> >>>>>>>>>>> SAN
> >>>>>>>>>>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a
> single
> >>>> LUN
> >>>>>>> (LUN
> >>>>>>>>>>> 0).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> >>>> Volumes
> >>>>>>> and
> >>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> >>>> management
> >>>>>>>>>>> server or
> >>>>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in,
> a
> >>>> SAN
> >>>>>>>>>>> volume
> >>>>>>>>>>>>> (same
> >>>>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server
> (via
> >>>> the
> >>>>>>> plug-in)
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> >>>> hypervisor.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire
> plug-
> >>>> in,
> >>>>>>> but
> >>>>>>>>>>>> still
> >>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do
> this
> >>>> already
> >>>>>>>>>>> today
> >>>>>>>>>>>>>> (prior
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN
> volume
> >>>> and -
> >>>>>>> in
> >>>>>>>>>>> this
> >>>>>>>>>>>>> case -
> >>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> >>>> While
> >>>>>>> you
> >>>>>>>>>>> can do
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce
> storage
> >>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> >>>> management
> >>>>>>> server
> >>>>>>>>>>> or
> >>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> >>>> SAN
> >>>>>>>>>>> volumes
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the
> plug-
> >>>> in).
> >>>>>>> There
> >>>>>>>>>>> is
> >>>>>>>>>>>>> no
> >>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between
> a
> >>>> SAN
> >>>>>>>>>>> volume
> >>>>>>>>>>>>> and a
> >>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage
> vendor
> >>>> that
> >>>>>>>>>>>>> supports
> >>>>>>>>>>>>>> true
> >>>>>>>>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these
> pieces
> >>>> are
> >>>>>>>>>> related
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure
> that
> >>>>>>> multiple
> >>>>>>>>>>> VMs
> >>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> >>>> storage."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need
> to
> >>>> map a
> >>>>>>>>>>> single
> >>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We
> cannot
> >>>> have
> >>>>>>>>>>> multiple
> >>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> >>>>>>> guarantee
> >>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single
> SAN
> >>>>>>> volume
> >>>>>>>>>>> house
> >>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do
> that
> >>>> today
> >>>>>>>>>>> without
> >>>>>>>>>>>>> any of
> >>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM,
> Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a
> SAN
> >>>>>>> volume
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> >>>> created."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM,
> John
> >>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> >>>>>>> concerns me
> >>>>>>>>>>> --
> >>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into
> storage
> >>>> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI
> type
> >>>> is
> >>>>>>> today
> >>>>>>>>>>>> only
> >>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when
> you
> >>>> have
> >>>>>>> not
> >>>>>>>>>>>>> PreSetup an
> >>>>>>>>>>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >>>> volume
> >>>>>>> from
> >>>>>>>>>>>>> VMware, it
> >>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >>>> volume
> >>>>>>> from
> >>>>>>>>>>> KVM,
> >>>>>>>>>>>>> it uses
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> >>>> thinking
> >>>>>>>>>>> here
> >>>>>>>>>>>>> was to
> >>>>>>>>>>>>>>>>> make a
> >>>>>>>>>>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this
> dynamic
> >>>> ability
> >>>>>>>>>>> Edison
> >>>>>>>>>>>>> added
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> >>>> more
> >>>>>>>>>>> specificy,
> >>>>>>>>>>>>> though:
> >>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> >>>> Mike
> >>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> >>>> shouldn't
> >>>>>>> know
> >>>>>>>>>>> or
> >>>>>>>>>>>>> care
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able
> to
> >>>> answer
> >>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is
> that
> >>>> the
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so
> that
> >>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing
> that
> >>>>>>> constraint
> >>>>>>>>>>>>> and just
> >>>>>>>>>>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just
> need a
> >>>> way
> >>>>>>> for
> >>>>>>>>>>> the
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement
> and
> >>>>>>> perform
> >>>>>>>>>>> the
> >>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> >>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM,
> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack
> may
> >>>> be
> >>>>>>> able to
> >>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it
> unmanaged by
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>> (e.g. the
> >>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> >>>> operator has
> >>>>>>>>>>> chosen
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> >>>> reasoning
> >>>>>>>>>>> make
> >>>>>>>>>>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> >>>> creating a
> >>>>>>> SAN
> >>>>>>>>>>>>> volume via
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> >>>> hypervisor)
> >>>>>>> would
> >>>>>>>>>>>>> work unless
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form
> of,
> >>>> say,
> >>>>>>> an
> >>>>>>>>>>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that,
> independent
> >>>> of
> >>>>>>>>>>>>> CloudStack, an
> >>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to
> CloudStack,
> >>>> and
> >>>>>>>>>>> exposes it
> >>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that
> LUN
> >>>> over
> >>>>>>> to
> >>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate
> anymore
> >>>> LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands
> prior
> >>>> to my
> >>>>>>>>>>>>> changes, we
> >>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not
> have
> >>>> the
> >>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> >>>> maybe
> >>>>>>> have
> >>>>>>>>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume
> needs
> >>>>>>> support
> >>>>>>>>>>> from
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> >>>> specifically
> >>>>>>> to
> >>>>>>>>>>> create
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> SR
> >>>>>>>>>>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> >>>> management
> >>>>>>> plugin
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> >>>> determine
> >>>>>>>>>>> available
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow
> the
> >>>>>>>>>>> interrogation of
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> >>>> device
> >>>>>>> itself
> >>>>>>>>>>>>> shouldn't
> >>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> >>>> simply be
> >>>>>>> able
> >>>>>>>>>>> to
> >>>>>>>>>>>>>> answer
> >>>>>>>>>>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01
> PM,
> >>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> >>>> SolidFire
> >>>>>>>>>>> driver
> >>>>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once
> this is
> >>>>>>> done, a
> >>>>>>>>>>> new
> >>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> >>>> Offering
> >>>>>>>>>>> that
> >>>>>>>>>>>>>> references
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> >>>> Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> >>>>>>> CloudStack
> >>>>>>>>>>>>> volume. This
> >>>>>>>>>>>>>>>>> leads
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches
> the
> >>>>>>> CloudStack
> >>>>>>>>>>>>> volume to a
> >>>>>>>>>>>>>>>>> VM
> >>>>>>>>>>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> >>>> framework
> >>>>>>> calling
> >>>>>>>>>>> the
> >>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in
> my
> >>>> case, a
> >>>>>>>>>>> SAN).
> >>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> >>>> applicable
> >>>>>>> data
> >>>>>>>>>>>>> (like the
> >>>>>>>>>>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> >>>> invoked if
> >>>>>>> the
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> >>>> attach,
> >>>>>>> the
> >>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach
> disk)
> >>>> and
> >>>>>>> the
> >>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> >>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach
> logic
> >>>> is run
> >>>>>>> and
> >>>>>>>>>>>>> detects
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance"
> in
> >>>> the
> >>>>>>> form
> >>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> 12:54
> >>>> PM,
> >>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> >>>> sync on
> >>>>>>>>>>>>> terminology,
> >>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> >>>> physical
> >>>>>>>>>>>>> allocation on
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume',
> I
> >>>> try to
> >>>>>>>>>>> mean
> >>>>>>>>>>>>> 'SAN
> >>>>>>>>>>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume'
> the
> >>>> end
> >>>>>>> user
> >>>>>>>>>>> can
> >>>>>>>>>>>>> make in
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> 12:50
> >>>> PM,
> >>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> >>>> very
> >>>>>>> well
> >>>>>>>>>>> make
> >>>>>>>>>>>>> sense, but
> >>>>>>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should
> draw
> >>>> Edison
> >>>>>>> in on
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest
> the
> >>>>>>>>>>> approach I
> >>>>>>>>>>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> 12:42
> >>>> PM,
> >>>>>>>>>>> John
> >>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> >>>> combining
> >>>>>>> two
> >>>>>>>>>>>>> distinct
> >>>>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and
> storage
> >>>>>>> protocols.
> >>>>>>>>>>> In
> >>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows
> the
> >>>>>>> system to
> >>>>>>>>>>>>>> create/delete
> >>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> >>>> requires
> >>>>>>> the
> >>>>>>>>>>>>> volume to
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the
> CloudStack
> >>>>>>> context.
> >>>>>>>>>>> To
> >>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume,
> in
> >>>> these
> >>>>>>>>>>>>> definitions,
> >>>>>>>>>>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the
> device,
> >>>>>>> correct?
> >>>>>>>>>>>>> Minimally,
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> >>>> storage
> >>>>>>> device
> >>>>>>>>>>> to
> >>>>>>>>>>>>> move bits
> >>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits,
> etc.
> >>>>>>>>>>> Optionally, a
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
> CloudStack.
> >>>>>>> Therefore,
> >>>>>>>>>>>>> we can have
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a
> Xen
> >>>> SR,
> >>>>>>> and
> >>>>>>>>>>> we
> >>>>>>>>>>>>> can have a
> >>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> >>>> CloudStack is
> >>>>>>>>>>> capable
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally,
> while
> >>>>>>> CloudStack
> >>>>>>>>>>> may
> >>>>>>>>>>>>> be able
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> >>>> leave it
> >>>>>>>>>>>>> unmanaged by
> >>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many
> services,
> >>>> and
> >>>>>>> the
> >>>>>>>>>>>>> operator has
> >>>>>>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> >>>> CloudStack).
> >>>>>>> Does
> >>>>>>>>>>> my
> >>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> >>>> above
> >>>>>>> are
> >>>>>>>>>>>>> reasonable, it
> >>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> >>>>>>> concerns
> >>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> >>>>>>> attached
> >>>>>>>>>>> driver
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by
> CloudStack,
> >>>> and
> >>>>>>>>>>> establish
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> >>>> separate
> >>>>>>> that
> >>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> associated
> >>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> >>>>>>> hypervisor
> >>>>>>>>>>>>> drivers can
> >>>>>>>>>>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when
> they
> >>>>>>> encounter
> >>>>>>>>>>> a
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management
> operations
> >>>>>>> exposed
> >>>>>>>>>>> by
> >>>>>>>>>>>>> the driver to
> >>>>>>>>>>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these
> thoughts/concepts
> >>>> make
> >>>>>>>>>>> sense,
> >>>>>>>>>>>>> then we can
> >>>>>>>>>>>>>>>>> sit
> >>>>>>>>>>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25
> PM,
> >>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the
> difference
> >>>>>>> between
> >>>>>>>>>>> the
> >>>>>>>>>>>>> current
> >>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> >>>> has to
> >>>>>>> go in
> >>>>>>>>>>> and
> >>>>>>>>>>>>> create
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> >>>> point in
> >>>>>>> time,
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is
> pre-
> >>>>>>> allocated).
> >>>>>>>>>>>>> Future
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR
> that
> >>>> was
> >>>>>>> created
> >>>>>>>>>>>>> behind the
> >>>>>>>>>>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The
> admin
> >>>> has
> >>>>>>> to
> >>>>>>>>>>> go in
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will
> create
> >>>> and
> >>>>>>> delete
> >>>>>>>>>>>>> volumes on
> >>>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> >>>> enabled via
> >>>>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> >>>> volume
> >>>>>>> that
> >>>>>>>>>>> was
> >>>>>>>>>>>>> created,
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a
> new
> >>>>>>> volume.
> >>>>>>>>>>> After
> >>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>> done,
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> >>>> question is
> >>>>>>> called.
> >>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because
> the
> >>>>>>> volume
> >>>>>>>>>>> was
> >>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> >>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> >>>> 3:21
> >>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new
> terms,
> >>>> so I
> >>>>>>> should
> >>>>>>>>>>>>> probably
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage:
> Storage
> >>>> that is
> >>>>>>>>>>> pre-
> >>>>>>>>>>>>> allocated
> >>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN),
> then
> >>>> a
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> >>>> XenServer
> >>>>>>> SR),
> >>>>>>>>>>> then
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> >>>> CloudStack.
> >>>>>>> Disks
> >>>>>>>>>>> (VDI)
> >>>>>>>>>>>>> are later
> >>>>>>>>>>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> >>>> needed. In
> >>>>>>>>>>> these
> >>>>>>>>>>>>> cases, the
> >>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >>>> structure is
> >>>>>>>>>>> already
> >>>>>>>>>>>>> in place
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor
> data
> >>>>>>> structure
> >>>>>>>>>>> to
> >>>>>>>>>>>>> the VM in
> >>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> >>>> Storage
> >>>>>>> that is
> >>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent
> storage,
> >>>> this
> >>>>>>> could
> >>>>>>>>>>> be a
> >>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>> (not
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> >>>> hypervisor
> >>>>>>> data
> >>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>> must be
> >>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is
> performed
> >>>>>>> because
> >>>>>>>>>>> these
> >>>>>>>>>>>>> types of
> >>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a
> hypervisor
> >>>> data
> >>>>>>>>>>>>> structure by an
> >>>>>>>>>>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR
> on
> >>>>>>> XenServer
> >>>>>>>>>>> for
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the
> SR
> >>>> to the
> >>>>>>> VM
> >>>>>>>>>>> in
> >>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> >>>> 3:13
> >>>>>>> PM,
> >>>>>>>>>>>>> John Burwell
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >>>>>>> implementation of
> >>>>>>>>>>> the
> >>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> >>>> which
> >>>>>>> why I
> >>>>>>>>>>> ask
> >>>>>>>>>>>>> about the
> >>>>>>>>>>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> >>>> what is
> >>>>>>> the
> >>>>>>>>>>>>> definition
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> 5:10
> >>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know,
> the
> >>>> iSCSI
> >>>>>>> type
> >>>>>>>>>>> is
> >>>>>>>>>>>>>> uniquely
> >>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> >>>> Primary
> >>>>>>>>>>> Storage
> >>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>> directly
> >>>>>>>>>>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> >>>> skip the
> >>>>>>>>>>> step of
> >>>>>>>>>>>>> going
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> repository
> >>>> based
> >>>>>>> on
> >>>>>>>>>>> that
> >>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think
> this is
> >>>> only
> >>>>>>>>>>>>> supported for
> >>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> >>>> must
> >>>>>>> first go
> >>>>>>>>>>> to
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really
> know
> >>>> what
> >>>>>>> RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> >>>> at
> >>>>>>> 2:13
> >>>>>>>>>>> PM,
> >>>>>>>>>>>>> John
> >>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading
> through
> >>>> the
> >>>>>>> code,
> >>>>>>>>>>> what
> >>>>>>>>>>>>> is the
> >>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> >>>> Why
> >>>>>>> isn't
> >>>>>>>>>>> RBD
> >>>>>>>>>>>>> considered
> >>>>>>>>>>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> at
> >>>> 3:46
> >>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type
> of
> >>>>>>> storage is
> >>>>>>>>>>>>> defined in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called
> Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static
> enum
> >>>>>>>>>>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Filesystem(false),
> >>>> //
> >>>>>>> local
> >>>>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> NetworkFilesystem(true),
> >>>>>>>>>>> //
> >>>>>>>>>>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true),
> //
> >>>>>>> shared
> >>>>>>>>>>> LUN,
> >>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), //
> for
> >>>> e.g.,
> >>>>>>> ZFS
> >>>>>>>>>>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), //
> for
> >>>> iso
> >>>>>>> image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> >>>>>>> XenServer
> >>>>>>>>>>> local
> >>>>>>>>>>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> >>>>>>> VMware
> >>>>>>>>>>> VMFS
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> PreSetup(true), //
> >>>> for
> >>>>>>>>>>>>> XenServer, Storage
> >>>>>>>>>>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> >>>>>>> XenServer
> >>>>>>>>>>> local
> >>>>>>>>>>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true);
> //
> >>>>>>> dynamic,
> >>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean
> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> StoragePoolType(boolean
> >>>>>>>>>>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> >>>> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> >>>>>>> isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>> 2013 at
> >>>>>>> 1:41
> >>>>>>>>>>> PM,
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> >>>> let's say
> >>>>>>>>>>> another
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> >>>> leverage its
> >>>>>>>>>>> Quality
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>> Service
> >>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>> storage, as
> >>>>>>> well.
> >>>>>>>>>>>>> They would
> >>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> >>>> made
> >>>>>>> the
> >>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support
> their
> >>>> plug-
> >>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> 3,
> >>>> 2013
> >>>>>>> at
> >>>>>>>>>>> 1:39
> >>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> >>>> imply
> >>>>>>> the
> >>>>>>>>>>>>> XenServer code
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> XenServer
> >>>>>>> attach
> >>>>>>>>>>> logic is
> >>>>>>>>>>>>> now aware
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and
> SolidFire is
> >>>> an
> >>>>>>>>>>>>> implementation of
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is
> new
> >>>> to 4.2
> >>>>>>>>>>> with
> >>>>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison
> created
> >>>> a
> >>>>>>> new
> >>>>>>>>>>>>> framework that
> >>>>>>>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> >>>>>>> dynamically.
> >>>>>>>>>>>>> However, when I
> >>>>>>>>>>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> >>>> realized
> >>>>>>> that
> >>>>>>>>>>> it
> >>>>>>>>>>>>> was not
> >>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> >>>> CloudStack
> >>>>>>> could
> >>>>>>>>>>> do
> >>>>>>>>>>>>> with these
> >>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> >>>>>>> recognize
> >>>>>>>>>>> this
> >>>>>>>>>>>>> new type of
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> data
> >>>>>>> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >>>> 2013
> >>>>>>> at
> >>>>>>>>>>> 1:28
> >>>>>>>>>>>>> PM, John
> >>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> generally
> >>>> odd
> >>>>>>> to me
> >>>>>>>>>>>>> that any
> >>>>>>>>>>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> understand or
> >>>> care
> >>>>>>>>>>> about
> >>>>>>>>>>>>> details. I
> >>>>>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a
> set
> >>>> of
> >>>>>>>>>>> operations
> >>>>>>>>>>>>> that can be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> implementations to
> >>>>>>>>>>>>> allocate
> >>>>>>>>>>>>>> space/create
> >>>>>>>>>>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> invert
> >>>> this
> >>>>>>>>>>>>> dependency, we are
> >>>>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem
> that
> >>>> will
> >>>>>>> make
> >>>>>>>>>>> the
> >>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance.
> Am
> >>>> I
> >>>>>>>>>>>>> understanding that the
> >>>>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in
> the
> >>>>>>>>>>>>> CitrixResourceBase
> >>>>>>>>>>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >>>>>>> 2013 at
> >>>>>>>>>>> 3:12
> >>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve
> into
> >>>> this
> >>>>>>> in a
> >>>>>>>>>>> bit
> >>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to
> 4.2
> >>>> and
> >>>>>>> aside
> >>>>>>>>>>> from
> >>>>>>>>>>>>> one setup
> >>>>>>>>>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first
> create
> >>>> a
> >>>>>>>>>>> volume on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up
> a
> >>>> data
> >>>>>>>>>>> structure
> >>>>>>>>>>>>> to make
> >>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> repository on
> >>>>>>>>>>> XenServer
> >>>>>>>>>>>>> or a
> >>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> this
> >>>>>>> storage
> >>>>>>>>>>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> Edison's
> >>>> new
> >>>>>>>>>>> storage
> >>>>>>>>>>>>> framework,
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you
> can
> >>>> easily
> >>>>>>>>>>> create
> >>>>>>>>>>>>> a 1:1
> >>>>>>>>>>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> and
> >>>> the
> >>>>>>> VM's
> >>>>>>>>>>> data
> >>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>> (necessary
> >>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can
> now
> >>>> write
> >>>>>>> a
> >>>>>>>>>>> plug-
> >>>>>>>>>>>>> in that is
> >>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> as
> >>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> problem
> >>>> that
> >>>>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting
> the
> >>>>>>>>>>> hypervisor-
> >>>>>>>>>>>>> specific data
> >>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> being
> >>>> the
> >>>>>>> case,
> >>>>>>>>>>> I've
> >>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>> enhancing
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked
> out
> >>>> and
> >>>>>>>>>>>>> submitted. I've got
> >>>>>>>>>>>>>>>>> ESX(i)
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we
> extend
> >>>> the
> >>>>>>> 4.2
> >>>>>>>>>>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> >>>> help a
> >>>>>>> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun
> >>>> 3,
> >>>>>>> 2013 at
> >>>>>>>>>>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> storage
> >>>>>>> plug-in -
> >>>>>>>>>>> by
> >>>>>>>>>>>>> itself - is
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> issue is
> >>>> with
> >>>>>>> the
> >>>>>>>>>>>>> volume-attach
> >>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> framework
> >>>> calls
> >>>>>>> into
> >>>>>>>>>>> the
> >>>>>>>>>>>>> plug-in to
> >>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> the
> >>>> time
> >>>>>>>>>>> comes to
> >>>>>>>>>>>>> attach the
> >>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to
> be
> >>>> smart
> >>>>>>>>>>> enough
> >>>>>>>>>>>>> to recognize
> >>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where
> the
> >>>>>>> volume
> >>>>>>>>>>> has
> >>>>>>>>>>>>> just been
> >>>>>>>>>>>>>>>>> created)
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> repository
> >>>> (for
> >>>>>>>>>>>>> XenServer) or a
> >>>>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> that
> >>>> was
> >>>>>>> just
> >>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> been
> >>>>>>> spending
> >>>>>>>>>>> most
> >>>>>>>>>>>>> of my time
> >>>>>>>>>>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
> agent
> >>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does
> that
> >>>> clear it
> >>>>>>> up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >>>> Jun 3,
> >>>>>>> 2013
> >>>>>>>>>>> at
> >>>>>>>>>>>>> 12:48 PM,
> >>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> you
> >>>> explain
> >>>>>>>>>>> why
> >>>>>>>>>>>>> the the storage
> >>>>>>>>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
> 3,
> >>>> 2013,
> >>>>>>> at
> >>>>>>>>>>> 1:21
> >>>>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> >>>>>>> ultimately I
> >>>>>>>>>>>>> would like to
> >>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> supports.
> >>>> I
> >>>>>>> think
> >>>>>>>>>>> I'm
> >>>>>>>>>>>>> just out of
> >>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> >>>> now this
> >>>>>>>>>>> plug-in
> >>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> regards
> >>>> to 4.2
> >>>>>>>>>>> feature
> >>>>>>>>>>>>> freeze, I
> >>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> just
> >>>> to
> >>>>>>> be
> >>>>>>>>>>> clear,
> >>>>>>>>>>>>> this is
> >>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> support
> >>>>>>> Compute
> >>>>>>>>>>>>> Offerings post
> >>>>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Mon,
> >>>> Jun 3,
> >>>>>>>>>>> 2013
> >>>>>>>>>>>>> at 11:14 AM,
> >>>>>>>>>>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is
> there
> >>>> any
> >>>>>>> plan
> >>>>>>>>>>> on
> >>>>>>>>>>>>> supporting
> >>>>>>>>>>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> >>>> Original
> >>>>>>>>>>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> From:
> >>>> "Mike
> >>>>>>>>>>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >>>>>>>>>>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Sent:
> >>>>>>> Monday,
> >>>>>>>>>>> June
> >>>>>>>>>>>>> 3, 2013
> >>>>>>>>>>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Subject:
> >>>> Re:
> >>>>>>>>>>> [MERGE]
> >>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> agree
> >>>> on
> >>>>>>>>>>> merging
> >>>>>>>>>>>>> Wei's feature
> >>>>>>>>>>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> his
> >>>> feature
> >>>>>>> is
> >>>>>>>>>>> for
> >>>>>>>>>>>>> KVM only,
> >>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> KVM in
> >>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Mon,
> >>>> Jun
> >>>>>>> 3,
> >>>>>>>>>>> 2013
> >>>>>>>>>>>>> at 8:55 AM,
> >>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> For
> >>>> the
> >>>>>>> billing,
> >>>>>>>>>>> as
> >>>>>>>>>>>>> no one works
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >>>> billing
> >>>>>>> by
> >>>>>>>>>>>>> themselves. They
> >>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> disk_offering of
> >>>>>>>>>>> a
> >>>>>>>>>>>>> VMs and
> >>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> tell
> >>>> user
> >>>>>>> the
> >>>>>>>>>>>>> exact
> >>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> rate
> >>>>>>> limitation
> >>>>>>>>>>> for
> >>>>>>>>>>>>> nics as
> >>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>> part of
> >>>>>>> I/O
> >>>>>>>>>>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Sorry
> >>>> my
> >>>>>>>>>>>>> misunstand the second
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Agree
> >>>> with
> >>>>>>>>>>> what
> >>>>>>>>>>>>> you said about
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> 2013/6/3
> >>>>>>> John
> >>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >>>> Jun 3,
> >>>>>>> 2013,
> >>>>>>>>>>> at
> >>>>>>>>>>>>> 2:13 AM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi
> >>>> John,
> >>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>> hope
> >>>>>>> Mike's
> >>>>>>>>>>>>> aswer helps you.
> >>>>>>>>>>>>>>>>> I am
> >>>>>>>>>>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (1) I
> >>>>>>> think
> >>>>>>>>>>> billing
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>> depend
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> limitation.
> >>>>>>>>>>> Please
> >>>>>>>>>>>>> review
> >>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> get
> >>>> the
> >>>>>>> IO
> >>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> virtual
> >>>>>>>>>>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Going by
> >>>>>>> the
> >>>>>>>>>>> AWS
> >>>>>>>>>>>>> model,
> >>>>>>>>>>>>>> customers
> >>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> provisioned
> >>>>>>>>>>> IOPS,
> >>>>>>>>>>>>> as well as,
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://aws.amazon.com/ebs/).
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> option to
> >>>>>>>>>>> employ
> >>>>>>>>>>>>> similar cost
> >>>>>>>>>>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> billing
> >>>>>>> model
> >>>>>>>>>>> in
> >>>>>>>>>>>>> the current
> >>>>>>>>>>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (2)
> >>>> Do
> >>>>>>> you
> >>>>>>>>>>> mean
> >>>>>>>>>>>>> IOPS runtime
> >>>>>>>>>>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> limitation for
> >>>>>>>>>>> a
> >>>>>>>>>>>>> running
> >>>>>>>>>>>>>> virtual
> >>>>>>>>>>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> CloudStack
> >>>>>>>>>>> does
> >>>>>>>>>>>>> not support
> >>>>>>>>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> (computer
> >>>>>>>>>>>>> offering or disk
> >>>>>>>>>>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>> meant at
> >>>>>>> the
> >>>>>>>>>>>>> Java interface
> >>>>>>>>>>>>>>>>> level.
> >>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> we
> >>>> more
> >>>>>>>>>>>>> generalize allocation
> >>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> describe
> >>>>>>> the
> >>>>>>>>>>>>> service guarantees
> >>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (3)
> >>>> It is a
> >>>>>>>>>>> good
> >>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>> Maybe it
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> disk_io_throttling as Mike
> >>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> type, I
> >>>>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> will
> >>>>>>> expand
> >>>>>>>>>>> on
> >>>>>>>>>>>>> my thoughts
> >>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> touch
> >>>>>>> points
> >>>>>>>>>>>>> between these two
> >>>>>>>>>>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> will
> >>>> need
> >>>>>>> to be
> >>>>>>>>>>>>> merged before
> >>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> coordination
> >>>>>>>>>>>>> between the
> >>>>>>>>>>>>>> branches
> >>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> disk_io_throttling)
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> 2013/6/3
> >>>>>>>>>>> John
> >>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> The
> >>>>>>> things I
> >>>>>>>>>>>>> want to
> >>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 1)
> >>>> Is
> >>>>>>> there
> >>>>>>>>>>>>> value in
> >>>>>>>>>>>>>> capturing
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> data
> >>>>>>> model
> >>>>>>>>>>> (e.g.
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2)
> >>>>>>> Should
> >>>>>>>>>>> there
> >>>>>>>>>>>>> be a common
> >>>>>>>>>>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> provisioning
> >>>>>>>>>>> at
> >>>>>>>>>>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 3)
> >>>> How
> >>>>>>> are
> >>>>>>>>>>>>> conflicting
> >>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> hypervisor
> >>>>>>>>>>> and
> >>>>>>>>>>>>> storage device
> >>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> is
> >>>> lead
> >>>>>>> to
> >>>>>>>>>>>>> believe (and
> >>>>>>>>>>>>>> billed)
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> storage
> >>>>>>>>>>> device
> >>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> could
> >>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> consistent
> >>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>> between a
> >>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> time,
> >>>>>>> but a
> >>>>>>>>>>>>> later
> >>>>>>>>>>>>>> modification
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >>>> Jun
> >>>>>>> 2,
> >>>>>>>>>>> 2013,
> >>>>>>>>>>>>> at 8:38 PM,
> >>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi
> >>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>>>>> believe
> >>>>>>>>>>> Wei's
> >>>>>>>>>>>>> feature deals
> >>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> My
> >>>>>>> feature
> >>>>>>>>>>> is
> >>>>>>>>>>>>> focused on
> >>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>> hope
> >>>>>>> that
> >>>>>>>>>>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >>>> Sun,
> >>>>>>> Jun
> >>>>>>>>>>> 2,
> >>>>>>>>>>>>> 2013 at 6:35
> >>>>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> My
> >>>>>>>>>>> opinion is
> >>>>>>>>>>>>> that no
> >>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> issues
> >>>>>>>>>>> have
> >>>>>>>>>>>>> been resolved
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> total
> >>>>>>> Ops
> >>>>>>>>>>> vs
> >>>>>>>>>>>>> discrete
> >>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> >>>>>>> reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido,
> >>>>>>> I
> >>>>>>>>>>> don't
> >>>>>>>>>>>>> think this
> >>>>>>>>>>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Also,
> >>>>>>> how
> >>>>>>>>>>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> also
> >>>>>>>>>>> involves
> >>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> scenario
> >>>>>>>>>>>>> where provisioned
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> -
> >>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> On
> >>>>>>> Jun 1,
> >>>>>>>>>>> 2013,
> >>>>>>>>>>>>> at 6:47 AM,
> >>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Sure. I
> >>>>>>> will
> >>>>>>>>>>>>> change it next
> >>>>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> 2013/6/1
> >>>>>>>>>>>>> Wido den Hollander
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> On
> >>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> Exactly. I
> >>>>>>>>>>> have
> >>>>>>>>>>>>> pushed the
> >>>>>>>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> If
> >>>>>>> anyone
> >>>>>>>>>>>>> object thems for
> >>>>>>>>>>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> For
> >>>>>>> the
> >>>>>>>>>>> sake
> >>>>>>>>>>>>> of clarity I
> >>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> the
> >>>>>>> total
> >>>>>>>>>>> IOps
> >>>>>>>>>>>>> to R/W IOps
> >>>>>>>>>>>>>>>>> asap
> >>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> only
> >>>>>>> total
> >>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> You
> >>>>>>> laid
> >>>>>>>>>>> the
> >>>>>>>>>>>>> groundwork for
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> however
> >>>>>>>>>>>>> prevent that we
> >>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> 2013/5/31
> >>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> On
> >>>>>>>>>>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> +1 --
> >>>>>>> this
> >>>>>>>>>>>>> enhancement must
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> don't
> >>>>>>> see
> >>>>>>>>>>> how
> >>>>>>>>>>>>> it could be
> >>>>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> split
> >>>>>>> total
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>> into read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> total
> >>>>>>>>>>>>> unless/until we
> >>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> What
> >>>>>>> Wei
> >>>>>>>>>>>>> meant was merging
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> 4.2
> >>>>>>> branch
> >>>>>>>>>>>>> and add Read /
> >>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> will be
> >>>>>>>>>>>>> released with Read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> This is
> >>>>>>> to
> >>>>>>>>>>>>> make the May 31st
> >>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> (see
> >>>>>>> other
> >>>>>>>>>>>>> threads) then it
> >>>>>>>>>>>>>>>>> won't
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> I
> >>>> also
> >>>>>>>>>>>>> completely agree that
> >>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> disk
> >>>>>>> I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> -
> >>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> On
> >>>>>>> May 31,
> >>>>>>>>>>>>> 2013, at 9:51 AM,
> >>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> On
> >>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Hi
> >>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> Thanks.
> >>>>>>>>>>> Good
> >>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> I
> >>>>>>> thought
> >>>>>>>>>>>>> about at the
> >>>>>>>>>>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> difference
> >>>>>>>>>>> of
> >>>>>>>>>>>>> read and write
> >>>>>>>>>>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> care
> >>>>>>> the
> >>>>>>>>>>>>> difference of sent
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> That
> >>>>>>>>>>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Disk
> >>>>>>> I/O is
> >>>>>>>>>>>>> much more
> >>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> Implementing
> >>>>>>>>>>>>> it will be some
> >>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> implemented
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> few
> >>>>>>> days.
> >>>>>>>>>>> For
> >>>>>>>>>>>>> the deadline
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> that ,
> >>>>>>> if
> >>>>>>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> It
> >>>>>>> think it's
> >>>>>>>>>>> a
> >>>>>>>>>>>>> feature we
> >>>>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> window
> >>>>>>>>>>> we
> >>>>>>>>>>>>> have to make sure
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> it
> >>>> in
> >>>>>>> 4.3,
> >>>>>>>>>>> that
> >>>>>>>>>>>>> will confuse
> >>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> -
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> 2013/5/31
> >>>>>>>>>>>>> Wido den
> >>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> On
> >>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> I
> >>>>>>> would
> >>>>>>>>>>> like to
> >>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> If
> >>>>>>> nobody
> >>>>>>>>>>>>> object, I will
> >>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> The
> >>>>>>>>>>> purpose
> >>>>>>>>>>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Virtual
> >>>>>>>>>>>>> machines are running
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> share
> >>>>>>>>>>> strage).
> >>>>>>>>>>>>> Because of
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> iops),
> >>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> one
> >>>>>>> VM
> >>>>>>>>>>> has
> >>>>>>>>>>>>> large disk
> >>>>>>>>>>>>>>>>> operation,
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> other
> >>>>>>> VMs
> >>>>>>>>>>>>> running on the
> >>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> It
> >>>> is
> >>>>>>>>>>> neccesary
> >>>>>>>>>>>>> to set the
> >>>>>>>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> Looking at
> >>>>>>>>>>> the
> >>>>>>>>>>>>> code I see
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Qemu
> >>>>>>> and
> >>>>>>>>>>>>> libvirt support
> >>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> IOps
> >>>>>>>>>>> which
> >>>>>>>>>>>>> could benefit a
> >>>>>>>>>>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> It's
> >>>>>>> also
> >>>>>>>>>>>>> strange, in the
> >>>>>>>>>>>>>>>>> polling
> >>>>>>>>>>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> IOps,
> >>>>>>> but
> >>>>>>>>>>> on
> >>>>>>>>>>>>> the throttling
> >>>>>>>>>>>>>>>>> side
> >>>>>>>>>>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Write
> >>>>>>> IOps
> >>>>>>>>>>> are
> >>>>>>>>>>>>> usually much
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> like a
> >>>>>>> valid
> >>>>>>>>>>>>> use-case where
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> IOps
> >>>>>>> vs
> >>>>>>>>>>> Read
> >>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Since
> >>>>>>> this
> >>>>>>>>>>> only
> >>>>>>>>>>>>> supports KVM
> >>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> value
> >>>>>>> to at
> >>>>>>>>>>>>> least have the
> >>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> this
> >>>>>>> later
> >>>>>>>>>>>>> would be a lot of
> >>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> If
> >>>> a
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> doesn't
> >>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> write
> >>>>>>> you
> >>>>>>>>>>> can
> >>>>>>>>>>>>> always sum
> >>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Can
> >>>>>>> you
> >>>>>>>>>>>>> explain why you
> >>>>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> The
> >>>>>>>>>>> feature
> >>>>>>>>>>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> (1) set
> >>>>>>> the
> >>>>>>>>>>>>> maximum rate of
> >>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> (2)
> >>>>>>> change
> >>>>>>>>>>> the
> >>>>>>>>>>>>> maximum rate
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> (3)
> >>>>>>> limit
> >>>>>>>>>>> the
> >>>>>>>>>>>>> disk rate
> >>>>>>>>>>>>>> (total
> >>>>>>>>>>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> JIRA
> >>>>>>> ticket:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-
> 1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>> 1192
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> FS (I
> >>>>>>> will
> >>>>>>>>>>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> Merge
> >>>>>>>>>>> check
> >>>>>>>>>>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *
> >>>> Did
> >>>>>>> you
> >>>>>>>>>>>>> check the branch's
> >>>>>>>>>>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *
> >>>> Are
> >>>>>>>>>>> there
> >>>>>>>>>>>>> new dependencies
> >>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *
> >>>>>>> What
> >>>>>>>>>>>>> automated testing
> >>>>>>>>>>>>>> (unit
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Unit
> >>>>>>> tests
> >>>>>>>>>>> are
> >>>>>>>>>>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *
> >>>>>>> What
> >>>>>>>>>>>>> testing has been done
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> (1) set
> >>>>>>> the
> >>>>>>>>>>>>> bytes rate and
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> (2)
> >>>>>>> VM
> >>>>>>>>>>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>> deploy,
> >>>>>>>>>>> stop,
> >>>>>>>>>>>>> start, reboot,
> >>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> (3)
> >>>>>>>>>>> Volume
> >>>>>>>>>>>>> operations,
> >>>>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Attach,
> >>>>>>>>>>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> To
> >>>>>>> review
> >>>>>>>>>>> the
> >>>>>>>>>>>>> code, you can
> >>>>>>>>>>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Best
> >>>>>>>>>>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> [2]
> >>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> >>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> -
> >>>>>>> VM
> >>>>>>>>>>> Disk
> >>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Mike
> >>>>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *Senior
> >>>>>>>>>>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> e:
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> o:
> >>>>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> Advancing
> >>>>>>>>>>> the
> >>>>>>>>>>>>> way the world
> >>>>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Mike
> >>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Advancing
> >>>>>>> the
> >>>>>>>>>>> way
> >>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Mike
> >>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Advancing the
> >>>>>>>>>>> way
> >>>>>>>>>>>>> the world uses
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> >>>> the
> >>>>>>> way
> >>>>>>>>>>> the
> >>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>> CloudStack
> >>>>>>>>>>>>> Developer,
> >>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> >>>> the
> >>>>>>> way
> >>>>>>>>>>> the
> >>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>>>>>>>>> Developer, SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >>>> way
> >>>>>>> the
> >>>>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >>>> way
> >>>>>>> the
> >>>>>>>>>>> world
> >>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>>> way the
> >>>>>>>>>>> world
> >>>>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> >>>> the
> >>>>>>> world
> >>>>>>>>>>> uses
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>>> Developer,
> >>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> >>>>>>> world
> >>>>>>>>>>> uses
> >>>>>>>>>>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> >>>> world
> >>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>> world
> >>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >>>> uses
> >>>>>>> the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >>>> uses
> >>>>>>> the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >>>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >>>> uses
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *(tm)*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *(tm)*****
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*****
> >>>>>>
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>
> >>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>
> >>>>>> o: 303.746.7302****
> >>>>>>
> >>>>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *(tm)*****
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ****
> >>>>>>
> >>>>>> ** **
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*****
> >>>>>>
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>>>
> >>>>>> e: mike.tutkowski@solidfire.com****
> >>>>>>
> >>>>>> o: 303.746.7302****
> >>>>>>
> >>>>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *(tm)*****
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *(tm)*
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Yes, a vendor may hand a non-managed plugin. For example, on the secondary storage side, S3 and Swift don't support the concept of being managed -- an operator is required to create a bucket and point CloudStack at it. Another example would be a generic iSCSI device (e.g. OpenFiler, FreeNAS) which CloudStack can not directly manage, but can consume a LUN that an operator has created for it.
From an operational perspective, an administrator may elect to say, "Hey, I know you can manage that device for me, but it is has a wider use than CloudStack can manage. Let me handle it, and just provide the information CloudStack needs." In this scenario, the operator is accepting a greater management burden in exchange for control. I don't think many will elect to manage these devices themselves, but this ability will be extremely important for the edge cases that require it. Speaking as someone who has run cloud operations, I can testify to this need -- nothing aggravates you more than a system insisting on managing a device or service without the opportunity to opt-out
Thanks,
-John
On Jun 10, 2013, at 3:19 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> So, would a storage vendor write a non-CloudStack-managed plug-in and maybe
> just implement the snapshot feature or something?
>
> In this case, they'd have to set the hypervisor up ahead of time with a
> large portion of their storage, but they'd still get the vendor-snapshot
> ability.
>
> Just trying to undertand this more.
>
> Thanks
>
>
> On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com> wrote:
>
>> Edison,
>>
>> TL;DR Always give operators/system administrators the ability to manage
>> things themselves, if they like.
>>
>> As more manageable drivers emerge, I think it will be import to allow
>> operators to opt-out of CloudStack device management. Therefore, a driver
>> declares whether or not it can support device management (many, in fact
>> most, won't). For those drivers that do support management, allow
>> operators to opt-in/out of CloudStack management on a per device basis. To
>> support this notion, we need a flag on the storage_pool table and
>> associated classes to indicate whether or not the operator wants CloudStack
>> to manage the device.
>>
>> Does this explanation make more sense?
>>
>> Thanks,
>> -John
>>
>> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>> I don't understand why we need to add a "managed" in the UI and in
>> "storage_pool" table?
>>> AFAIK, what you guys trying to do is to enable
>> attachVolume/DettachVolume if the volume is created on Solidfire, right?
>>> If the volume is created on Solidfire already, by createAsync, then why
>> we can't just send out all the information about the volume(it's volumeTO
>> which will be returned by each storage driver) to hypervisor, then at the
>> hypervisor side, check the volumeTO, if the volumeTO is managed, then doing
>> whatever necessary operations(create SR, create VDI etc), then attach the
>> VDI to volume.
>>>
>>>
>>>> -----Original Message-----
>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>> Sent: Monday, June 10, 2013 11:10 AM
>>>> To: John Burwell
>>>> Cc: Edison Su; dev@cloudstack.apache.org
>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>
>>>> I believe we're on the same page, John.
>>>>
>>>> I would add a new parameter to the createStoragePool API command (let's
>>>> call it "managed" and it can be true or false). If you pass in "true",
>> but
>>>> the driver does not support being managed by CloudStack, an exception is
>>>> thrown by the driver. If you pass in "false", but the driver does not
>>>> support being managed outside of CloudStack, an exception is thrown by
>> the
>>>> driver. It is possible one could write a driver that can be managed both
>>>> inside or outside of CloudStack (the SolidFire driver requires being
>>>> managed only by CloudStack).
>>>>
>>>> This managed value would end up in the storage_pool table in a new
>> column
>>>> ("managed").
>>>>
>>>> An example usage: After the storage framework calls on the driver's
>>>> createAsync method, it will check to see if the driver is managed. If
>> the
>>>> driver is managed, the storage framework will invoke the proper (new)
>>>> command of the hypervisor in use to set up, say, the SR for the SAN
>> volume.
>>>>
>>>> There may be other cases when the isManaged() method is invoked on the
>>>> driver and - if true is returned - the storage framework sends a
>> particular
>>>> command to the hypervisor in use.
>>>>
>>>> ********************
>>>>
>>>> As far as reworking the default storage plug-in (which sends commands to
>>>> hypervisors) for 4.2, I would check with Edison on his schedule.
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> I want to make sure we are on the same page regarding isManaged. I see
>>>>> the addition of the following operations:
>>>>>
>>>>> * DataStoreDriver#supportsManagement() : boolean -- indicates whether
>>>> or
>>>>> not the driver support/implements the management functions
>>>>> * DataStore#isManaged() : boolean -- indicates whether or not a device
>> is
>>>>> managed by CloudStack
>>>>>
>>>>> Only DataStores associated with DataStoreDrivers that support
>>>> management
>>>>> would be able to have the management flag enabled. This behavior
>> allows
>>>>> operators to declare their intention to allow CloudStack to manage a
>>>>> device. This behavior would need to be exposed as flag on the HTTP
>> API.
>>>>> Additionally, this flag would only be settable when a device is
>>>>> defined/created in the system (i.e. once a device is managed by
>> CloudStack,
>>>>> you can't unmanaged it without dissociating it from CloudStack). In
>> the
>>>>> future, we may consider relaxing that rule, but, in the short term, it
>>>>> greatly decreases the alternative flows.
>>>>>
>>>>> In terms of removing the dependencies from Storage to Hypervisor, I
>>>> would
>>>>> really like to see this accomplished in 4.2. It will be much more
>>>>> difficult to address this issue post 4.2 because we will have more
>> plugins
>>>>> to address. I also see this issue as a critical architectural issues
>> that
>>>>> for the management server. I recommend we identify the dependencies
>>>> that
>>>>> need to be inverted, and see if we can divide the work across a few
>> more
>>>>> resources.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
>>>> <mi...@solidfire.com>
>>>>> wrote:
>>>>>
>>>>> Sounds good, Edison :)
>>>>>
>>>>> I can implement John's isManaged() logic and resubmit my code for
>> review.
>>>>>
>>>>> Perhaps in 4.3 I could address the direct-attach scenario. If it
>> requires
>>>>> some refactoring of the storage framework, then I could do that.
>>>>>
>>>>> Also, the default storage plug-in does have hypervisor logic in it in
>> 4.2.
>>>>> I recommend we leave this as is for 4.2 and it can be refactored in 4.3
>>>>> with the above work I mentioned.
>>>>>
>>>>> Are you OK with that, guys?
>>>>>
>>>>>
>>>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
>> wrote:
>>>>>
>>>>>> Ok, I am Ok with the way you are talking about in 4.2.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
>>>>>> *To:* dev@cloudstack.apache.org
>>>>>> *Cc:* Edison Su; John Burwell
>>>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Another thing we might want to consider for 4.2 is that the way the
>> code
>>>>>> is currently implemented in master, I don't think it's even possible
>> to
>>>>>> write a plug-in that connects storage directly to a VM because the
>>>>>> CloudStack storage framework will call the "attach" method of the
>>>>>> hypervisor in use and this method will fail because it assumes there
>> is
>>>>>> (talking Xen here) an SR existent and in this situation there isn't
>> one.*
>>>>>> ***
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> My thinking is that we should ignore the direct-attach-of-volume-to-VM
>>>>>> use case for 4.2. I think it will require too much of a rewrite at
>> this
>>>>>> point in development (fewer than three weeks left until code
>>>> freeze).****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> If the plug-in implements an isManaged() method (as John suggested),
>>>> the
>>>>>> storage framework can invoke the necessary commands on the
>>>> hypervisor in
>>>>>> use.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> For example, the createAsync method is called. The storage plug-in
>>>>>> creates a SAN volume and updates the CS DB. The storage framework
>>>> then asks
>>>>>> the driver if it's managed. If it is, the storage framework executes a
>>>>>> command against the hypervisor in use to have it create, say, an SR
>> (with
>>>> a
>>>>>> single VDI that takes up the entire SR). Now, when the storage
>>>> framework
>>>>>> executes the "attach" command against the hypervisor, this command
>> will
>>>>>> work, as well (because that logic assumes the prior existence of an
>> SR and
>>>>>> it will, in fact, be present).****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:****
>>>>>>
>>>>>> Hi Edison and John,****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> I wanted to point out something I thought was relevant to this
>>>>>> conversation.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> My plug-in supports both XenServer and ESX(i).****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> These two hypervisors handle their respective hypervisor data
>> structures
>>>>>> (XenServer SR or ESX(i) datastore) quite differently.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
>>>>>> when you issue a remove-SR command to XenServer.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> When you later want to access the contents of that data, it is still
>>>>>> safely stored on the LUN and can be accessed from XenServer (from the
>>>> same
>>>>>> or different cluster (what XenServer calls a resource pool)) by
>> creating a
>>>>>> new SR that is based on the same IQN and LUN.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> XenServer SRs exist at the cluster level.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
>>>>>> destroyed when you issue a remove-datastore command to ESX(i). In
>>>> other
>>>>>> words, the contents of the SAN LUN are destroyed.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ESX(i) datastores exist at the datacenter level.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> The XenServer behavior (not destroying the contents of the LUN when
>>>> the
>>>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
>> contents
>>>>>> of the LUN when the datastore is removed) is not desirable.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> I can get around the ESX(i) behavior by removing the referenced IQN
>>>> from
>>>>>> each host in the cluster when the CS storage framework notifies my
>> plug-
>>>> in
>>>>>> of a detach-volume event. When the CS volume is later attached again
>>>>>> (either to the same or a different ESX(i) cluster), I can have each
>> host in
>>>>>> the relevant ESX(i) cluster reference the IQN and the datastore will
>> be
>>>>>> back again.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> This sequence of events is critical for my plug-in to support both
>>>>>> environments:****
>>>>>>
>>>>>> *XenServer flow (events for Driver class):*****
>>>>>>
>>>>>> createAsync (called right before first attach) = creates SAN
>> volume****
>>>>>>
>>>>>> preAttachVolume = creates SR and creates VDI or just introduces SR****
>>>>>>
>>>>>> (CS volume gets attached and later detached)****
>>>>>>
>>>>>> postDetachVolume = removes SR (this does not delete data on SAN
>>>> volume)**
>>>>>> **
>>>>>>
>>>>>> deleteAsync (can be called when volume is not attached) = deletes the
>>>> SAN
>>>>>> volume****
>>>>>>
>>>>>> *VMware flow (events for Driver class):*****
>>>>>>
>>>>>> createAsync (called right before first attach) = creates SAN volume,
>>>>>> creates datastore, and creates VMDK file****
>>>>>>
>>>>>> preAttachVolume = adds iSCSI target to each host in cluster****
>>>>>>
>>>>>> (CS volume gets attached and later detached)****
>>>>>>
>>>>>> postDetachVolume = removes iSCSI target from each host in cluster****
>>>>>>
>>>>>> deleteAsync (can be called when volume is not attached) = deletes
>>>>>> datastore and deletes the SAN volume****
>>>>>>
>>>>>> ************************
>>>>>>
>>>>>> So, the way I have it written up here, the driver is talking to the
>>>>>> hypervisor. As John has pointed out, this may not be ideal.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Whatever we decide on, it needs to be able to support this kind of
>> flow.
>>>>>> :)****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> We could go John's route, where the driver says if it's managed or
>> not.**
>>>>>> **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> If it is, then the storage framework could send the necessary
>> messages to
>>>>>> the hypervisor in question at the right times (just migrate the
>> hypervisor
>>>>>> logic above from the driver to the right parts of the storage
>> framework).
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> This is fine, except that it does not resolve Edison's use case of
>>>>>> desiring to have the storage LUN potentially connected directly to a
>> VM.*
>>>>>> ***
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> I'm not sure we should design around that use case, though. Is it
>> common
>>>>>> enough? Has anyone asked for it? Is it even desirable? How are usage
>>>>>> statistics gathered for such volumes? It's nice to offer the
>> flexibility,
>>>>>> but is it a good idea in this case?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Thanks!****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
>> wrote:**
>>>>>> **
>>>>>>
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>> Sent: Friday, June 07, 2013 4:08 PM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>
>>>>>>> Hi John,
>>>>>>>
>>>>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
>>>>>> volume
>>>>>>> to use it. Same story for VMware: ESX(i) must layer a datastore on
>> top
>>>>>> of
>>>>>>> the iSCSI volume to use it. I look at it like they are layering a
>>>>>> clustered
>>>>>>> file system on the SAN volume so the hypervisors can share access to
>>>> the
>>>>>>> contents of the volume.
>>>>>>>
>>>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called, the
>>>>>> SR or
>>>>>>> datastore has already been created (usually manually by an admin).
>>>>>>>
>>>>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
>>>>>>> environment where each CloudStack volume that a user creates is
>>>> mapped
>>>>>>> to a
>>>>>>> single SAN volume (via the SR).
>>>>>>>
>>>>>>> The question comes down to who should allocate the SR dynamically.
>>>>>>>
>>>>>>> We could have the storage framework ask the storage plug-in if it is
>>>>>>> managed. If it is, then the storage framework could send a message to
>>>>>> the
>>>>>>> hypervisor in question to create (let's talk Xen here) the SR ahead
>> of
>>>>>>> time. Then, when the storage framework next sends the attach-volume
>>>>>>> command
>>>>>>> to the hypervisor, it should work without changes to that
>> attach-volume
>>>>>>> logic (because - from the point of view of the attach logic - the SR
>> is
>>>>>>> already existent, as expected).
>>>>>>>
>>>>>>> Now, as Edison has pointed out, this limits the power of the storage
>>>>>>> plug-in. A storage plug-in in this model cannot directly attach
>> storage
>>>>>> to
>>>>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We need
>>>> to
>>>>>>> make a call on that.
>>>>>>
>>>>>>
>>>>>> Yes, I think we need to address this usage case, and won't limit the
>>>>>> power of the storage plug-in.
>>>>>> John, what's your idea to fit this usage case into your abstraction?
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> If we want to give more power to the storage vendor, we would have to
>>>>>>> have
>>>>>>> the storage framework call into the storage plug-in to send the
>>>>>> appropriate
>>>>>>> attach commands to the hypervisor.
>>>>>>>
>>>>>>> Let's discuss and come to a consensus (or at least agree on a path)
>> as
>>>>>> soon
>>>>>>> as we can. :)
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> My understanding is the Xen expects an SR structure on all ISCSI
>>>>>> devices
>>>>>>>> -- at least that is how I read the code in your patch. Is my
>>>>>> understanding
>>>>>>>> correct? If so, the Xen plugin should be able to query the storage
>>>>>> device
>>>>>>>> to determine the presence of the SR structure and create it if it
>>>>>> does not
>>>>>>>> exist. Am I missing something in the implementation that makes that
>>>>>> type
>>>>>>>> of implementation impossible?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yeah, if a storage vendor wanted to attach a volume directly to a
>>>>>> VM in
>>>>>>>> 4.2
>>>>>>>>> today, it would probably fail because the attach-volume logic
>>>>>> assumes
>>>>>>> the
>>>>>>>>> existence of the necessary hypervisor data structure (ex. SR on
>>>>>> Xen).
>>>>>>>>>
>>>>>>>>> If we wanted to enable such an attach, we could do it the way
>>>> Edison
>>>>>>>>> suggests.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>
>>>>>>>>>>> As we only have three weeks until feature freeze, we should
>>>> come
>>>>>> to
>>>>>>> a
>>>>>>>>>>> consensus on this design point as soon as possible.
>>>>>>>>>>>
>>>>>>>>>>> Right now, if the storage framework asks my driver it is is
>>>>>> managed, it
>>>>>>>>>>> will say 'yes.' This means the framework will tell the driver to
>>>>>>>> perform
>>>>>>>>>>> its management activities. This then means the driver will call
>>>>>> into
>>>>>>>> the
>>>>>>>>>>> host (it doesn't know which hypervisor, by the way) to perform
>>>> the
>>>>>>>>>> activity
>>>>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
>>>>>>>>>>>
>>>>>>>>>>> The driver doesn't know which hypervisor it's talking to, it just
>>>>>>>> sends a
>>>>>>>>>>> message to the host to perform the necessary pre-attach work.
>>>>>>>>>>
>>>>>>>>>> Could we just expose a method like
>>>> "attachVolume/dettachVolume" on
>>>>>>> the
>>>>>>>>>> PrimaryDataStoreDriver driver?
>>>>>>>>>> For most of cases, the implementation of each driver would just
>>>>>> send
>>>>>>>>>> attachvolumecommand/dettachvolumecmd to hypervisor(we can
>>>> put
>>>>>>> the
>>>>>>>>>> implementation in a base class, so that can be shared by all of
>>>>>> this
>>>>>>>>>> drivers), and in each hypervisor resource code, which may just
>> call
>>>>>>>>>> hypervisor's api to attach the volume to VM, while for certain
>>>>>> storage,
>>>>>>>>>> like SolidFire, may need to create a SR first, and create volume
>>>>>> on it,
>>>>>>>>>> then call hypervisor's API to attach volume to VM.
>>>>>>>>>> While for some other storage vendor, may want to bypass
>>>> hypervisor
>>>>>>>> during
>>>>>>>>>> attaching volume, so inside driver's attachvolume implementation,
>>>>>> the
>>>>>>>>>> driver can do some magic, such as, directly talk to an agent
>>>>>> inside VM
>>>>>>>>>> instance, then create a disk inside VM.
>>>>>>>>>>
>>>>>>>>>> How do you guys think?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> How's about this:
>>>>>>>>>>>>>
>>>>>>>>>>>>> The driver can implement an isManaged() method. The
>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>> can
>>>>>>>>>>>>> call into the driver to see if its managed. If it is, the
>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>> (which is responsible for calling into the hypervisor to attach
>>>>>> the
>>>>>>>>>> disk)
>>>>>>>>>>>>> can call into the hypervisor to create the necessary hypervisor
>>>>>> data
>>>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
>>>>>>>>>>>>
>>>>>>>>>>>> The problem here is that storage vendor may work differently
>>>> with
>>>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while
>>>>>> maybe
>>>>>>>>>> other
>>>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the LUN
>>>>>>> directly
>>>>>>>>>> to
>>>>>>>>>>>> VM instance, see the discuss(
>>>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>>>>>>>>>>>
>>>>>>>
>>>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>>>>>>>>>>> %3E
>>>>>>>>>>>> ).
>>>>>>>>>>>> So I would let storage provider to implement attach disk to VM,
>>>>>>>>>> instead of
>>>>>>>>>>>> implemented by cloudstack itself.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> If that's what you're going for, that works for me. By the way,
>>>>>>>>>> Edison's
>>>>>>>>>>>>> default storage plug-in (which handles the default storage
>>>>>> behavior
>>>>>>>>>> in
>>>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that
>>>>>> talks to
>>>>>>>>>>>>> hypervisors. You might want to contact him and inform him of
>>>>>> your
>>>>>>>>>>>> concerns
>>>>>>>>>>>>> or that logic (as is) will make it to production.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please let me know if what I wrote in above (for my solution)
>>>>>> is OK
>>>>>>>>>> with
>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please see my responses in-line below.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I build
>>>>>> up
>>>>>>>>>> a
>>>>>>>>>>>> patch
>>>>>>>>>>>>>>> file, but this is the gist of what I've done:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * During a volume-attach operation, after
>>>> VolumeManagerImpl
>>>>>>> tells
>>>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a volume, I
>>>> have
>>>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
>>>> driver if
>>>>>> it
>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
>>>>>>>>>> whatever
>>>>>>>>>>>>>>> activity is required. In my case, this includes sending a
>>>>>>>>>> message to
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> host where the VM is running to have, say XenServer, add a
>>>>>>>>>> storage
>>>>>>>>>>>>>>> repository (based on the IP address of the SAN, the IQN of
>>>> the
>>>>>>>>>> SAN
>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
>>>>>> can
>>>>>>>>>> on
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> storage repository). After this, the normal attach-volume
>>>>>>>>>> message is
>>>>>>>>>>>> sent
>>>>>>>>>>>>>>> to the host by VolumeManagerImpl.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There should be **no** code from a storage driver to a
>>>>>> hypervisor.
>>>>>>>>>> I
>>>>>>>>>>>>>> apologize for the repetition, but we simply can not have
>>>>>> hypervisor
>>>>>>>>>>>>>> specific code in the storage layer. The circular dependencies
>>>>>>>>>> between
>>>>>>>>>>>> the
>>>>>>>>>>>>>> two layers are not sustainable in the long term. Either the
>>>>>>>>>>>> VirtualManager
>>>>>>>>>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>>>>>>>>>>> coordinate
>>>>>>>>>>>>>> volume creation and then populating the SR. Ideally, we can
>>>>>>>>>>>> generalize the
>>>>>>>>>>>>>> process flow for attaching volumes such that the Xen
>>>> hypervisor
>>>>>>>>>> plugin
>>>>>>>>>>>>>> would only implement callbacks to perform the attach action
>>>> and
>>>>>>>>>> create
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver should
>>>>>> only be
>>>>>>>>>>>>>> allocating space and providing information about contents
>>>> (e.g.
>>>>>>>>>> space
>>>>>>>>>>>>>> available, space consumed, streams to a URI, file handle for a
>>>>>> URI,
>>>>>>>>>>>> etc)
>>>>>>>>>>>>>> and capabilities.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * The reverse is performed for a detach-volume command.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in my
>>>>>> driver.
>>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>> default driver simply does a "return false;". We could add a
>>>>>> new
>>>>>>>>>>>>>> parameter
>>>>>>>>>>>>>>> to the createStoragePool API command, if we want, to
>>>> remove
>>>>>>> the
>>>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>>>> return values in the drivers (although my driver will
>> probably
>>>>>>>>>> just
>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>> this parameter and always return true since it wouldn't make
>>>>>>>>>> sense
>>>>>>>>>>>> for it
>>>>>>>>>>>>>>> to ever return false). We'd need another column in the
>>>>>>>>>> storage_pool
>>>>>>>>>>>>> table
>>>>>>>>>>>>>>> to store this value.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, I think we should have a parameter added to the
>>>>>>>>>>> createStoragePool
>>>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
>>>>>> configured
>>>>>>>>>> for
>>>>>>>>>>>>>> management when their underlying drivers support it. To
>>>>>> simplify
>>>>>>>>>>>> things,
>>>>>>>>>>>>>> this flag should only be mutable when the DataStore is
>>>>>> created. It
>>>>>>>>>>>> would be
>>>>>>>>>>>>>> a bit crazy to take a DataStore from managed to unmanaged
>>>> after
>>>>>>>>>>>> creation.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
>>>> working
>>>>>>>>>> GUI
>>>>>>>>>>>> for
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> to review (in that document), if you'd like to check it out.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I would like the UIs of two features reviewed together to
>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>> consistency across the concepts of hypervisor throttled
>>>> IOPs
>>>>>>>>>> and
>>>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the potential for
>>>>>>>>>>>> confusion,
>>>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these features will
>>>>>>>>>> help
>>>>>>>>>>>>>>>>> minimize any potential confusion.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will work
>>>>>> itself
>>>>>>>>>> if
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
>> hypervisor
>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
>>>> sent
>>>>>>> to
>>>>>>>>>>> you
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> other day to see how the GUI looks for provisioned
>>>> storage
>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-mail
>> list
>>>>>>>>>> and we
>>>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
>>>>>>> Offering
>>>>>>>>>>>> dialog.
>>>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS, so they
>>>>>>> should
>>>>>>>>>>> be
>>>>>>>>>>>>> able
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say, only
>>>>>> use
>>>>>>>>>> Max
>>>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>> These fields are optional and can be ignored for storage
>>>>>> that
>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
>>>>>> the
>>>>>>>>>>>> admin can
>>>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and
>>>> Burst
>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature
>>>> or
>>>>>>>>>> mine,
>>>>>>>>>>>> as
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> decided by the admin).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm not sure what we can do about these two features
>>>>>>>>>> expressing
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> speed
>>>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express the
>>>> IOPS
>>>>>> for
>>>>>>>>>>>> QoS in
>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
>>>>>> read/write
>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
>>>>>>>>>>> Storage/Hypervisor
>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the behavior
>>>> of
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
>>>>>>>>>> following
>>>>>>>>>>>>>> issues
>>>>>>>>>>>>>>>>>>> outstanding:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are expressed as
>>>> total
>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
>>>> to
>>>>>>>>>>> storage
>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>>>>>>>>>>> 3. How should usage data be captured for throttled and
>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>> that will permit providers to discriminate these
>>>>>> guaranteed
>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
>>>>>> provisioned
>>>>>>>>>>> IOPS
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
>>>>>> hypervisor
>>>>>>>>>>>> throttled
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
>>>>>> solve
>>>>>>>>>>>> items 1
>>>>>>>>>>>>>>>>> and 2.
>>>>>>>>>>>>>>>>>>> Since the two facilities would not be permitted to
>>>> operate
>>>>>>>>>>>> together,
>>>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I think item
>>>> 3
>>>>>>>>>> can be
>>>>>>>>>>>>>>>>> resolved
>>>>>>>>>>>>>>>>>>> with an additional flag or two on the usage records. As
>>>>>> for
>>>>>>>>>>>> Item 4,
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>>>> not familiar with how these two enhancements are
>>>>>>> depicted in
>>>>>>>>>>> the
>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
>>>> enhancements
>>>>>>> for
>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>>>>>>> <us...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> John and Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
>>>>>>> 2780/2729,
>>>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>>>>>>>>>>> CLOUDSTACK-
>>>>>>>>>>>>> 2875 ) this
>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
>>>> and
>>>>>>>>>> ask
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> second
>>>>>>>>>>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
>>>> <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I believe I understand where you're going with this,
>>>>>> John,
>>>>>>>>>> and
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager determining
>>>>>>> whether or
>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
>>>> but an
>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
>>>>>>>>>> resources
>>>>>>>>>>>> for
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
>>>>>>> method
>>>>>>>>>>>>> added to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
>>>>>>>>>> then the
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or whatever
>>>> name)
>>>>>>>>>>> method
>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
>>>> well)
>>>>>>> and
>>>>>>>>>>> this
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to create,
>>>> say
>>>>>>> on
>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>> SR?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>>>>>>>>>>>>> DataStoreDriver.
>>>>>>>>>>>>>>>>> Since
>>>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary
>>>> storage,
>>>>>>> I
>>>>>>>>>>>> would
>>>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
>>>>>>> straw
>>>>>>>>>>> man
>>>>>>>>>>>>> term
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> something other than volume) methods to
>>>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a
>>>> mutable
>>>>>>>>>>> property
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> DataStore
>>>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the underlying
>>>>>>>>>>> DataStoreDriver
>>>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
>>>>>>> manageability
>>>>>>>>>> of
>>>>>>>>>>>>>> devices.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
>>>> Xen
>>>>>>>>>> plugin
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> responsible for composing DataStore/Volume
>>>> methods
>>>>>>> to
>>>>>>>>>>>>> create any
>>>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR. There
>>>>>> should
>>>>>>>>>> be no
>>>>>>>>>>>>>>>>>>>>> dependencies
>>>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
>>>>>>>>>> earlier,
>>>>>>>>>>>> such
>>>>>>>>>>>>>>>>>>>>> circular
>>>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
>>>> unmaintainable
>>>>>>> mess.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page with
>>>> you.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>>>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
>>>> layer
>>>>>>> that
>>>>>>>>>>>>>> supports n
>>>>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors of m
>>>>>>>>>> hypervisors.
>>>>>>>>>>>>>> Such
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
>>>> untestable
>>>>>>>>>>> beast.
>>>>>>>>>>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven to
>>>> evolve
>>>>>>> the
>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager determining
>>>>>>> whether
>>>>>>>>>>> or
>>>>>>>>>>>>> not a
>>>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
>>>> but
>>>>>>> an
>>>>>>>>>>> actual
>>>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
>>>>>>>>>> resources
>>>>>>>>>>>> for
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
>>>>>> needs
>>>>>>>>>> to
>>>>>>>>>>>>>> indicate
>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
>>>> Finally, I
>>>>>>>>>> think
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to elect to
>>>>>> have
>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
>>>> capable
>>>>>>>>>>> managing
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
>>>>>>> unmanaged).
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume should be
>>>> done
>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed by the
>>>>>>> Storage
>>>>>>>>>>>>> layer to
>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
>>>> create a
>>>>>>>>>> file,
>>>>>>>>>>>> etc).
>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>>>>>>>>>>>> hypervisor, as
>>>>>>>>>>>>>>>>> such,
>>>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation. From
>>>> my
>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>> nothing
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
>>>> content.
>>>>>>> From
>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides the
>>>> means
>>>>>>> to
>>>>>>>>>>>> query
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> support the interpretation of the content by
>>>> higher-
>>>>>>> level
>>>>>>>>>>>>> layers
>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
>>>>>>> composition
>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on the
>>>>>>> Volume
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> protocol
>>>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
>>>>>>> implemented
>>>>>>>>>> in
>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>> patch,
>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>>>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
>>>> the
>>>>>>>>>>>>> hypervisor.
>>>>>>>>>>>>>> Before
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if we're
>>>>>>> dealing
>>>>>>>>>>> with
>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>> (or
>>>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
>>>>>> send a
>>>>>>>>>>>> "Create
>>>>>>>>>>>>>>>>> SR"
>>>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
>>>>>>> would
>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> proceed
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same and
>>>> we just
>>>>>>>>>> add
>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called for this
>>>>>>>>>> particular
>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer today that
>>>>>>> outlines
>>>>>>>>>>> how
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will probably
>>>> be
>>>>>> of
>>>>>>>>>> use
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> you, as
>>>>>>>>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you (you
>>>> should
>>>>>>> have
>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
>>>>>>> thoughts,
>>>>>>>>>>> but
>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
>>>>>> solidfire,
>>>>>>>>>>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
>>>> in a
>>>>>>>>>>>>> coherent
>>>>>>>>>>>>>>>>> form
>>>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
>>>> introduces
>>>>>>> the
>>>>>>>>>>>> notion
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
>>>> introduces
>>>>>>>>>> the
>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages the
>>>> plug-in
>>>>>>> was
>>>>>>>>>>>>>>>>> incomplete,
>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
>>>> spending on
>>>>>>>>>> this.
>>>>>>>>>>>> :)
>>>>>>>>>>>>> Do
>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
>>>> accomplish
>>>>>>>>>>> what
>>>>>>>>>>>>> you're
>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
>>>> <
>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
>>>> introduces
>>>>>>> the
>>>>>>>>>>>> notion
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
>>>> management
>>>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>>>> (i.e.
>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
>>>> it
>>>>>>>>>>> managed
>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
>>>> with
>>>>>>> a
>>>>>>>>>>>>> semantic
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
>>>> managed.
>>>>>>> I
>>>>>>>>>>> also
>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
>>>> from
>>>>>>> the
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
>>>> Burwell <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
>>>>>>> questions
>>>>>>>>>>> for
>>>>>>>>>>>>> me
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
>>>> LUNs
>>>>>>> and
>>>>>>>>>>> SAN
>>>>>>>>>>>>>> volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
>>>> LUN
>>>>>>> (LUN
>>>>>>>>>>> 0).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
>>>> Volumes
>>>>>>> and
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
>>>> management
>>>>>>>>>>> server or
>>>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
>>>> SAN
>>>>>>>>>>> volume
>>>>>>>>>>>>> (same
>>>>>>>>>>>>>> as a
>>>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server (via
>>>> the
>>>>>>> plug-in)
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> first
>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
>>>> hypervisor.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
>>>> in,
>>>>>>> but
>>>>>>>>>>>> still
>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
>>>> already
>>>>>>>>>>> today
>>>>>>>>>>>>>> (prior
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
>>>> and -
>>>>>>> in
>>>>>>>>>>> this
>>>>>>>>>>>>> case -
>>>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
>>>> While
>>>>>>> you
>>>>>>>>>>> can do
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
>>>> management
>>>>>>> server
>>>>>>>>>>> or
>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
>>>> SAN
>>>>>>>>>>> volumes
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
>>>> in).
>>>>>>> There
>>>>>>>>>>> is
>>>>>>>>>>>>> no
>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
>>>> SAN
>>>>>>>>>>> volume
>>>>>>>>>>>>> and a
>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
>>>> that
>>>>>>>>>>>>> supports
>>>>>>>>>>>>>> true
>>>>>>>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
>>>> are
>>>>>>>>>> related
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
>>>>>>> multiple
>>>>>>>>>>> VMs
>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
>>>> storage."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
>>>> map a
>>>>>>>>>>> single
>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
>>>> have
>>>>>>>>>>> multiple
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
>>>>>>> guarantee
>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
>>>>>>> volume
>>>>>>>>>>> house
>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
>>>> today
>>>>>>>>>>> without
>>>>>>>>>>>>> any of
>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
>>>>>>> volume
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
>>>> created."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
>>>>>>> concerns me
>>>>>>>>>>> --
>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
>>>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
>>>> is
>>>>>>> today
>>>>>>>>>>>> only
>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>> (as
>>>>>>>>>>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
>>>> have
>>>>>>> not
>>>>>>>>>>>>> PreSetup an
>>>>>>>>>>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>>>> volume
>>>>>>> from
>>>>>>>>>>>>> VMware, it
>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>>>> volume
>>>>>>> from
>>>>>>>>>>> KVM,
>>>>>>>>>>>>> it uses
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
>>>> thinking
>>>>>>>>>>> here
>>>>>>>>>>>>> was to
>>>>>>>>>>>>>>>>> make a
>>>>>>>>>>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
>>>> ability
>>>>>>>>>>> Edison
>>>>>>>>>>>>> added
>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
>>>> more
>>>>>>>>>>> specificy,
>>>>>>>>>>>>> though:
>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
>>>> Mike
>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
>>>> shouldn't
>>>>>>> know
>>>>>>>>>>> or
>>>>>>>>>>>>> care
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
>>>> answer
>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
>>>> the
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
>>>> the
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>> knows
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
>>>>>>> constraint
>>>>>>>>>>>>> and just
>>>>>>>>>>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
>>>> way
>>>>>>> for
>>>>>>>>>>> the
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
>>>>>>> perform
>>>>>>>>>>> the
>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
>>>> John
>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
>>>> be
>>>>>>> able to
>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>> (e.g. the
>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
>>>> operator has
>>>>>>>>>>> chosen
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
>>>> reasoning
>>>>>>>>>>> make
>>>>>>>>>>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
>>>> creating a
>>>>>>> SAN
>>>>>>>>>>>>> volume via
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
>>>> hypervisor)
>>>>>>> would
>>>>>>>>>>>>> work unless
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
>>>> say,
>>>>>>> an
>>>>>>>>>>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
>>>> of
>>>>>>>>>>>>> CloudStack, an
>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
>>>> and
>>>>>>>>>>> exposes it
>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
>>>> over
>>>>>>> to
>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
>>>> LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
>>>> to my
>>>>>>>>>>>>> changes, we
>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
>>>> the
>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
>>>> maybe
>>>>>>> have
>>>>>>>>>>> the
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
>>>>>>> support
>>>>>>>>>>> from
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
>>>> specifically
>>>>>>> to
>>>>>>>>>>> create
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> SR
>>>>>>>>>>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
>>>> management
>>>>>>> plugin
>>>>>>>>>>>>> should
>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
>>>> determine
>>>>>>>>>>> available
>>>>>>>>>>>>> for
>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>>>>>>>>>>> interrogation of
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
>>>> device
>>>>>>> itself
>>>>>>>>>>>>> shouldn't
>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
>>>> simply be
>>>>>>> able
>>>>>>>>>>> to
>>>>>>>>>>>>>> answer
>>>>>>>>>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
>>>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
>>>> SolidFire
>>>>>>>>>>> driver
>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
>>>>>>> done, a
>>>>>>>>>>> new
>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
>>>> Offering
>>>>>>>>>>> that
>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
>>>> Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
>>>>>>> CloudStack
>>>>>>>>>>>>> volume. This
>>>>>>>>>>>>>>>>> leads
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
>>>>>>> CloudStack
>>>>>>>>>>>>> volume to a
>>>>>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
>>>> framework
>>>>>>> calling
>>>>>>>>>>> the
>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
>>>> case, a
>>>>>>>>>>> SAN).
>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
>>>> applicable
>>>>>>> data
>>>>>>>>>>>>> (like the
>>>>>>>>>>>>>>>>> IQN of
>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
>>>> invoked if
>>>>>>> the
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
>>>> attach,
>>>>>>> the
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
>>>> and
>>>>>>> the
>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
>>>> is run
>>>>>>> and
>>>>>>>>>>>>> detects
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
>>>> the
>>>>>>> form
>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
>>>> PM,
>>>>>>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
>>>> sync on
>>>>>>>>>>>>> terminology,
>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
>>>> physical
>>>>>>>>>>>>> allocation on
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
>>>> try to
>>>>>>>>>>> mean
>>>>>>>>>>>>> 'SAN
>>>>>>>>>>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
>>>> end
>>>>>>> user
>>>>>>>>>>> can
>>>>>>>>>>>>> make in
>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
>>>> PM,
>>>>>>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
>>>> very
>>>>>>> well
>>>>>>>>>>> make
>>>>>>>>>>>>> sense, but
>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
>>>> Edison
>>>>>>> in on
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> conversation
>>>>>>>>>>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>>>>>>>>>>> approach I
>>>>>>>>>>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
>>>> PM,
>>>>>>>>>>> John
>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
>>>> combining
>>>>>>> two
>>>>>>>>>>>>> distinct
>>>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
>>>>>>> protocols.
>>>>>>>>>>> In
>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>> cases, we
>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
>>>>>>> system to
>>>>>>>>>>>>>> create/delete
>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
>>>> requires
>>>>>>> the
>>>>>>>>>>>>> volume to
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
>>>>>>> context.
>>>>>>>>>>> To
>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
>>>> these
>>>>>>>>>>>>> definitions,
>>>>>>>>>>>>>>>>> refers to
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
>>>>>>> correct?
>>>>>>>>>>>>> Minimally,
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
>>>> storage
>>>>>>> device
>>>>>>>>>>> to
>>>>>>>>>>>>> move bits
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>>>>>>>>>>> Optionally, a
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
>>>>>>> Therefore,
>>>>>>>>>>>>> we can have
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
>>>> SR,
>>>>>>> and
>>>>>>>>>>> we
>>>>>>>>>>>>> can have a
>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
>>>> CloudStack is
>>>>>>>>>>> capable
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> allocating
>>>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
>>>>>>> CloudStack
>>>>>>>>>>> may
>>>>>>>>>>>>> be able
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
>>>> leave it
>>>>>>>>>>>>> unmanaged by
>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
>>>> and
>>>>>>> the
>>>>>>>>>>>>> operator has
>>>>>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
>>>> CloudStack).
>>>>>>> Does
>>>>>>>>>>> my
>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
>>>> above
>>>>>>> are
>>>>>>>>>>>>> reasonable, it
>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
>>>>>>> concerns
>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
>>>>>>> attached
>>>>>>>>>>> driver
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
>>>> and
>>>>>>>>>>> establish
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
>>>> separate
>>>>>>> that
>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> associated
>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
>>>>>>> hypervisor
>>>>>>>>>>>>> drivers can
>>>>>>>>>>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
>>>>>>> encounter
>>>>>>>>>>> a
>>>>>>>>>>>>> device
>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
>>>>>>> exposed
>>>>>>>>>>> by
>>>>>>>>>>>>> the driver to
>>>>>>>>>>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
>>>> make
>>>>>>>>>>> sense,
>>>>>>>>>>>>> then we can
>>>>>>>>>>>>>>>>> sit
>>>>>>>>>>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
>>>>>>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
>>>>>>> between
>>>>>>>>>>> the
>>>>>>>>>>>>> current
>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
>>>> has to
>>>>>>> go in
>>>>>>>>>>> and
>>>>>>>>>>>>> create
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
>>>> point in
>>>>>>> time,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
>>>>>>> allocated).
>>>>>>>>>>>>> Future
>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
>>>> was
>>>>>>> created
>>>>>>>>>>>>> behind the
>>>>>>>>>>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
>>>> has
>>>>>>> to
>>>>>>>>>>> go in
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
>>>> and
>>>>>>> delete
>>>>>>>>>>>>> volumes on
>>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
>>>> enabled via
>>>>>>> the
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
>>>> volume
>>>>>>> that
>>>>>>>>>>> was
>>>>>>>>>>>>> created,
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
>>>>>>> volume.
>>>>>>>>>>> After
>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>> done,
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
>>>> question is
>>>>>>> called.
>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
>>>>>>> volume
>>>>>>>>>>> was
>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>>> 3:21
>>>>>>> PM,
>>>>>>>>>>> Mike
>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
>>>> so I
>>>>>>> should
>>>>>>>>>>>>> probably
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
>>>> that is
>>>>>>>>>>> pre-
>>>>>>>>>>>>> allocated
>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
>>>> a
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
>>>> XenServer
>>>>>>> SR),
>>>>>>>>>>> then
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
>>>> CloudStack.
>>>>>>> Disks
>>>>>>>>>>> (VDI)
>>>>>>>>>>>>> are later
>>>>>>>>>>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
>>>> needed. In
>>>>>>>>>>> these
>>>>>>>>>>>>> cases, the
>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>>>> structure is
>>>>>>>>>>> already
>>>>>>>>>>>>> in place
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
>>>>>>> structure
>>>>>>>>>>> to
>>>>>>>>>>>>> the VM in
>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
>>>> Storage
>>>>>>> that is
>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
>>>> this
>>>>>>> could
>>>>>>>>>>> be a
>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>> (not
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
>>>> hypervisor
>>>>>>> data
>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>> must be
>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
>>>>>>> because
>>>>>>>>>>> these
>>>>>>>>>>>>> types of
>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
>>>> data
>>>>>>>>>>>>> structure by an
>>>>>>>>>>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
>>>>>>> XenServer
>>>>>>>>>>> for
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
>>>> to the
>>>>>>> VM
>>>>>>>>>>> in
>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>>> 3:13
>>>>>>> PM,
>>>>>>>>>>>>> John Burwell
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>>>>>>> implementation of
>>>>>>>>>>> the
>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
>>>> which
>>>>>>> why I
>>>>>>>>>>> ask
>>>>>>>>>>>>> about the
>>>>>>>>>>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
>>>> what is
>>>>>>> the
>>>>>>>>>>>>> definition
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
>>>> PM,
>>>>>>>>>>> Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
>>>> iSCSI
>>>>>>> type
>>>>>>>>>>> is
>>>>>>>>>>>>>> uniquely
>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
>>>> Primary
>>>>>>>>>>> Storage
>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>> directly
>>>>>>>>>>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
>>>> skip the
>>>>>>>>>>> step of
>>>>>>>>>>>>> going
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
>>>> based
>>>>>>> on
>>>>>>>>>>> that
>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>> target as
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
>>>> only
>>>>>>>>>>>>> supported for
>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
>>>> must
>>>>>>> first go
>>>>>>>>>>> to
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
>>>> what
>>>>>>> RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>>>> at
>>>>>>> 2:13
>>>>>>>>>>> PM,
>>>>>>>>>>>>> John
>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
>>>> the
>>>>>>> code,
>>>>>>>>>>> what
>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
>>>> Why
>>>>>>> isn't
>>>>>>>>>>> RBD
>>>>>>>>>>>>> considered
>>>>>>>>>>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>>>> 3:46
>>>>>>> PM,
>>>>>>>>>>> Mike
>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
>>>>>>> storage is
>>>>>>>>>>>>> defined in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>>>>>>>>>>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
>>>> //
>>>>>>> local
>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> NetworkFilesystem(true),
>>>>>>>>>>> //
>>>>>>>>>>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
>>>>>>> shared
>>>>>>>>>>> LUN,
>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
>>>> e.g.,
>>>>>>> ZFS
>>>>>>>>>>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
>>>> iso
>>>>>>> image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
>>>>>>> XenServer
>>>>>>>>>>> local
>>>>>>>>>>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
>>>>>>> VMware
>>>>>>>>>>> VMFS
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
>>>> for
>>>>>>>>>>>>> XenServer, Storage
>>>>>>>>>>>>>>>>> Pool
>>>>>>>>>>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
>>>>>>> XenServer
>>>>>>>>>>> local
>>>>>>>>>>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
>>>>>>> dynamic,
>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> StoragePoolType(boolean
>>>>>>>>>>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
>>>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
>>>>>>> isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013 at
>>>>>>> 1:41
>>>>>>>>>>> PM,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
>>>> let's say
>>>>>>>>>>> another
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> company
>>>>>>>>>>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
>>>> leverage its
>>>>>>>>>>> Quality
>>>>>>>>>>>>> of
>>>>>>>>>>>>>> Service
>>>>>>>>>>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>> storage, as
>>>>>>> well.
>>>>>>>>>>>>> They would
>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
>>>> made
>>>>>>> the
>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>> changes to
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
>>>> plug-
>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013
>>>>>>> at
>>>>>>>>>>> 1:39
>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
>>>> imply
>>>>>>> the
>>>>>>>>>>>>> XenServer code
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
>>>>>>> attach
>>>>>>>>>>> logic is
>>>>>>>>>>>>> now aware
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
>>>> an
>>>>>>>>>>>>> implementation of
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
>>>> to 4.2
>>>>>>>>>>> with
>>>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
>>>> a
>>>>>>> new
>>>>>>>>>>>>> framework that
>>>>>>>>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
>>>>>>> dynamically.
>>>>>>>>>>>>> However, when I
>>>>>>>>>>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
>>>> realized
>>>>>>> that
>>>>>>>>>>> it
>>>>>>>>>>>>> was not
>>>>>>>>>>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
>>>> CloudStack
>>>>>>> could
>>>>>>>>>>> do
>>>>>>>>>>>>> with these
>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
>>>>>>> recognize
>>>>>>>>>>> this
>>>>>>>>>>>>> new type of
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>>>>>>> structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013
>>>>>>> at
>>>>>>>>>>> 1:28
>>>>>>>>>>>>> PM, John
>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
>>>> odd
>>>>>>> to me
>>>>>>>>>>>>> that any
>>>>>>>>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
>>>> care
>>>>>>>>>>> about
>>>>>>>>>>>>> details. I
>>>>>>>>>>>>>>>>> expect
>>>>>>>>>>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
>>>> of
>>>>>>>>>>> operations
>>>>>>>>>>>>> that can be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> implementations to
>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>> space/create
>>>>>>>>>>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
>>>> this
>>>>>>>>>>>>> dependency, we are
>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
>>>> will
>>>>>>> make
>>>>>>>>>>> the
>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
>>>> I
>>>>>>>>>>>>> understanding that the
>>>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>>>>>>>>>>>>> CitrixResourceBase
>>>>>>>>>>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>>>>> 2013 at
>>>>>>>>>>> 3:12
>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
>>>> this
>>>>>>> in a
>>>>>>>>>>> bit
>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
>>>> and
>>>>>>> aside
>>>>>>>>>>> from
>>>>>>>>>>>>> one setup
>>>>>>>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
>>>> a
>>>>>>>>>>> volume on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
>>>> data
>>>>>>>>>>> structure
>>>>>>>>>>>>> to make
>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>>>>>>>>>>> XenServer
>>>>>>>>>>>>> or a
>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
>>>>>>> storage
>>>>>>>>>>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
>>>> new
>>>>>>>>>>> storage
>>>>>>>>>>>>> framework,
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
>>>> easily
>>>>>>>>>>> create
>>>>>>>>>>>>> a 1:1
>>>>>>>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
>>>> the
>>>>>>> VM's
>>>>>>>>>>> data
>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>> (necessary
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
>>>> write
>>>>>>> a
>>>>>>>>>>> plug-
>>>>>>>>>>>>> in that is
>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
>>>> that
>>>>>>> the
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>>>>>>>>>>> hypervisor-
>>>>>>>>>>>>> specific data
>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
>>>> the
>>>>>>> case,
>>>>>>>>>>> I've
>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>> enhancing
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
>>>> and
>>>>>>>>>>>>> submitted. I've got
>>>>>>>>>>>>>>>>> ESX(i)
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
>>>> the
>>>>>>> 4.2
>>>>>>>>>>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
>>>> help a
>>>>>>> bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
>>>> 3,
>>>>>>> 2013 at
>>>>>>>>>>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
>>>>>>> plug-in -
>>>>>>>>>>> by
>>>>>>>>>>>>> itself - is
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
>>>> with
>>>>>>> the
>>>>>>>>>>>>> volume-attach
>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>> calls
>>>>>>> into
>>>>>>>>>>> the
>>>>>>>>>>>>> plug-in to
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
>>>> time
>>>>>>>>>>> comes to
>>>>>>>>>>>>> attach the
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
>>>> smart
>>>>>>>>>>> enough
>>>>>>>>>>>>> to recognize
>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
>>>>>>> volume
>>>>>>>>>>> has
>>>>>>>>>>>>> just been
>>>>>>>>>>>>>>>>> created)
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
>>>> (for
>>>>>>>>>>>>> XenServer) or a
>>>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
>>>> was
>>>>>>> just
>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
>>>>>>> spending
>>>>>>>>>>> most
>>>>>>>>>>>>> of my time
>>>>>>>>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
>>>> clear it
>>>>>>> up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>> Jun 3,
>>>>>>> 2013
>>>>>>>>>>> at
>>>>>>>>>>>>> 12:48 PM,
>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
>>>> explain
>>>>>>>>>>> why
>>>>>>>>>>>>> the the storage
>>>>>>>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>>>> 2013,
>>>>>>> at
>>>>>>>>>>> 1:21
>>>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
>>>>>>> ultimately I
>>>>>>>>>>>>> would like to
>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
>>>> I
>>>>>>> think
>>>>>>>>>>> I'm
>>>>>>>>>>>>> just out of
>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
>>>> now this
>>>>>>>>>>> plug-in
>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
>>>> to 4.2
>>>>>>>>>>> feature
>>>>>>>>>>>>> freeze, I
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
>>>> to
>>>>>>> be
>>>>>>>>>>> clear,
>>>>>>>>>>>>> this is
>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>> Compute
>>>>>>>>>>>>> Offerings post
>>>>>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>> Jun 3,
>>>>>>>>>>> 2013
>>>>>>>>>>>>> at 11:14 AM,
>>>>>>>>>>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
>>>> any
>>>>>>> plan
>>>>>>>>>>> on
>>>>>>>>>>>>> supporting
>>>>>>>>>>>>>>>>> KVM in
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
>>>> Original
>>>>>>>>>>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
>>>> "Mike
>>>>>>>>>>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>>>>>>>>>>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
>>>>>>> Monday,
>>>>>>>>>>> June
>>>>>>>>>>>>> 3, 2013
>>>>>>>>>>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
>>>> Re:
>>>>>>>>>>> [MERGE]
>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
>>>> on
>>>>>>>>>>> merging
>>>>>>>>>>>>> Wei's feature
>>>>>>>>>>>>>>>>>>> first,
>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
>>>> feature
>>>>>>> is
>>>>>>>>>>> for
>>>>>>>>>>>>> KVM only,
>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>>>> Jun
>>>>>>> 3,
>>>>>>>>>>> 2013
>>>>>>>>>>>>> at 8:55 AM,
>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>> the
>>>>>>> billing,
>>>>>>>>>>> as
>>>>>>>>>>>>> no one works
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> billing
>>>>>>> by
>>>>>>>>>>>>> themselves. They
>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> disk_offering of
>>>>>>>>>>> a
>>>>>>>>>>>>> VMs and
>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
>>>> user
>>>>>>> the
>>>>>>>>>>>>> exact
>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>> limitation
>>>>>>>>>>> for
>>>>>>>>>>>>> nics as
>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
>>>> part of
>>>>>>> I/O
>>>>>>>>>>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
>>>> my
>>>>>>>>>>>>> misunstand the second
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
>>>> with
>>>>>>>>>>> what
>>>>>>>>>>>>> you said about
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> 2013/6/3
>>>>>>> John
>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> Jun 3,
>>>>>>> 2013,
>>>>>>>>>>> at
>>>>>>>>>>>>> 2:13 AM, Wei
>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>>>> John,
>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>> hope
>>>>>>> Mike's
>>>>>>>>>>>>> aswer helps you.
>>>>>>>>>>>>>>>>> I am
>>>>>>>>>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
>>>>>>> think
>>>>>>>>>>> billing
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> depend
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> limitation.
>>>>>>>>>>> Please
>>>>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>> the
>>>>>>> IO
>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> virtual
>>>>>>>>>>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Going by
>>>>>>> the
>>>>>>>>>>> AWS
>>>>>>>>>>>>> model,
>>>>>>>>>>>>>> customers
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> provisioned
>>>>>>>>>>> IOPS,
>>>>>>>>>>>>> as well as,
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://aws.amazon.com/ebs/).
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> option to
>>>>>>>>>>> employ
>>>>>>>>>>>>> similar cost
>>>>>>>>>>>>>>>>>>> models.
>>>>>>>>>>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> billing
>>>>>>> model
>>>>>>>>>>> in
>>>>>>>>>>>>> the current
>>>>>>>>>>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>>>> Do
>>>>>>> you
>>>>>>>>>>> mean
>>>>>>>>>>>>> IOPS runtime
>>>>>>>>>>>>>>>>>>> change?
>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> limitation for
>>>>>>>>>>> a
>>>>>>>>>>>>> running
>>>>>>>>>>>>>> virtual
>>>>>>>>>>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> CloudStack
>>>>>>>>>>> does
>>>>>>>>>>>>> not support
>>>>>>>>>>>>>>>>>>> changing
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> (computer
>>>>>>>>>>>>> offering or disk
>>>>>>>>>>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>> meant at
>>>>>>> the
>>>>>>>>>>>>> Java interface
>>>>>>>>>>>>>>>>> level.
>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>> more
>>>>>>>>>>>>> generalize allocation
>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> describe
>>>>>>> the
>>>>>>>>>>>>> service guarantees
>>>>>>>>>>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>>>> It is a
>>>>>>>>>>> good
>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>> Maybe it
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> disk_io_throttling as Mike
>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> type, I
>>>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
>>>>>>> expand
>>>>>>>>>>> on
>>>>>>>>>>>>> my thoughts
>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> touch
>>>>>>> points
>>>>>>>>>>>>> between these two
>>>>>>>>>>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>> need
>>>>>>> to be
>>>>>>>>>>>>> merged before
>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> coordination
>>>>>>>>>>>>> between the
>>>>>>>>>>>>>> branches
>>>>>>>>>>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> disk_io_throttling)
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> 2013/6/3
>>>>>>>>>>> John
>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> The
>>>>>>> things I
>>>>>>>>>>>>> want to
>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
>>>> Is
>>>>>>> there
>>>>>>>>>>>>> value in
>>>>>>>>>>>>>> capturing
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> data
>>>>>>> model
>>>>>>>>>>> (e.g.
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
>>>>>>> Should
>>>>>>>>>>> there
>>>>>>>>>>>>> be a common
>>>>>>>>>>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> provisioning
>>>>>>>>>>> at
>>>>>>>>>>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
>>>> How
>>>>>>> are
>>>>>>>>>>>>> conflicting
>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> hypervisor
>>>>>>>>>>> and
>>>>>>>>>>>>> storage device
>>>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>> lead
>>>>>>> to
>>>>>>>>>>>>> believe (and
>>>>>>>>>>>>>> billed)
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> storage
>>>>>>>>>>> device
>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> consistent
>>>>>>>>>>>>> configuration
>>>>>>>>>>>>>>>>> between a
>>>>>>>>>>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> time,
>>>>>>> but a
>>>>>>>>>>>>> later
>>>>>>>>>>>>>> modification
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> Jun
>>>>>>> 2,
>>>>>>>>>>> 2013,
>>>>>>>>>>>>> at 8:38 PM,
>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>> believe
>>>>>>>>>>> Wei's
>>>>>>>>>>>>> feature deals
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> the
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> My
>>>>>>> feature
>>>>>>>>>>> is
>>>>>>>>>>>>> focused on
>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>> hope
>>>>>>> that
>>>>>>>>>>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> Sun,
>>>>>>> Jun
>>>>>>>>>>> 2,
>>>>>>>>>>>>> 2013 at 6:35
>>>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> My
>>>>>>>>>>> opinion is
>>>>>>>>>>>>> that no
>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues
>>>>>>>>>>> have
>>>>>>>>>>>>> been resolved
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> total
>>>>>>> Ops
>>>>>>>>>>> vs
>>>>>>>>>>>>> discrete
>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
>>>>>>> reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido,
>>>>>>> I
>>>>>>>>>>> don't
>>>>>>>>>>>>> think this
>>>>>>>>>>>>>>>>> criteria
>>>>>>>>>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Also,
>>>>>>> how
>>>>>>>>>>>>> does this work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> also
>>>>>>>>>>> involves
>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> scenario
>>>>>>>>>>>>> where provisioned
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>>> Jun 1,
>>>>>>>>>>> 2013,
>>>>>>>>>>>>> at 6:47 AM,
>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Sure. I
>>>>>>> will
>>>>>>>>>>>>> change it next
>>>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> 2013/6/1
>>>>>>>>>>>>> Wido den Hollander
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> Exactly. I
>>>>>>>>>>> have
>>>>>>>>>>>>> pushed the
>>>>>>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>> anyone
>>>>>>>>>>>>> object thems for
>>>>>>>>>>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> For
>>>>>>> the
>>>>>>>>>>> sake
>>>>>>>>>>>>> of clarity I
>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> the
>>>>>>> total
>>>>>>>>>>> IOps
>>>>>>>>>>>>> to R/W IOps
>>>>>>>>>>>>>>>>> asap
>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> only
>>>>>>> total
>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> You
>>>>>>> laid
>>>>>>>>>>> the
>>>>>>>>>>>>> groundwork for
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> however
>>>>>>>>>>>>> prevent that we
>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> 2013/5/31
>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>>>>>>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> +1 --
>>>>>>> this
>>>>>>>>>>>>> enhancement must
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> don't
>>>>>>> see
>>>>>>>>>>> how
>>>>>>>>>>>>> it could be
>>>>>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> split
>>>>>>> total
>>>>>>>>>>> IOPS
>>>>>>>>>>>>> into read
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> total
>>>>>>>>>>>>> unless/until we
>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> What
>>>>>>> Wei
>>>>>>>>>>>>> meant was merging
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> 4.2
>>>>>>> branch
>>>>>>>>>>>>> and add Read /
>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> will be
>>>>>>>>>>>>> released with Read
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> This is
>>>>>>> to
>>>>>>>>>>>>> make the May 31st
>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (see
>>>>>>> other
>>>>>>>>>>>>> threads) then it
>>>>>>>>>>>>>>>>> won't
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>> also
>>>>>>>>>>>>> completely agree that
>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> disk
>>>>>>> I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>>> May 31,
>>>>>>>>>>>>> 2013, at 9:51 AM,
>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Hi
>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> Thanks.
>>>>>>>>>>> Good
>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>> thought
>>>>>>>>>>>>> about at the
>>>>>>>>>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> difference
>>>>>>>>>>> of
>>>>>>>>>>>>> read and write
>>>>>>>>>>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> care
>>>>>>> the
>>>>>>>>>>>>> difference of sent
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> That
>>>>>>>>>>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Disk
>>>>>>> I/O is
>>>>>>>>>>>>> much more
>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> Implementing
>>>>>>>>>>>>> it will be some
>>>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> implemented
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> few
>>>>>>> days.
>>>>>>>>>>> For
>>>>>>>>>>>>> the deadline
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> that ,
>>>>>>> if
>>>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>> think it's
>>>>>>>>>>> a
>>>>>>>>>>>>> feature we
>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> window
>>>>>>>>>>> we
>>>>>>>>>>>>> have to make sure
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>> in
>>>>>>> 4.3,
>>>>>>>>>>> that
>>>>>>>>>>>>> will confuse
>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> 2013/5/31
>>>>>>>>>>>>> Wido den
>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> On
>>>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>> would
>>>>>>>>>>> like to
>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>> nobody
>>>>>>>>>>>>> object, I will
>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> The
>>>>>>>>>>> purpose
>>>>>>>>>>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Virtual
>>>>>>>>>>>>> machines are running
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> share
>>>>>>>>>>> strage).
>>>>>>>>>>>>> Because of
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> iops),
>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> one
>>>>>>> VM
>>>>>>>>>>> has
>>>>>>>>>>>>> large disk
>>>>>>>>>>>>>>>>> operation,
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> other
>>>>>>> VMs
>>>>>>>>>>>>> running on the
>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>> is
>>>>>>>>>>> neccesary
>>>>>>>>>>>>> to set the
>>>>>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> Looking at
>>>>>>>>>>> the
>>>>>>>>>>>>> code I see
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Qemu
>>>>>>> and
>>>>>>>>>>>>> libvirt support
>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOps
>>>>>>>>>>> which
>>>>>>>>>>>>> could benefit a
>>>>>>>>>>>>>>>>> lot of
>>>>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> It's
>>>>>>> also
>>>>>>>>>>>>> strange, in the
>>>>>>>>>>>>>>>>> polling
>>>>>>>>>>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOps,
>>>>>>> but
>>>>>>>>>>> on
>>>>>>>>>>>>> the throttling
>>>>>>>>>>>>>>>>> side
>>>>>>>>>>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Write
>>>>>>> IOps
>>>>>>>>>>> are
>>>>>>>>>>>>> usually much
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> like a
>>>>>>> valid
>>>>>>>>>>>>> use-case where
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> IOps
>>>>>>> vs
>>>>>>>>>>> Read
>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Since
>>>>>>> this
>>>>>>>>>>> only
>>>>>>>>>>>>> supports KVM
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> value
>>>>>>> to at
>>>>>>>>>>>>> least have the
>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> this
>>>>>>> later
>>>>>>>>>>>>> would be a lot of
>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>> a
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> write
>>>>>>> you
>>>>>>>>>>> can
>>>>>>>>>>>>> always sum
>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Can
>>>>>>> you
>>>>>>>>>>>>> explain why you
>>>>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> The
>>>>>>>>>>> feature
>>>>>>>>>>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (1) set
>>>>>>> the
>>>>>>>>>>>>> maximum rate of
>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (2)
>>>>>>> change
>>>>>>>>>>> the
>>>>>>>>>>>>> maximum rate
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (3)
>>>>>>> limit
>>>>>>>>>>> the
>>>>>>>>>>>>> disk rate
>>>>>>>>>>>>>> (total
>>>>>>>>>>>>>>>>>>> bps
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> JIRA
>>>>>>> ticket:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>> 1192
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> FS (I
>>>>>>> will
>>>>>>>>>>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> Merge
>>>>>>>>>>> check
>>>>>>>>>>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>>>> Did
>>>>>>> you
>>>>>>>>>>>>> check the branch's
>>>>>>>>>>>>>>>>> RAT
>>>>>>>>>>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>>>> Are
>>>>>>>>>>> there
>>>>>>>>>>>>> new dependencies
>>>>>>>>>>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>>>>>>> What
>>>>>>>>>>>>> automated testing
>>>>>>>>>>>>>> (unit
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Unit
>>>>>>> tests
>>>>>>>>>>> are
>>>>>>>>>>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>>>>>>> What
>>>>>>>>>>>>> testing has been done
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (1) set
>>>>>>> the
>>>>>>>>>>>>> bytes rate and
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (2)
>>>>>>> VM
>>>>>>>>>>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> deploy,
>>>>>>>>>>> stop,
>>>>>>>>>>>>> start, reboot,
>>>>>>>>>>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (3)
>>>>>>>>>>> Volume
>>>>>>>>>>>>> operations,
>>>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Attach,
>>>>>>>>>>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> To
>>>>>>> review
>>>>>>>>>>> the
>>>>>>>>>>>>> code, you can
>>>>>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Best
>>>>>>>>>>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> [2]
>>>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>>>>> VM
>>>>>>>>>>> Disk
>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Mike
>>>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *Senior
>>>>>>>>>>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> Advancing
>>>>>>>>>>> the
>>>>>>>>>>>>> way the world
>>>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Advancing
>>>>>>> the
>>>>>>>>>>> way
>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Advancing the
>>>>>>>>>>> way
>>>>>>>>>>>>> the world uses
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>>>> the
>>>>>>> way
>>>>>>>>>>> the
>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>> CloudStack
>>>>>>>>>>>>> Developer,
>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>>>> the
>>>>>>> way
>>>>>>>>>>> the
>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>>>>>>>>> Developer, SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>> way
>>>>>>> the
>>>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>> way
>>>>>>> the
>>>>>>>>>>> world
>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>> way the
>>>>>>>>>>> world
>>>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>>> Developer,
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>>>> the
>>>>>>> world
>>>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>>> Developer,
>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>>>>> world
>>>>>>>>>>> uses
>>>>>>>>>>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>>> uses
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>>> uses
>>>>>>>>>>> the
>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>>> the
>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>>> the
>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>>>> the
>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *(tm)*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *(tm)*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *(tm)*****
>>>>>>
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*****
>>>>>>
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>
>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>
>>>>>> o: 303.746.7302****
>>>>>>
>>>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *(tm)*****
>>>>>>
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*****
>>>>>>
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>>>
>>>>>> e: mike.tutkowski@solidfire.com****
>>>>>>
>>>>>> o: 303.746.7302****
>>>>>>
>>>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *(tm)*****
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *(tm)*
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
So, would a storage vendor write a non-CloudStack-managed plug-in and maybe
just implement the snapshot feature or something?
In this case, they'd have to set the hypervisor up ahead of time with a
large portion of their storage, but they'd still get the vendor-snapshot
ability.
Just trying to undertand this more.
Thanks
On Mon, Jun 10, 2013 at 1:04 PM, John Burwell <jb...@basho.com> wrote:
> Edison,
>
> TL;DR Always give operators/system administrators the ability to manage
> things themselves, if they like.
>
> As more manageable drivers emerge, I think it will be import to allow
> operators to opt-out of CloudStack device management. Therefore, a driver
> declares whether or not it can support device management (many, in fact
> most, won't). For those drivers that do support management, allow
> operators to opt-in/out of CloudStack management on a per device basis. To
> support this notion, we need a flag on the storage_pool table and
> associated classes to indicate whether or not the operator wants CloudStack
> to manage the device.
>
> Does this explanation make more sense?
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
>
> > I don't understand why we need to add a "managed" in the UI and in
> "storage_pool" table?
> > AFAIK, what you guys trying to do is to enable
> attachVolume/DettachVolume if the volume is created on Solidfire, right?
> > If the volume is created on Solidfire already, by createAsync, then why
> we can't just send out all the information about the volume(it's volumeTO
> which will be returned by each storage driver) to hypervisor, then at the
> hypervisor side, check the volumeTO, if the volumeTO is managed, then doing
> whatever necessary operations(create SR, create VDI etc), then attach the
> VDI to volume.
> >
> >
> >> -----Original Message-----
> >> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >> Sent: Monday, June 10, 2013 11:10 AM
> >> To: John Burwell
> >> Cc: Edison Su; dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>
> >> I believe we're on the same page, John.
> >>
> >> I would add a new parameter to the createStoragePool API command (let's
> >> call it "managed" and it can be true or false). If you pass in "true",
> but
> >> the driver does not support being managed by CloudStack, an exception is
> >> thrown by the driver. If you pass in "false", but the driver does not
> >> support being managed outside of CloudStack, an exception is thrown by
> the
> >> driver. It is possible one could write a driver that can be managed both
> >> inside or outside of CloudStack (the SolidFire driver requires being
> >> managed only by CloudStack).
> >>
> >> This managed value would end up in the storage_pool table in a new
> column
> >> ("managed").
> >>
> >> An example usage: After the storage framework calls on the driver's
> >> createAsync method, it will check to see if the driver is managed. If
> the
> >> driver is managed, the storage framework will invoke the proper (new)
> >> command of the hypervisor in use to set up, say, the SR for the SAN
> volume.
> >>
> >> There may be other cases when the isManaged() method is invoked on the
> >> driver and - if true is returned - the storage framework sends a
> particular
> >> command to the hypervisor in use.
> >>
> >> ********************
> >>
> >> As far as reworking the default storage plug-in (which sends commands to
> >> hypervisors) for 4.2, I would check with Edison on his schedule.
> >>
> >> Thanks!
> >>
> >>
> >> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>
> >>> Mike,
> >>>
> >>> I want to make sure we are on the same page regarding isManaged. I see
> >>> the addition of the following operations:
> >>>
> >>> * DataStoreDriver#supportsManagement() : boolean -- indicates whether
> >> or
> >>> not the driver support/implements the management functions
> >>> * DataStore#isManaged() : boolean -- indicates whether or not a device
> is
> >>> managed by CloudStack
> >>>
> >>> Only DataStores associated with DataStoreDrivers that support
> >> management
> >>> would be able to have the management flag enabled. This behavior
> allows
> >>> operators to declare their intention to allow CloudStack to manage a
> >>> device. This behavior would need to be exposed as flag on the HTTP
> API.
> >>> Additionally, this flag would only be settable when a device is
> >>> defined/created in the system (i.e. once a device is managed by
> CloudStack,
> >>> you can't unmanaged it without dissociating it from CloudStack). In
> the
> >>> future, we may consider relaxing that rule, but, in the short term, it
> >>> greatly decreases the alternative flows.
> >>>
> >>> In terms of removing the dependencies from Storage to Hypervisor, I
> >> would
> >>> really like to see this accomplished in 4.2. It will be much more
> >>> difficult to address this issue post 4.2 because we will have more
> plugins
> >>> to address. I also see this issue as a critical architectural issues
> that
> >>> for the management server. I recommend we identify the dependencies
> >> that
> >>> need to be inverted, and see if we can divide the work across a few
> more
> >>> resources.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> >> <mi...@solidfire.com>
> >>> wrote:
> >>>
> >>> Sounds good, Edison :)
> >>>
> >>> I can implement John's isManaged() logic and resubmit my code for
> review.
> >>>
> >>> Perhaps in 4.3 I could address the direct-attach scenario. If it
> requires
> >>> some refactoring of the storage framework, then I could do that.
> >>>
> >>> Also, the default storage plug-in does have hypervisor logic in it in
> 4.2.
> >>> I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> >>> with the above work I mentioned.
> >>>
> >>> Are you OK with that, guys?
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
> wrote:
> >>>
> >>>> Ok, I am Ok with the way you are talking about in 4.2.****
> >>>>
> >>>> ** **
> >>>>
> >>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>> *Sent:* Sunday, June 09, 2013 8:59 PM
> >>>> *To:* dev@cloudstack.apache.org
> >>>> *Cc:* Edison Su; John Burwell
> >>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>>>
> >>>> ** **
> >>>>
> >>>> Another thing we might want to consider for 4.2 is that the way the
> code
> >>>> is currently implemented in master, I don't think it's even possible
> to
> >>>> write a plug-in that connects storage directly to a VM because the
> >>>> CloudStack storage framework will call the "attach" method of the
> >>>> hypervisor in use and this method will fail because it assumes there
> is
> >>>> (talking Xen here) an SR existent and in this situation there isn't
> one.*
> >>>> ***
> >>>>
> >>>> ** **
> >>>>
> >>>> My thinking is that we should ignore the direct-attach-of-volume-to-VM
> >>>> use case for 4.2. I think it will require too much of a rewrite at
> this
> >>>> point in development (fewer than three weeks left until code
> >> freeze).****
> >>>>
> >>>> ** **
> >>>>
> >>>> If the plug-in implements an isManaged() method (as John suggested),
> >> the
> >>>> storage framework can invoke the necessary commands on the
> >> hypervisor in
> >>>> use.****
> >>>>
> >>>> ** **
> >>>>
> >>>> For example, the createAsync method is called. The storage plug-in
> >>>> creates a SAN volume and updates the CS DB. The storage framework
> >> then asks
> >>>> the driver if it's managed. If it is, the storage framework executes a
> >>>> command against the hypervisor in use to have it create, say, an SR
> (with
> >> a
> >>>> single VDI that takes up the entire SR). Now, when the storage
> >> framework
> >>>> executes the "attach" command against the hypervisor, this command
> will
> >>>> work, as well (because that logic assumes the prior existence of an
> SR and
> >>>> it will, in fact, be present).****
> >>>>
> >>>> ** **
> >>>>
> >>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:****
> >>>>
> >>>> Hi Edison and John,****
> >>>>
> >>>> ** **
> >>>>
> >>>> I wanted to point out something I thought was relevant to this
> >>>> conversation.****
> >>>>
> >>>> ** **
> >>>>
> >>>> My plug-in supports both XenServer and ESX(i).****
> >>>>
> >>>> ** **
> >>>>
> >>>> These two hypervisors handle their respective hypervisor data
> structures
> >>>> (XenServer SR or ESX(i) datastore) quite differently.****
> >>>>
> >>>> ** **
> >>>>
> >>>> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> >>>> when you issue a remove-SR command to XenServer.****
> >>>>
> >>>> ** **
> >>>>
> >>>> When you later want to access the contents of that data, it is still
> >>>> safely stored on the LUN and can be accessed from XenServer (from the
> >> same
> >>>> or different cluster (what XenServer calls a resource pool)) by
> creating a
> >>>> new SR that is based on the same IQN and LUN.****
> >>>>
> >>>> ** **
> >>>>
> >>>> XenServer SRs exist at the cluster level.****
> >>>>
> >>>> ** **
> >>>>
> >>>> In ESX(i), the datastore that resides on the SAN LUN is actually
> >>>> destroyed when you issue a remove-datastore command to ESX(i). In
> >> other
> >>>> words, the contents of the SAN LUN are destroyed.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ESX(i) datastores exist at the datacenter level.****
> >>>>
> >>>> ** **
> >>>>
> >>>> The XenServer behavior (not destroying the contents of the LUN when
> >> the
> >>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
> contents
> >>>> of the LUN when the datastore is removed) is not desirable.****
> >>>>
> >>>> ** **
> >>>>
> >>>> I can get around the ESX(i) behavior by removing the referenced IQN
> >> from
> >>>> each host in the cluster when the CS storage framework notifies my
> plug-
> >> in
> >>>> of a detach-volume event. When the CS volume is later attached again
> >>>> (either to the same or a different ESX(i) cluster), I can have each
> host in
> >>>> the relevant ESX(i) cluster reference the IQN and the datastore will
> be
> >>>> back again.****
> >>>>
> >>>> ** **
> >>>>
> >>>> This sequence of events is critical for my plug-in to support both
> >>>> environments:****
> >>>>
> >>>> *XenServer flow (events for Driver class):*****
> >>>>
> >>>> createAsync (called right before first attach) = creates SAN
> volume****
> >>>>
> >>>> preAttachVolume = creates SR and creates VDI or just introduces SR****
> >>>>
> >>>> (CS volume gets attached and later detached)****
> >>>>
> >>>> postDetachVolume = removes SR (this does not delete data on SAN
> >> volume)**
> >>>> **
> >>>>
> >>>> deleteAsync (can be called when volume is not attached) = deletes the
> >> SAN
> >>>> volume****
> >>>>
> >>>> *VMware flow (events for Driver class):*****
> >>>>
> >>>> createAsync (called right before first attach) = creates SAN volume,
> >>>> creates datastore, and creates VMDK file****
> >>>>
> >>>> preAttachVolume = adds iSCSI target to each host in cluster****
> >>>>
> >>>> (CS volume gets attached and later detached)****
> >>>>
> >>>> postDetachVolume = removes iSCSI target from each host in cluster****
> >>>>
> >>>> deleteAsync (can be called when volume is not attached) = deletes
> >>>> datastore and deletes the SAN volume****
> >>>>
> >>>> ************************
> >>>>
> >>>> So, the way I have it written up here, the driver is talking to the
> >>>> hypervisor. As John has pointed out, this may not be ideal.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Whatever we decide on, it needs to be able to support this kind of
> flow.
> >>>> :)****
> >>>>
> >>>> ** **
> >>>>
> >>>> We could go John's route, where the driver says if it's managed or
> not.**
> >>>> **
> >>>>
> >>>> ** **
> >>>>
> >>>> If it is, then the storage framework could send the necessary
> messages to
> >>>> the hypervisor in question at the right times (just migrate the
> hypervisor
> >>>> logic above from the driver to the right parts of the storage
> framework).
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> This is fine, except that it does not resolve Edison's use case of
> >>>> desiring to have the storage LUN potentially connected directly to a
> VM.*
> >>>> ***
> >>>>
> >>>> ** **
> >>>>
> >>>> I'm not sure we should design around that use case, though. Is it
> common
> >>>> enough? Has anyone asked for it? Is it even desirable? How are usage
> >>>> statistics gathered for such volumes? It's nice to offer the
> flexibility,
> >>>> but is it a good idea in this case?****
> >>>>
> >>>> ** **
> >>>>
> >>>> Thanks!****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> wrote:**
> >>>> **
> >>>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>> Sent: Friday, June 07, 2013 4:08 PM
> >>>>> To: dev@cloudstack.apache.org
> >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>
> >>>>> Hi John,
> >>>>>
> >>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
> >>>> volume
> >>>>> to use it. Same story for VMware: ESX(i) must layer a datastore on
> top
> >>>> of
> >>>>> the iSCSI volume to use it. I look at it like they are layering a
> >>>> clustered
> >>>>> file system on the SAN volume so the hypervisors can share access to
> >> the
> >>>>> contents of the volume.
> >>>>>
> >>>>> In 4.1, by the time the hypervisor-attach-volume logic is called, the
> >>>> SR or
> >>>>> datastore has already been created (usually manually by an admin).
> >>>>>
> >>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
> >>>>> environment where each CloudStack volume that a user creates is
> >> mapped
> >>>>> to a
> >>>>> single SAN volume (via the SR).
> >>>>>
> >>>>> The question comes down to who should allocate the SR dynamically.
> >>>>>
> >>>>> We could have the storage framework ask the storage plug-in if it is
> >>>>> managed. If it is, then the storage framework could send a message to
> >>>> the
> >>>>> hypervisor in question to create (let's talk Xen here) the SR ahead
> of
> >>>>> time. Then, when the storage framework next sends the attach-volume
> >>>>> command
> >>>>> to the hypervisor, it should work without changes to that
> attach-volume
> >>>>> logic (because - from the point of view of the attach logic - the SR
> is
> >>>>> already existent, as expected).
> >>>>>
> >>>>> Now, as Edison has pointed out, this limits the power of the storage
> >>>>> plug-in. A storage plug-in in this model cannot directly attach
> storage
> >>>> to
> >>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We need
> >> to
> >>>>> make a call on that.
> >>>>
> >>>>
> >>>> Yes, I think we need to address this usage case, and won't limit the
> >>>> power of the storage plug-in.
> >>>> John, what's your idea to fit this usage case into your abstraction?
> >>>>
> >>>>
> >>>>>
> >>>>> If we want to give more power to the storage vendor, we would have to
> >>>>> have
> >>>>> the storage framework call into the storage plug-in to send the
> >>>> appropriate
> >>>>> attach commands to the hypervisor.
> >>>>>
> >>>>> Let's discuss and come to a consensus (or at least agree on a path)
> as
> >>>> soon
> >>>>> as we can. :)
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> My understanding is the Xen expects an SR structure on all ISCSI
> >>>> devices
> >>>>>> -- at least that is how I read the code in your patch. Is my
> >>>> understanding
> >>>>>> correct? If so, the Xen plugin should be able to query the storage
> >>>> device
> >>>>>> to determine the presence of the SR structure and create it if it
> >>>> does not
> >>>>>> exist. Am I missing something in the implementation that makes that
> >>>> type
> >>>>>> of implementation impossible?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Yeah, if a storage vendor wanted to attach a volume directly to a
> >>>> VM in
> >>>>>> 4.2
> >>>>>>> today, it would probably fail because the attach-volume logic
> >>>> assumes
> >>>>> the
> >>>>>>> existence of the necessary hypervisor data structure (ex. SR on
> >>>> Xen).
> >>>>>>>
> >>>>>>> If we wanted to enable such an attach, we could do it the way
> >> Edison
> >>>>>>> suggests.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>
> >>>>>>>>> As we only have three weeks until feature freeze, we should
> >> come
> >>>> to
> >>>>> a
> >>>>>>>>> consensus on this design point as soon as possible.
> >>>>>>>>>
> >>>>>>>>> Right now, if the storage framework asks my driver it is is
> >>>> managed, it
> >>>>>>>>> will say 'yes.' This means the framework will tell the driver to
> >>>>>> perform
> >>>>>>>>> its management activities. This then means the driver will call
> >>>> into
> >>>>>> the
> >>>>>>>>> host (it doesn't know which hypervisor, by the way) to perform
> >> the
> >>>>>>>> activity
> >>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
> >>>>>>>>>
> >>>>>>>>> The driver doesn't know which hypervisor it's talking to, it just
> >>>>>> sends a
> >>>>>>>>> message to the host to perform the necessary pre-attach work.
> >>>>>>>>
> >>>>>>>> Could we just expose a method like
> >> "attachVolume/dettachVolume" on
> >>>>> the
> >>>>>>>> PrimaryDataStoreDriver driver?
> >>>>>>>> For most of cases, the implementation of each driver would just
> >>>> send
> >>>>>>>> attachvolumecommand/dettachvolumecmd to hypervisor(we can
> >> put
> >>>>> the
> >>>>>>>> implementation in a base class, so that can be shared by all of
> >>>> this
> >>>>>>>> drivers), and in each hypervisor resource code, which may just
> call
> >>>>>>>> hypervisor's api to attach the volume to VM, while for certain
> >>>> storage,
> >>>>>>>> like SolidFire, may need to create a SR first, and create volume
> >>>> on it,
> >>>>>>>> then call hypervisor's API to attach volume to VM.
> >>>>>>>> While for some other storage vendor, may want to bypass
> >> hypervisor
> >>>>>> during
> >>>>>>>> attaching volume, so inside driver's attachvolume implementation,
> >>>> the
> >>>>>>>> driver can do some magic, such as, directly talk to an agent
> >>>> inside VM
> >>>>>>>> instance, then create a disk inside VM.
> >>>>>>>>
> >>>>>>>> How do you guys think?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> Hi John,
> >>>>>>>>>>>
> >>>>>>>>>>> How's about this:
> >>>>>>>>>>>
> >>>>>>>>>>> The driver can implement an isManaged() method. The
> >>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>> can
> >>>>>>>>>>> call into the driver to see if its managed. If it is, the
> >>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>> (which is responsible for calling into the hypervisor to attach
> >>>> the
> >>>>>>>> disk)
> >>>>>>>>>>> can call into the hypervisor to create the necessary hypervisor
> >>>> data
> >>>>>>>>>>> structure (ex. for XenServer, a storage repository).
> >>>>>>>>>>
> >>>>>>>>>> The problem here is that storage vendor may work differently
> >> with
> >>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> >>>> maybe
> >>>>>>>> other
> >>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the LUN
> >>>>> directly
> >>>>>>>> to
> >>>>>>>>>> VM instance, see the discuss(
> >>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>>>>>>>>
> >>>>>
> >> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>>>>>>>> %3E
> >>>>>>>>>> ).
> >>>>>>>>>> So I would let storage provider to implement attach disk to VM,
> >>>>>>>> instead of
> >>>>>>>>>> implemented by cloudstack itself.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> If that's what you're going for, that works for me. By the way,
> >>>>>>>> Edison's
> >>>>>>>>>>> default storage plug-in (which handles the default storage
> >>>> behavior
> >>>>>>>> in
> >>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> >>>> talks to
> >>>>>>>>>>> hypervisors. You might want to contact him and inform him of
> >>>> your
> >>>>>>>>>> concerns
> >>>>>>>>>>> or that logic (as is) will make it to production.
> >>>>>>>>>>>
> >>>>>>>>>>> Please let me know if what I wrote in above (for my solution)
> >>>> is OK
> >>>>>>>> with
> >>>>>>>>>>> you. :)
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >>>> jburwell@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please see my responses in-line below.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I still have a bit more testing I'd like to do before I build
> >>>> up
> >>>>>>>> a
> >>>>>>>>>> patch
> >>>>>>>>>>>>> file, but this is the gist of what I've done:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * During a volume-attach operation, after
> >> VolumeManagerImpl
> >>>>> tells
> >>>>>>>>>>>>> VolumeServiceImpl to have the driver create a volume, I
> >> have
> >>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> >> driver if
> >>>> it
> >>>>>>>>>>> managed.
> >>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> >>>>>>>> whatever
> >>>>>>>>>>>>> activity is required. In my case, this includes sending a
> >>>>>>>> message to
> >>>>>>>>>> the
> >>>>>>>>>>>>> host where the VM is running to have, say XenServer, add a
> >>>>>>>> storage
> >>>>>>>>>>>>> repository (based on the IP address of the SAN, the IQN of
> >> the
> >>>>>>>> SAN
> >>>>>>>>>>>> volume,
> >>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
> >>>> can
> >>>>>>>> on
> >>>>>>>>>> the
> >>>>>>>>>>>>> storage repository). After this, the normal attach-volume
> >>>>>>>> message is
> >>>>>>>>>> sent
> >>>>>>>>>>>>> to the host by VolumeManagerImpl.
> >>>>>>>>>>>>
> >>>>>>>>>>>> There should be **no** code from a storage driver to a
> >>>> hypervisor.
> >>>>>>>> I
> >>>>>>>>>>>> apologize for the repetition, but we simply can not have
> >>>> hypervisor
> >>>>>>>>>>>> specific code in the storage layer. The circular dependencies
> >>>>>>>> between
> >>>>>>>>>> the
> >>>>>>>>>>>> two layers are not sustainable in the long term. Either the
> >>>>>>>>>> VirtualManager
> >>>>>>>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >>>>>>>>> coordinate
> >>>>>>>>>>>> volume creation and then populating the SR. Ideally, we can
> >>>>>>>>>> generalize the
> >>>>>>>>>>>> process flow for attaching volumes such that the Xen
> >> hypervisor
> >>>>>>>> plugin
> >>>>>>>>>>>> would only implement callbacks to perform the attach action
> >> and
> >>>>>>>> create
> >>>>>>>>>>> the
> >>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver should
> >>>> only be
> >>>>>>>>>>>> allocating space and providing information about contents
> >> (e.g.
> >>>>>>>> space
> >>>>>>>>>>>> available, space consumed, streams to a URI, file handle for a
> >>>> URI,
> >>>>>>>>>> etc)
> >>>>>>>>>>>> and capabilities.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * The reverse is performed for a detach-volume command.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in my
> >>>> driver.
> >>>>>>>>>>>> Edison's
> >>>>>>>>>>>>> default driver simply does a "return false;". We could add a
> >>>> new
> >>>>>>>>>>>> parameter
> >>>>>>>>>>>>> to the createStoragePool API command, if we want, to
> >> remove
> >>>>> the
> >>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>> return values in the drivers (although my driver will
> probably
> >>>>>>>> just
> >>>>>>>>>>>> ignore
> >>>>>>>>>>>>> this parameter and always return true since it wouldn't make
> >>>>>>>> sense
> >>>>>>>>>> for it
> >>>>>>>>>>>>> to ever return false). We'd need another column in the
> >>>>>>>> storage_pool
> >>>>>>>>>>> table
> >>>>>>>>>>>>> to store this value.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes, I think we should have a parameter added to the
> >>>>>>>>> createStoragePool
> >>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
> >>>> configured
> >>>>>>>> for
> >>>>>>>>>>>> management when their underlying drivers support it. To
> >>>> simplify
> >>>>>>>>>> things,
> >>>>>>>>>>>> this flag should only be mutable when the DataStore is
> >>>> created. It
> >>>>>>>>>> would be
> >>>>>>>>>>>> a bit crazy to take a DataStore from managed to unmanaged
> >> after
> >>>>>>>>>> creation.
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
> >> working
> >>>>>>>> GUI
> >>>>>>>>>> for
> >>>>>>>>>>>> you
> >>>>>>>>>>>>>> to review (in that document), if you'd like to check it out.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I would like the UIs of two features reviewed together to
> >>>>>>>> ensure
> >>>>>>>>>>>>>>> consistency across the concepts of hypervisor throttled
> >> IOPs
> >>>>>>>> and
> >>>>>>>>>>>>>>> storage device provisioned IOPs. I see the potential for
> >>>>>>>>>> confusion,
> >>>>>>>>>>>>>>> and I think a side-by-side Ui review of these features will
> >>>>>>>> help
> >>>>>>>>>>>>>>> minimize any potential confusion.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will work
> >>>> itself
> >>>>>>>> if
> >>>>>>>>>> it
> >>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
> hypervisor
> >>>>>>>>>> throttled
> >>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
> >> sent
> >>>>> to
> >>>>>>>>> you
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>> other day to see how the GUI looks for provisioned
> >> storage
> >>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-mail
> list
> >>>>>>>> and we
> >>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> >>>>> Offering
> >>>>>>>>>> dialog.
> >>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS, so they
> >>>>> should
> >>>>>>>>> be
> >>>>>>>>>>> able
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say, only
> >>>> use
> >>>>>>>> Max
> >>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>> These fields are optional and can be ignored for storage
> >>>> that
> >>>>>>>>>> does not
> >>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
> >>>> the
> >>>>>>>>>> admin can
> >>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> >> Burst
> >>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature
> >> or
> >>>>>>>> mine,
> >>>>>>>>>> as
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> decided by the admin).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm not sure what we can do about these two features
> >>>>>>>> expressing
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express the
> >> IOPS
> >>>> for
> >>>>>>>>>> QoS in
> >>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
> >>>> read/write
> >>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
> >>>>>>>>> Storage/Hypervisor
> >>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>> separation, but we still need to reconcile the behavior
> >> of
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> >>>>>>>> following
> >>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>> outstanding:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >>>>>>>>> read/write
> >>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>> where as storage provisioned IOPS are expressed as
> >> total
> >>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
> >> to
> >>>>>>>>> storage
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>>>>>>>> 3. How should usage data be captured for throttled and
> >>>>>>>>>> provisioned
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> that will permit providers to discriminate these
> >>>> guaranteed
> >>>>>>>>>>>> operations
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
> >>>> provisioned
> >>>>>>>>> IOPS
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
> >>>> hypervisor
> >>>>>>>>>> throttled
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> >>>> solve
> >>>>>>>>>> items 1
> >>>>>>>>>>>>>>> and 2.
> >>>>>>>>>>>>>>>>> Since the two facilities would not be permitted to
> >> operate
> >>>>>>>>>> together,
> >>>>>>>>>>>>>>> they
> >>>>>>>>>>>>>>>>> do not need to be semantically compatible. I think item
> >> 3
> >>>>>>>> can be
> >>>>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>> with an additional flag or two on the usage records. As
> >>>> for
> >>>>>>>>>> Item 4,
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>> not familiar with how these two enhancements are
> >>>>> depicted in
> >>>>>>>>> the
> >>>>>>>>>>> user
> >>>>>>>>>>>>>>>>> interface. I think we need to review the UI
> >> enhancements
> >>>>> for
> >>>>>>>>>> both
> >>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >>>>> <us...@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> >>>>> 2780/2729,
> >>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >>>>>>>>> CLOUDSTACK-
> >>>>>>>>>>> 2875 ) this
> >>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
> >> and
> >>>>>>>> ask
> >>>>>>>>>> for
> >>>>>>>>>>>>>>> second
> >>>>>>>>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> >> <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I believe I understand where you're going with this,
> >>>> John,
> >>>>>>>> and
> >>>>>>>>>>> have
> >>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> >>>>> whether or
> >>>>>>>>>>> not a
> >>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> >> but an
> >>>>>>>>> actual
> >>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >>>>>>>> resources
> >>>>>>>>>> for
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> >>>>> method
> >>>>>>>>>>> added to
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> >>>>>>>> then the
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> framework could call the manage() (or whatever
> >> name)
> >>>>>>>>> method
> >>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
> >> well)
> >>>>> and
> >>>>>>>>> this
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to create,
> >> say
> >>>>> on
> >>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >>>>>>>>>>> DataStoreDriver.
> >>>>>>>>>>>>>>> Since
> >>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> >> storage,
> >>>>> I
> >>>>>>>>>> would
> >>>>>>>>>>>> add
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> >>>>> straw
> >>>>>>>>> man
> >>>>>>>>>>> term
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> >> mutable
> >>>>>>>>> property
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>>>>>>>> which can be enabled if/when the underlying
> >>>>>>>>> DataStoreDriver
> >>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>>>>> This approach allows operators to override
> >>>>> manageability
> >>>>>>>> of
> >>>>>>>>>>>> devices.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
> >> Xen
> >>>>>>>> plugin
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> responsible for composing DataStore/Volume
> >> methods
> >>>>> to
> >>>>>>>>>>> create any
> >>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR. There
> >>>> should
> >>>>>>>> be no
> >>>>>>>>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> >>>>>>>> earlier,
> >>>>>>>>>> such
> >>>>>>>>>>>>>>>>>>> circular
> >>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
> >> unmaintainable
> >>>>> mess.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page with
> >> you.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
> >> layer
> >>>>> that
> >>>>>>>>>>>> supports n
> >>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors of m
> >>>>>>>> hypervisors.
> >>>>>>>>>>>> Such
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
> >> untestable
> >>>>>>>>> beast.
> >>>>>>>>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven to
> >> evolve
> >>>>> the
> >>>>>>>>>>> Storage
> >>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> >>>>> whether
> >>>>>>>>> or
> >>>>>>>>>>> not a
> >>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> >> but
> >>>>> an
> >>>>>>>>> actual
> >>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >>>>>>>> resources
> >>>>>>>>>> for
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
> >>>> needs
> >>>>>>>> to
> >>>>>>>>>>>> indicate
> >>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
> >> Finally, I
> >>>>>>>> think
> >>>>>>>>>> we
> >>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to elect to
> >>>> have
> >>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> >> capable
> >>>>>>>>> managing
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
> >>>>> unmanaged).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume should be
> >> done
> >>>>> in
> >>>>>>>>> the
> >>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> >>>>> Storage
> >>>>>>>>>>> layer to
> >>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> >> create a
> >>>>>>>> file,
> >>>>>>>>>> etc).
> >>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >>>>>>>>>> hypervisor, as
> >>>>>>>>>>>>>>> such,
> >>>>>>>>>>>>>>>>>>>>> should be confined to its implementation. From
> >> my
> >>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>> nothing
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
> >> content.
> >>>>> From
> >>>>>>>>> its
> >>>>>>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides the
> >> means
> >>>>> to
> >>>>>>>>>> query
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> support the interpretation of the content by
> >> higher-
> >>>>> level
> >>>>>>>>>>> layers
> >>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> >>>>> composition
> >>>>>>>>> of
> >>>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on the
> >>>>> Volume
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> protocol
> >>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> >>>>> implemented
> >>>>>>>> in
> >>>>>>>>>> my
> >>>>>>>>>>>>>>> patch,
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
> >> the
> >>>>>>>>>>> hypervisor.
> >>>>>>>>>>>> Before
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if we're
> >>>>> dealing
> >>>>>>>>> with
> >>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>> (or
> >>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> >>>> send a
> >>>>>>>>>> "Create
> >>>>>>>>>>>>>>> SR"
> >>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> >>>>> would
> >>>>>>>>> then
> >>>>>>>>>>>>>>> proceed
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same and
> >> we just
> >>>>>>>> add
> >>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>>>>>> command to the agent code that is called for this
> >>>>>>>> particular
> >>>>>>>>>>> type
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I created a document for a customer today that
> >>>>> outlines
> >>>>>>>>> how
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will probably
> >> be
> >>>> of
> >>>>>>>> use
> >>>>>>>>>> to
> >>>>>>>>>>>>>>> you, as
> >>>>>>>>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I have shared this document with you (you
> >> should
> >>>>> have
> >>>>>>>>>>> received
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> >>>>> thoughts,
> >>>>>>>>> but
> >>>>>>>>>>>> want
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> >>>> solidfire,
> >>>>>>>>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
> >> in a
> >>>>>>>>>>> coherent
> >>>>>>>>>>>>>>> form
> >>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> >> introduces
> >>>>> the
> >>>>>>>>>> notion
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> >> introduces
> >>>>>>>> the
> >>>>>>>>>>> notion
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages the
> >> plug-in
> >>>>> was
> >>>>>>>>>>>>>>> incomplete,
> >>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> >>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> >> spending on
> >>>>>>>> this.
> >>>>>>>>>> :)
> >>>>>>>>>>> Do
> >>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
> >> accomplish
> >>>>>>>>> what
> >>>>>>>>>>> you're
> >>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
> >> <
> >>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> >> introduces
> >>>>> the
> >>>>>>>>>> notion
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> >> management
> >>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>> (i.e.
> >>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
> >> it
> >>>>>>>>> managed
> >>>>>>>>>>> by
> >>>>>>>>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
> >> with
> >>>>> a
> >>>>>>>>>>> semantic
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> >> managed.
> >>>>> I
> >>>>>>>>> also
> >>>>>>>>>>> want
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
> >> from
> >>>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> >> Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> >>>>> questions
> >>>>>>>>> for
> >>>>>>>>>>> me
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> >> LUNs
> >>>>> and
> >>>>>>>>> SAN
> >>>>>>>>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
> >> LUN
> >>>>> (LUN
> >>>>>>>>> 0).
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> >> Volumes
> >>>>> and
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> >> management
> >>>>>>>>> server or
> >>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
> >> SAN
> >>>>>>>>> volume
> >>>>>>>>>>> (same
> >>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server (via
> >> the
> >>>>> plug-in)
> >>>>>>>>>>> the
> >>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> >> hypervisor.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
> >> in,
> >>>>> but
> >>>>>>>>>> still
> >>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
> >> already
> >>>>>>>>> today
> >>>>>>>>>>>> (prior
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
> >> and -
> >>>>> in
> >>>>>>>>> this
> >>>>>>>>>>> case -
> >>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> >> While
> >>>>> you
> >>>>>>>>> can do
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
> >> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> >> management
> >>>>> server
> >>>>>>>>> or
> >>>>>>>>>>> externally
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> >> SAN
> >>>>>>>>> volumes
> >>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
> >> in).
> >>>>> There
> >>>>>>>>> is
> >>>>>>>>>>> no
> >>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
> >> SAN
> >>>>>>>>> volume
> >>>>>>>>>>> and a
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
> >> that
> >>>>>>>>>>> supports
> >>>>>>>>>>>> true
> >>>>>>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
> >> are
> >>>>>>>> related
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> >>>>> multiple
> >>>>>>>>> VMs
> >>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> >> storage."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
> >> map a
> >>>>>>>>> single
> >>>>>>>>>>> SAN
> >>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
> >> have
> >>>>>>>>> multiple
> >>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> >>>>> guarantee
> >>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> >>>>> volume
> >>>>>>>>> house
> >>>>>>>>>>> more
> >>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
> >> today
> >>>>>>>>> without
> >>>>>>>>>>> any of
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> >>>>> volume
> >>>>>>>>> for
> >>>>>>>>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> >> created."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> >>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> >>>>> concerns me
> >>>>>>>>> --
> >>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
> >> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
> >> is
> >>>>> today
> >>>>>>>>>> only
> >>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
> >> have
> >>>>> not
> >>>>>>>>>>> PreSetup an
> >>>>>>>>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >> volume
> >>>>> from
> >>>>>>>>>>> VMware, it
> >>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >> volume
> >>>>> from
> >>>>>>>>> KVM,
> >>>>>>>>>>> it uses
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> >> thinking
> >>>>>>>>> here
> >>>>>>>>>>> was to
> >>>>>>>>>>>>>>> make a
> >>>>>>>>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
> >> ability
> >>>>>>>>> Edison
> >>>>>>>>>>> added
> >>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> >> more
> >>>>>>>>> specificy,
> >>>>>>>>>>> though:
> >>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> >> Mike
> >>>>>>>>> Tutkowski
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> >> shouldn't
> >>>>> know
> >>>>>>>>> or
> >>>>>>>>>>> care
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
> >> answer
> >>>>>>>>>>> questions
> >>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
> >> the
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
> >> the
> >>>>>>>>> hypervisor
> >>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> >>>>> constraint
> >>>>>>>>>>> and just
> >>>>>>>>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
> >> way
> >>>>> for
> >>>>>>>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> >>>>> perform
> >>>>>>>>> the
> >>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> >> John
> >>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> >>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
> >> be
> >>>>> able to
> >>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> >>>>>>>>> CloudStack
> >>>>>>>>>>> (e.g. the
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> >> operator has
> >>>>>>>>> chosen
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> >> reasoning
> >>>>>>>>> make
> >>>>>>>>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> >> creating a
> >>>>> SAN
> >>>>>>>>>>> volume via
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> >> hypervisor)
> >>>>> would
> >>>>>>>>>>> work unless
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
> >> say,
> >>>>> an
> >>>>>>>>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
> >> of
> >>>>>>>>>>> CloudStack, an
> >>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
> >> and
> >>>>>>>>> exposes it
> >>>>>>>>>>>> through
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
> >> over
> >>>>> to
> >>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
> >> LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
> >> to my
> >>>>>>>>>>> changes, we
> >>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
> >> the
> >>>>>>>>>>> necessary
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> >> maybe
> >>>>> have
> >>>>>>>>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> >>>>> support
> >>>>>>>>> from
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> >> specifically
> >>>>> to
> >>>>>>>>> create
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> SR
> >>>>>>>>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> >> management
> >>>>> plugin
> >>>>>>>>>>> should
> >>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> >> determine
> >>>>>>>>> available
> >>>>>>>>>>> for
> >>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >>>>>>>>> interrogation of
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> >> device
> >>>>> itself
> >>>>>>>>>>> shouldn't
> >>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> >> simply be
> >>>>> able
> >>>>>>>>> to
> >>>>>>>>>>>> answer
> >>>>>>>>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
> >> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> >> SolidFire
> >>>>>>>>> driver
> >>>>>>>>>>> (which
> >>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> >>>>> done, a
> >>>>>>>>> new
> >>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> >> Offering
> >>>>>>>>> that
> >>>>>>>>>>>> references
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> >> Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> >>>>> CloudStack
> >>>>>>>>>>> volume. This
> >>>>>>>>>>>>>>> leads
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> >>>>> CloudStack
> >>>>>>>>>>> volume to a
> >>>>>>>>>>>>>>> VM
> >>>>>>>>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> >> framework
> >>>>> calling
> >>>>>>>>> the
> >>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
> >> case, a
> >>>>>>>>> SAN).
> >>>>>>>>>>> The
> >>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> >> applicable
> >>>>> data
> >>>>>>>>>>> (like the
> >>>>>>>>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> >> invoked if
> >>>>> the
> >>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> >> attach,
> >>>>> the
> >>>>>>>>>>> volume
> >>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
> >> and
> >>>>> the
> >>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> >> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
> >> is run
> >>>>> and
> >>>>>>>>>>> detects
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
> >> the
> >>>>> form
> >>>>>>>>> of a
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
> >> PM,
> >>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> >> sync on
> >>>>>>>>>>> terminology,
> >>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> >> physical
> >>>>>>>>>>> allocation on
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
> >> try to
> >>>>>>>>> mean
> >>>>>>>>>>> 'SAN
> >>>>>>>>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
> >> end
> >>>>> user
> >>>>>>>>> can
> >>>>>>>>>>> make in
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
> >> PM,
> >>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> >> very
> >>>>> well
> >>>>>>>>> make
> >>>>>>>>>>> sense, but
> >>>>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
> >> Edison
> >>>>> in on
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> >>>>>>>>> approach I
> >>>>>>>>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
> >> PM,
> >>>>>>>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> >> combining
> >>>>> two
> >>>>>>>>>>> distinct
> >>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> >>>>> protocols.
> >>>>>>>>> In
> >>>>>>>>>>> both
> >>>>>>>>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> >>>>> system to
> >>>>>>>>>>>> create/delete
> >>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> >> requires
> >>>>> the
> >>>>>>>>>>> volume to
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> >>>>> context.
> >>>>>>>>> To
> >>>>>>>>>>> ensure
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
> >> these
> >>>>>>>>>>> definitions,
> >>>>>>>>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> >>>>> correct?
> >>>>>>>>>>> Minimally,
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> >> storage
> >>>>> device
> >>>>>>>>> to
> >>>>>>>>>>> move bits
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >>>>>>>>> Optionally, a
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> >>>>> Therefore,
> >>>>>>>>>>> we can have
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
> >> SR,
> >>>>> and
> >>>>>>>>> we
> >>>>>>>>>>> can have a
> >>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> >> CloudStack is
> >>>>>>>>> capable
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> >>>>> CloudStack
> >>>>>>>>> may
> >>>>>>>>>>> be able
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> >> leave it
> >>>>>>>>>>> unmanaged by
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
> >> and
> >>>>> the
> >>>>>>>>>>> operator has
> >>>>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> >> CloudStack).
> >>>>> Does
> >>>>>>>>> my
> >>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> >> above
> >>>>> are
> >>>>>>>>>>> reasonable, it
> >>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> >>>>> concerns
> >>>>>>>>> from
> >>>>>>>>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> >>>>> attached
> >>>>>>>>> driver
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
> >> and
> >>>>>>>>> establish
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> >> separate
> >>>>> that
> >>>>>>>>> is
> >>>>>>>>>>>>>>> associated
> >>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> >>>>> hypervisor
> >>>>>>>>>>> drivers can
> >>>>>>>>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> >>>>> encounter
> >>>>>>>>> a
> >>>>>>>>>>> device
> >>>>>>>>>>>> managed
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> >>>>> exposed
> >>>>>>>>> by
> >>>>>>>>>>> the driver to
> >>>>>>>>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
> >> make
> >>>>>>>>> sense,
> >>>>>>>>>>> then we can
> >>>>>>>>>>>>>>> sit
> >>>>>>>>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> >>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> >>>>> between
> >>>>>>>>> the
> >>>>>>>>>>> current
> >>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> >> has to
> >>>>> go in
> >>>>>>>>> and
> >>>>>>>>>>> create
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> >> point in
> >>>>> time,
> >>>>>>>>>>> the
> >>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> >>>>> allocated).
> >>>>>>>>>>> Future
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
> >> was
> >>>>> created
> >>>>>>>>>>> behind the
> >>>>>>>>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
> >> has
> >>>>> to
> >>>>>>>>> go in
> >>>>>>>>>>> and
> >>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
> >> and
> >>>>> delete
> >>>>>>>>>>> volumes on
> >>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> >> enabled via
> >>>>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> >> volume
> >>>>> that
> >>>>>>>>> was
> >>>>>>>>>>> created,
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> >>>>> volume.
> >>>>>>>>> After
> >>>>>>>>>>> this is
> >>>>>>>>>>>>>>> done,
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> >> question is
> >>>>> called.
> >>>>>>>>>>> No
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> >>>>> volume
> >>>>>>>>> was
> >>>>>>>>>>> just
> >>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> >> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 3:21
> >>>>> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
> >> so I
> >>>>> should
> >>>>>>>>>>> probably
> >>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
> >> that is
> >>>>>>>>> pre-
> >>>>>>>>>>> allocated
> >>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
> >> a
> >>>>>>>>> hypervisor
> >>>>>>>>>>> data
> >>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> >> XenServer
> >>>>> SR),
> >>>>>>>>> then
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> >> CloudStack.
> >>>>> Disks
> >>>>>>>>> (VDI)
> >>>>>>>>>>> are later
> >>>>>>>>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> >> needed. In
> >>>>>>>>> these
> >>>>>>>>>>> cases, the
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >> structure is
> >>>>>>>>> already
> >>>>>>>>>>> in place
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> >>>>> structure
> >>>>>>>>> to
> >>>>>>>>>>> the VM in
> >>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> >> Storage
> >>>>> that is
> >>>>>>>>> not
> >>>>>>>>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
> >> this
> >>>>> could
> >>>>>>>>> be a
> >>>>>>>>>>> SAN
> >>>>>>>>>>>> (not
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> >> hypervisor
> >>>>> data
> >>>>>>>>>>> structure
> >>>>>>>>>>>>>>> must be
> >>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> >>>>> because
> >>>>>>>>> these
> >>>>>>>>>>> types of
> >>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
> >> data
> >>>>>>>>>>> structure by an
> >>>>>>>>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> >>>>> XenServer
> >>>>>>>>> for
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
> >> to the
> >>>>> VM
> >>>>>>>>> in
> >>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 3:13
> >>>>> PM,
> >>>>>>>>>>> John Burwell
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >>>>> implementation of
> >>>>>>>>> the
> >>>>>>>>>>> Dynamic
> >>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> >> which
> >>>>> why I
> >>>>>>>>> ask
> >>>>>>>>>>> about the
> >>>>>>>>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> >> what is
> >>>>> the
> >>>>>>>>>>> definition
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
> >> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
> >> iSCSI
> >>>>> type
> >>>>>>>>> is
> >>>>>>>>>>>> uniquely
> >>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> >> Primary
> >>>>>>>>> Storage
> >>>>>>>>>>> that is
> >>>>>>>>>>>>>>> directly
> >>>>>>>>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> >> skip the
> >>>>>>>>> step of
> >>>>>>>>>>> going
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
> >> based
> >>>>> on
> >>>>>>>>> that
> >>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
> >> only
> >>>>>>>>>>> supported for
> >>>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> >> must
> >>>>> first go
> >>>>>>>>> to
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
> >> what
> >>>>> RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> >> at
> >>>>> 2:13
> >>>>>>>>> PM,
> >>>>>>>>>>> John
> >>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
> >> the
> >>>>> code,
> >>>>>>>>> what
> >>>>>>>>>>> is the
> >>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> >> Why
> >>>>> isn't
> >>>>>>>>> RBD
> >>>>>>>>>>> considered
> >>>>>>>>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> >> 3:46
> >>>>> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> >>>>> storage is
> >>>>>>>>>>> defined in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> >>>>>>>>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
> >> //
> >>>>> local
> >>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> NetworkFilesystem(true),
> >>>>>>>>> //
> >>>>>>>>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> >>>>> shared
> >>>>>>>>> LUN,
> >>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
> >> e.g.,
> >>>>> ZFS
> >>>>>>>>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
> >> iso
> >>>>> image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> >>>>> XenServer
> >>>>>>>>> local
> >>>>>>>>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> >>>>> VMware
> >>>>>>>>> VMFS
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
> >> for
> >>>>>>>>>>> XenServer, Storage
> >>>>>>>>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> >>>>> XenServer
> >>>>>>>>> local
> >>>>>>>>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> >>>>> dynamic,
> >>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> StoragePoolType(boolean
> >>>>>>>>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> >> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> >>>>> isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013 at
> >>>>> 1:41
> >>>>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> >> let's say
> >>>>>>>>> another
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> >> leverage its
> >>>>>>>>> Quality
> >>>>>>>>>>> of
> >>>>>>>>>>>> Service
> >>>>>>>>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >> storage, as
> >>>>> well.
> >>>>>>>>>>> They would
> >>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> >> made
> >>>>> the
> >>>>>>>>>>> necessary
> >>>>>>>>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
> >> plug-
> >>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>>> at
> >>>>>>>>> 1:39
> >>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> >> imply
> >>>>> the
> >>>>>>>>>>> XenServer code
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> >>>>> attach
> >>>>>>>>> logic is
> >>>>>>>>>>> now aware
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
> >> an
> >>>>>>>>>>> implementation of
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
> >> to 4.2
> >>>>>>>>> with
> >>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
> >> a
> >>>>> new
> >>>>>>>>>>> framework that
> >>>>>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> >>>>> dynamically.
> >>>>>>>>>>> However, when I
> >>>>>>>>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> >> realized
> >>>>> that
> >>>>>>>>> it
> >>>>>>>>>>> was not
> >>>>>>>>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> >> CloudStack
> >>>>> could
> >>>>>>>>> do
> >>>>>>>>>>> with these
> >>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> >>>>> recognize
> >>>>>>>>> this
> >>>>>>>>>>> new type of
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >>>>> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>>> at
> >>>>>>>>> 1:28
> >>>>>>>>>>> PM, John
> >>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
> >> odd
> >>>>> to me
> >>>>>>>>>>> that any
> >>>>>>>>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
> >> care
> >>>>>>>>> about
> >>>>>>>>>>> details. I
> >>>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
> >> of
> >>>>>>>>> operations
> >>>>>>>>>>> that can be
> >>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> implementations to
> >>>>>>>>>>> allocate
> >>>>>>>>>>>> space/create
> >>>>>>>>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
> >> this
> >>>>>>>>>>> dependency, we are
> >>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
> >> will
> >>>>> make
> >>>>>>>>> the
> >>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
> >> I
> >>>>>>>>>>> understanding that the
> >>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >>>>>>>>>>> CitrixResourceBase
> >>>>>>>>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>>> 2013 at
> >>>>>>>>> 3:12
> >>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
> >> this
> >>>>> in a
> >>>>>>>>> bit
> >>>>>>>>>>> more
> >>>>>>>>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
> >> and
> >>>>> aside
> >>>>>>>>> from
> >>>>>>>>>>> one setup
> >>>>>>>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
> >> a
> >>>>>>>>> volume on
> >>>>>>>>>>> the
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
> >> data
> >>>>>>>>> structure
> >>>>>>>>>>> to make
> >>>>>>>>>>>> use
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> >>>>>>>>> XenServer
> >>>>>>>>>>> or a
> >>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> >>>>> storage
> >>>>>>>>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
> >> new
> >>>>>>>>> storage
> >>>>>>>>>>> framework,
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
> >> easily
> >>>>>>>>> create
> >>>>>>>>>>> a 1:1
> >>>>>>>>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
> >> the
> >>>>> VM's
> >>>>>>>>> data
> >>>>>>>>>>> disk
> >>>>>>>>>>>>>>> (necessary
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
> >> write
> >>>>> a
> >>>>>>>>> plug-
> >>>>>>>>>>> in that is
> >>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> >>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
> >> that
> >>>>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>> framework
> >>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >>>>>>>>> hypervisor-
> >>>>>>>>>>> specific data
> >>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
> >> the
> >>>>> case,
> >>>>>>>>> I've
> >>>>>>>>>>> been
> >>>>>>>>>>>>>>> enhancing
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
> >> and
> >>>>>>>>>>> submitted. I've got
> >>>>>>>>>>>>>>> ESX(i)
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
> >> the
> >>>>> 4.2
> >>>>>>>>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> >> help a
> >>>>> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> >> 3,
> >>>>> 2013 at
> >>>>>>>>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> >>>>> plug-in -
> >>>>>>>>> by
> >>>>>>>>>>> itself - is
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
> >> with
> >>>>> the
> >>>>>>>>>>> volume-attach
> >>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> >> calls
> >>>>> into
> >>>>>>>>> the
> >>>>>>>>>>> plug-in to
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
> >> time
> >>>>>>>>> comes to
> >>>>>>>>>>> attach the
> >>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
> >> smart
> >>>>>>>>> enough
> >>>>>>>>>>> to recognize
> >>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> >>>>> volume
> >>>>>>>>> has
> >>>>>>>>>>> just been
> >>>>>>>>>>>>>>> created)
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> >> (for
> >>>>>>>>>>> XenServer) or a
> >>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
> >> was
> >>>>> just
> >>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> >>>>> spending
> >>>>>>>>> most
> >>>>>>>>>>> of my time
> >>>>>>>>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> >>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> >> clear it
> >>>>> up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun 3,
> >>>>> 2013
> >>>>>>>>> at
> >>>>>>>>>>> 12:48 PM,
> >>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> >> explain
> >>>>>>>>> why
> >>>>>>>>>>> the the storage
> >>>>>>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> >> 2013,
> >>>>> at
> >>>>>>>>> 1:21
> >>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> >>>>> ultimately I
> >>>>>>>>>>> would like to
> >>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
> >> I
> >>>>> think
> >>>>>>>>> I'm
> >>>>>>>>>>> just out of
> >>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> >> now this
> >>>>>>>>> plug-in
> >>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> >> to 4.2
> >>>>>>>>> feature
> >>>>>>>>>>> freeze, I
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
> >> to
> >>>>> be
> >>>>>>>>> clear,
> >>>>>>>>>>> this is
> >>>>>>>>>>>> all
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>> Compute
> >>>>>>>>>>> Offerings post
> >>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun 3,
> >>>>>>>>> 2013
> >>>>>>>>>>> at 11:14 AM,
> >>>>>>>>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
> >> any
> >>>>> plan
> >>>>>>>>> on
> >>>>>>>>>>> supporting
> >>>>>>>>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> >> Original
> >>>>>>>>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
> >> "Mike
> >>>>>>>>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >>>>>>>>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> >>>>> Monday,
> >>>>>>>>> June
> >>>>>>>>>>> 3, 2013
> >>>>>>>>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
> >> Re:
> >>>>>>>>> [MERGE]
> >>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
> >> on
> >>>>>>>>> merging
> >>>>>>>>>>> Wei's feature
> >>>>>>>>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
> >> feature
> >>>>> is
> >>>>>>>>> for
> >>>>>>>>>>> KVM only,
> >>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
> >> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun
> >>>>> 3,
> >>>>>>>>> 2013
> >>>>>>>>>>> at 8:55 AM,
> >>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >> the
> >>>>> billing,
> >>>>>>>>> as
> >>>>>>>>>>> no one works
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> billing
> >>>>> by
> >>>>>>>>>>> themselves. They
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> disk_offering of
> >>>>>>>>> a
> >>>>>>>>>>> VMs and
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
> >> user
> >>>>> the
> >>>>>>>>>>> exact
> >>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>> limitation
> >>>>>>>>> for
> >>>>>>>>>>> nics as
> >>>>>>>>>>>> well.
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >> part of
> >>>>> I/O
> >>>>>>>>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
> >> my
> >>>>>>>>>>> misunstand the second
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
> >> with
> >>>>>>>>> what
> >>>>>>>>>>> you said about
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 2013/6/3
> >>>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >> Jun 3,
> >>>>> 2013,
> >>>>>>>>> at
> >>>>>>>>>>> 2:13 AM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> >> John,
> >>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> hope
> >>>>> Mike's
> >>>>>>>>>>> aswer helps you.
> >>>>>>>>>>>>>>> I am
> >>>>>>>>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> >>>>> think
> >>>>>>>>> billing
> >>>>>>>>>>> should
> >>>>>>>>>>>>>>> depend
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> limitation.
> >>>>>>>>> Please
> >>>>>>>>>>> review
> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >> the
> >>>>> IO
> >>>>>>>>>>> statistics
> >>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> virtual
> >>>>>>>>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Going by
> >>>>> the
> >>>>>>>>> AWS
> >>>>>>>>>>> model,
> >>>>>>>>>>>> customers
> >>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> provisioned
> >>>>>>>>> IOPS,
> >>>>>>>>>>> as well as,
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://aws.amazon.com/ebs/).
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> option to
> >>>>>>>>> employ
> >>>>>>>>>>> similar cost
> >>>>>>>>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> billing
> >>>>> model
> >>>>>>>>> in
> >>>>>>>>>>> the current
> >>>>>>>>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> >> Do
> >>>>> you
> >>>>>>>>> mean
> >>>>>>>>>>> IOPS runtime
> >>>>>>>>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> limitation for
> >>>>>>>>> a
> >>>>>>>>>>> running
> >>>>>>>>>>>> virtual
> >>>>>>>>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> CloudStack
> >>>>>>>>> does
> >>>>>>>>>>> not support
> >>>>>>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> (computer
> >>>>>>>>>>> offering or disk
> >>>>>>>>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> meant at
> >>>>> the
> >>>>>>>>>>> Java interface
> >>>>>>>>>>>>>>> level.
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >> more
> >>>>>>>>>>> generalize allocation
> >>>>>>>>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> describe
> >>>>> the
> >>>>>>>>>>> service guarantees
> >>>>>>>>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> >> It is a
> >>>>>>>>> good
> >>>>>>>>>>> question.
> >>>>>>>>>>>>>>> Maybe it
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> disk_io_throttling as Mike
> >>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> type, I
> >>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> >>>>> expand
> >>>>>>>>> on
> >>>>>>>>>>> my thoughts
> >>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> touch
> >>>>> points
> >>>>>>>>>>> between these two
> >>>>>>>>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >> need
> >>>>> to be
> >>>>>>>>>>> merged before
> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> coordination
> >>>>>>>>>>> between the
> >>>>>>>>>>>> branches
> >>>>>>>>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> disk_io_throttling)
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 2013/6/3
> >>>>>>>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> The
> >>>>> things I
> >>>>>>>>>>> want to
> >>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
> >> Is
> >>>>> there
> >>>>>>>>>>> value in
> >>>>>>>>>>>> capturing
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> data
> >>>>> model
> >>>>>>>>> (e.g.
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> >>>>> Should
> >>>>>>>>> there
> >>>>>>>>>>> be a common
> >>>>>>>>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> provisioning
> >>>>>>>>> at
> >>>>>>>>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
> >> How
> >>>>> are
> >>>>>>>>>>> conflicting
> >>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> hypervisor
> >>>>>>>>> and
> >>>>>>>>>>> storage device
> >>>>>>>>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> lead
> >>>>> to
> >>>>>>>>>>> believe (and
> >>>>>>>>>>>> billed)
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> storage
> >>>>>>>>> device
> >>>>>>>>>>> has been
> >>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> consistent
> >>>>>>>>>>> configuration
> >>>>>>>>>>>>>>> between a
> >>>>>>>>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> time,
> >>>>> but a
> >>>>>>>>>>> later
> >>>>>>>>>>>> modification
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >> Jun
> >>>>> 2,
> >>>>>>>>> 2013,
> >>>>>>>>>>> at 8:38 PM,
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> >> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>> believe
> >>>>>>>>> Wei's
> >>>>>>>>>>> feature deals
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>>>>>> hypervisor
> >>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> My
> >>>>> feature
> >>>>>>>>> is
> >>>>>>>>>>> focused on
> >>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> hope
> >>>>> that
> >>>>>>>>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >> Sun,
> >>>>> Jun
> >>>>>>>>> 2,
> >>>>>>>>>>> 2013 at 6:35
> >>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> My
> >>>>>>>>> opinion is
> >>>>>>>>>>> that no
> >>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> issues
> >>>>>>>>> have
> >>>>>>>>>>> been resolved
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> total
> >>>>> Ops
> >>>>>>>>> vs
> >>>>>>>>>>> discrete
> >>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> >>>>> reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>> I
> >>>>>>>>> don't
> >>>>>>>>>>> think this
> >>>>>>>>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Also,
> >>>>> how
> >>>>>>>>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> also
> >>>>>>>>> involves
> >>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> scenario
> >>>>>>>>>>> where provisioned
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>> Jun 1,
> >>>>>>>>> 2013,
> >>>>>>>>>>> at 6:47 AM,
> >>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Sure. I
> >>>>> will
> >>>>>>>>>>> change it next
> >>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> 2013/6/1
> >>>>>>>>>>> Wido den Hollander
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Exactly. I
> >>>>>>>>> have
> >>>>>>>>>>> pushed the
> >>>>>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>> anyone
> >>>>>>>>>>> object thems for
> >>>>>>>>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> For
> >>>>> the
> >>>>>>>>> sake
> >>>>>>>>>>> of clarity I
> >>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>> total
> >>>>>>>>> IOps
> >>>>>>>>>>> to R/W IOps
> >>>>>>>>>>>>>>> asap
> >>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> only
> >>>>> total
> >>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> You
> >>>>> laid
> >>>>>>>>> the
> >>>>>>>>>>> groundwork for
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> however
> >>>>>>>>>>> prevent that we
> >>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> 2013/5/31
> >>>>>>>>>>> Wido den
> >>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> +1 --
> >>>>> this
> >>>>>>>>>>> enhancement must
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> don't
> >>>>> see
> >>>>>>>>> how
> >>>>>>>>>>> it could be
> >>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> split
> >>>>> total
> >>>>>>>>> IOPS
> >>>>>>>>>>> into read
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> total
> >>>>>>>>>>> unless/until we
> >>>>>>>>>>>> decided
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> What
> >>>>> Wei
> >>>>>>>>>>> meant was merging
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 4.2
> >>>>> branch
> >>>>>>>>>>> and add Read /
> >>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> will be
> >>>>>>>>>>> released with Read
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> This is
> >>>>> to
> >>>>>>>>>>> make the May 31st
> >>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (see
> >>>>> other
> >>>>>>>>>>> threads) then it
> >>>>>>>>>>>>>>> won't
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> also
> >>>>>>>>>>> completely agree that
> >>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> disk
> >>>>> I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>> May 31,
> >>>>>>>>>>> 2013, at 9:51 AM,
> >>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi
> >>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Thanks.
> >>>>>>>>> Good
> >>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>> thought
> >>>>>>>>>>> about at the
> >>>>>>>>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> difference
> >>>>>>>>> of
> >>>>>>>>>>> read and write
> >>>>>>>>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> care
> >>>>> the
> >>>>>>>>>>> difference of sent
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> That
> >>>>>>>>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Disk
> >>>>> I/O is
> >>>>>>>>>>> much more
> >>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> Implementing
> >>>>>>>>>>> it will be some
> >>>>>>>>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> implemented
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> few
> >>>>> days.
> >>>>>>>>> For
> >>>>>>>>>>> the deadline
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> that ,
> >>>>> if
> >>>>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>> think it's
> >>>>>>>>> a
> >>>>>>>>>>> feature we
> >>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> window
> >>>>>>>>> we
> >>>>>>>>>>> have to make sure
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> in
> >>>>> 4.3,
> >>>>>>>>> that
> >>>>>>>>>>> will confuse
> >>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> 2013/5/31
> >>>>>>>>>>> Wido den
> >>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>> would
> >>>>>>>>> like to
> >>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>> nobody
> >>>>>>>>>>> object, I will
> >>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> The
> >>>>>>>>> purpose
> >>>>>>>>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Virtual
> >>>>>>>>>>> machines are running
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> share
> >>>>>>>>> strage).
> >>>>>>>>>>> Because of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> iops),
> >>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> one
> >>>>> VM
> >>>>>>>>> has
> >>>>>>>>>>> large disk
> >>>>>>>>>>>>>>> operation,
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> other
> >>>>> VMs
> >>>>>>>>>>> running on the
> >>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >> is
> >>>>>>>>> neccesary
> >>>>>>>>>>> to set the
> >>>>>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Looking at
> >>>>>>>>> the
> >>>>>>>>>>> code I see
> >>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Qemu
> >>>>> and
> >>>>>>>>>>> libvirt support
> >>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps
> >>>>>>>>> which
> >>>>>>>>>>> could benefit a
> >>>>>>>>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> It's
> >>>>> also
> >>>>>>>>>>> strange, in the
> >>>>>>>>>>>>>>> polling
> >>>>>>>>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps,
> >>>>> but
> >>>>>>>>> on
> >>>>>>>>>>> the throttling
> >>>>>>>>>>>>>>> side
> >>>>>>>>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>> IOps
> >>>>>>>>> are
> >>>>>>>>>>> usually much
> >>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> like a
> >>>>> valid
> >>>>>>>>>>> use-case where
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps
> >>>>> vs
> >>>>>>>>> Read
> >>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Since
> >>>>> this
> >>>>>>>>> only
> >>>>>>>>>>> supports KVM
> >>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> value
> >>>>> to at
> >>>>>>>>>>> least have the
> >>>>>>>>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> this
> >>>>> later
> >>>>>>>>>>> would be a lot of
> >>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> a
> >>>>>>>>> hypervisor
> >>>>>>>>>>> doesn't
> >>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> write
> >>>>> you
> >>>>>>>>> can
> >>>>>>>>>>> always sum
> >>>>>>>>>>>> both
> >>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Can
> >>>>> you
> >>>>>>>>>>> explain why you
> >>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> The
> >>>>>>>>> feature
> >>>>>>>>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (1) set
> >>>>> the
> >>>>>>>>>>> maximum rate of
> >>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (2)
> >>>>> change
> >>>>>>>>> the
> >>>>>>>>>>> maximum rate
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (3)
> >>>>> limit
> >>>>>>>>> the
> >>>>>>>>>>> disk rate
> >>>>>>>>>>>> (total
> >>>>>>>>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> JIRA
> >>>>> ticket:
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>> 1192
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> FS (I
> >>>>> will
> >>>>>>>>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Merge
> >>>>>>>>> check
> >>>>>>>>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >> Did
> >>>>> you
> >>>>>>>>>>> check the branch's
> >>>>>>>>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >> Are
> >>>>>>>>> there
> >>>>>>>>>>> new dependencies
> >>>>>>>>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >>>>> What
> >>>>>>>>>>> automated testing
> >>>>>>>>>>>> (unit
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Unit
> >>>>> tests
> >>>>>>>>> are
> >>>>>>>>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >>>>> What
> >>>>>>>>>>> testing has been done
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (1) set
> >>>>> the
> >>>>>>>>>>> bytes rate and
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (2)
> >>>>> VM
> >>>>>>>>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> deploy,
> >>>>>>>>> stop,
> >>>>>>>>>>> start, reboot,
> >>>>>>>>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (3)
> >>>>>>>>> Volume
> >>>>>>>>>>> operations,
> >>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Attach,
> >>>>>>>>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> To
> >>>>> review
> >>>>>>>>> the
> >>>>>>>>>>> code, you can
> >>>>>>>>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Best
> >>>>>>>>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> [2]
> >>>>>>>>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> >> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>>> VM
> >>>>>>>>> Disk
> >>>>>>>>>>> I/O
> >>>>>>>>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Mike
> >>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Senior
> >>>>>>>>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Advancing
> >>>>>>>>> the
> >>>>>>>>>>> way the world
> >>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Advancing
> >>>>> the
> >>>>>>>>> way
> >>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Advancing the
> >>>>>>>>> way
> >>>>>>>>>>> the world uses
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >> the
> >>>>> way
> >>>>>>>>> the
> >>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >> the
> >>>>> way
> >>>>>>>>> the
> >>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>>>> Developer, SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>>> the
> >>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>>> the
> >>>>>>>>> world
> >>>>>>>>>>> uses the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way the
> >>>>>>>>> world
> >>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >> the
> >>>>> world
> >>>>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>> Developer,
> >>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>>> world
> >>>>>>>>> uses
> >>>>>>>>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>>> uses
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>>> uses
> >>>>>>>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >> the
> >>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *(tm)*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *(tm)*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *(tm)*****
> >>>>
> >>>>
> >>>>
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*****
> >>>>
> >>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>
> >>>> e: mike.tutkowski@solidfire.com****
> >>>>
> >>>> o: 303.746.7302****
> >>>>
> >>>> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*****
> >>>>
> >>>>
> >>>>
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*****
> >>>>
> >>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>
> >>>> e: mike.tutkowski@solidfire.com****
> >>>>
> >>>> o: 303.746.7302****
> >>>>
> >>>> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*****
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *(tm)*
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: John Burwell [mailto:jburwell@basho.com]
> Sent: Monday, June 10, 2013 12:05 PM
> To: Edison Su
> Cc: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Edison,
>
> TL;DR Always give operators/system administrators the ability to manage
> things themselves, if they like.
>
> As more manageable drivers emerge, I think it will be import to allow
> operators to opt-out of CloudStack device management. Therefore, a driver
> declares whether or not it can support device management (many, in fact
> most, won't). For those drivers that do support management, allow
> operators to opt-in/out of CloudStack management on a per device basis. To
> support this notion, we need a flag on the storage_pool table and associated
> classes to indicate whether or not the operator wants CloudStack to manage
> the device.
>
> Does this explanation make more sense?
We already hand out almost all the storage related operations to storage driver, why cloudstack needs to care about it's managed or un-managed?
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
>
> > I don't understand why we need to add a "managed" in the UI and in
> "storage_pool" table?
> > AFAIK, what you guys trying to do is to enable
> attachVolume/DettachVolume if the volume is created on Solidfire, right?
> > If the volume is created on Solidfire already, by createAsync, then why we
> can't just send out all the information about the volume(it's volumeTO which
> will be returned by each storage driver) to hypervisor, then at the hypervisor
> side, check the volumeTO, if the volumeTO is managed, then doing whatever
> necessary operations(create SR, create VDI etc), then attach the VDI to
> volume.
> >
> >
> >> -----Original Message-----
> >> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >> Sent: Monday, June 10, 2013 11:10 AM
> >> To: John Burwell
> >> Cc: Edison Su; dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>
> >> I believe we're on the same page, John.
> >>
> >> I would add a new parameter to the createStoragePool API command
> (let's
> >> call it "managed" and it can be true or false). If you pass in "true", but
> >> the driver does not support being managed by CloudStack, an exception is
> >> thrown by the driver. If you pass in "false", but the driver does not
> >> support being managed outside of CloudStack, an exception is thrown by
> the
> >> driver. It is possible one could write a driver that can be managed both
> >> inside or outside of CloudStack (the SolidFire driver requires being
> >> managed only by CloudStack).
> >>
> >> This managed value would end up in the storage_pool table in a new
> column
> >> ("managed").
> >>
> >> An example usage: After the storage framework calls on the driver's
> >> createAsync method, it will check to see if the driver is managed. If the
> >> driver is managed, the storage framework will invoke the proper (new)
> >> command of the hypervisor in use to set up, say, the SR for the SAN
> volume.
> >>
> >> There may be other cases when the isManaged() method is invoked on
> the
> >> driver and - if true is returned - the storage framework sends a particular
> >> command to the hypervisor in use.
> >>
> >> ********************
> >>
> >> As far as reworking the default storage plug-in (which sends commands to
> >> hypervisors) for 4.2, I would check with Edison on his schedule.
> >>
> >> Thanks!
> >>
> >>
> >> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>
> >>> Mike,
> >>>
> >>> I want to make sure we are on the same page regarding isManaged. I
> see
> >>> the addition of the following operations:
> >>>
> >>> * DataStoreDriver#supportsManagement() : boolean -- indicates
> whether
> >> or
> >>> not the driver support/implements the management functions
> >>> * DataStore#isManaged() : boolean -- indicates whether or not a device
> is
> >>> managed by CloudStack
> >>>
> >>> Only DataStores associated with DataStoreDrivers that support
> >> management
> >>> would be able to have the management flag enabled. This behavior
> allows
> >>> operators to declare their intention to allow CloudStack to manage a
> >>> device. This behavior would need to be exposed as flag on the HTTP API.
> >>> Additionally, this flag would only be settable when a device is
> >>> defined/created in the system (i.e. once a device is managed by
> CloudStack,
> >>> you can't unmanaged it without dissociating it from CloudStack). In the
> >>> future, we may consider relaxing that rule, but, in the short term, it
> >>> greatly decreases the alternative flows.
> >>>
> >>> In terms of removing the dependencies from Storage to Hypervisor, I
> >> would
> >>> really like to see this accomplished in 4.2. It will be much more
> >>> difficult to address this issue post 4.2 because we will have more plugins
> >>> to address. I also see this issue as a critical architectural issues that
> >>> for the management server. I recommend we identify the
> dependencies
> >> that
> >>> need to be inverted, and see if we can divide the work across a few
> more
> >>> resources.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> >> <mi...@solidfire.com>
> >>> wrote:
> >>>
> >>> Sounds good, Edison :)
> >>>
> >>> I can implement John's isManaged() logic and resubmit my code for
> review.
> >>>
> >>> Perhaps in 4.3 I could address the direct-attach scenario. If it requires
> >>> some refactoring of the storage framework, then I could do that.
> >>>
> >>> Also, the default storage plug-in does have hypervisor logic in it in 4.2.
> >>> I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> >>> with the above work I mentioned.
> >>>
> >>> Are you OK with that, guys?
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com>
> wrote:
> >>>
> >>>> Ok, I am Ok with the way you are talking about in 4.2.****
> >>>>
> >>>> ** **
> >>>>
> >>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>> *Sent:* Sunday, June 09, 2013 8:59 PM
> >>>> *To:* dev@cloudstack.apache.org
> >>>> *Cc:* Edison Su; John Burwell
> >>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>>>
> >>>> ** **
> >>>>
> >>>> Another thing we might want to consider for 4.2 is that the way the
> code
> >>>> is currently implemented in master, I don't think it's even possible to
> >>>> write a plug-in that connects storage directly to a VM because the
> >>>> CloudStack storage framework will call the "attach" method of the
> >>>> hypervisor in use and this method will fail because it assumes there is
> >>>> (talking Xen here) an SR existent and in this situation there isn't one.*
> >>>> ***
> >>>>
> >>>> ** **
> >>>>
> >>>> My thinking is that we should ignore the direct-attach-of-volume-to-
> VM
> >>>> use case for 4.2. I think it will require too much of a rewrite at this
> >>>> point in development (fewer than three weeks left until code
> >> freeze).****
> >>>>
> >>>> ** **
> >>>>
> >>>> If the plug-in implements an isManaged() method (as John suggested),
> >> the
> >>>> storage framework can invoke the necessary commands on the
> >> hypervisor in
> >>>> use.****
> >>>>
> >>>> ** **
> >>>>
> >>>> For example, the createAsync method is called. The storage plug-in
> >>>> creates a SAN volume and updates the CS DB. The storage framework
> >> then asks
> >>>> the driver if it's managed. If it is, the storage framework executes a
> >>>> command against the hypervisor in use to have it create, say, an SR
> (with
> >> a
> >>>> single VDI that takes up the entire SR). Now, when the storage
> >> framework
> >>>> executes the "attach" command against the hypervisor, this command
> will
> >>>> work, as well (because that logic assumes the prior existence of an SR
> and
> >>>> it will, in fact, be present).****
> >>>>
> >>>> ** **
> >>>>
> >>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:****
> >>>>
> >>>> Hi Edison and John,****
> >>>>
> >>>> ** **
> >>>>
> >>>> I wanted to point out something I thought was relevant to this
> >>>> conversation.****
> >>>>
> >>>> ** **
> >>>>
> >>>> My plug-in supports both XenServer and ESX(i).****
> >>>>
> >>>> ** **
> >>>>
> >>>> These two hypervisors handle their respective hypervisor data
> structures
> >>>> (XenServer SR or ESX(i) datastore) quite differently.****
> >>>>
> >>>> ** **
> >>>>
> >>>> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> >>>> when you issue a remove-SR command to XenServer.****
> >>>>
> >>>> ** **
> >>>>
> >>>> When you later want to access the contents of that data, it is still
> >>>> safely stored on the LUN and can be accessed from XenServer (from
> the
> >> same
> >>>> or different cluster (what XenServer calls a resource pool)) by creating
> a
> >>>> new SR that is based on the same IQN and LUN.****
> >>>>
> >>>> ** **
> >>>>
> >>>> XenServer SRs exist at the cluster level.****
> >>>>
> >>>> ** **
> >>>>
> >>>> In ESX(i), the datastore that resides on the SAN LUN is actually
> >>>> destroyed when you issue a remove-datastore command to ESX(i). In
> >> other
> >>>> words, the contents of the SAN LUN are destroyed.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ESX(i) datastores exist at the datacenter level.****
> >>>>
> >>>> ** **
> >>>>
> >>>> The XenServer behavior (not destroying the contents of the LUN when
> >> the
> >>>> SR is removed) is desirable; the ESX(i) behavior (destroying the
> contents
> >>>> of the LUN when the datastore is removed) is not desirable.****
> >>>>
> >>>> ** **
> >>>>
> >>>> I can get around the ESX(i) behavior by removing the referenced IQN
> >> from
> >>>> each host in the cluster when the CS storage framework notifies my
> plug-
> >> in
> >>>> of a detach-volume event. When the CS volume is later attached again
> >>>> (either to the same or a different ESX(i) cluster), I can have each host in
> >>>> the relevant ESX(i) cluster reference the IQN and the datastore will be
> >>>> back again.****
> >>>>
> >>>> ** **
> >>>>
> >>>> This sequence of events is critical for my plug-in to support both
> >>>> environments:****
> >>>>
> >>>> *XenServer flow (events for Driver class):*****
> >>>>
> >>>> createAsync (called right before first attach) = creates SAN
> volume****
> >>>>
> >>>> preAttachVolume = creates SR and creates VDI or just introduces
> SR****
> >>>>
> >>>> (CS volume gets attached and later detached)****
> >>>>
> >>>> postDetachVolume = removes SR (this does not delete data on SAN
> >> volume)**
> >>>> **
> >>>>
> >>>> deleteAsync (can be called when volume is not attached) = deletes the
> >> SAN
> >>>> volume****
> >>>>
> >>>> *VMware flow (events for Driver class):*****
> >>>>
> >>>> createAsync (called right before first attach) = creates SAN volume,
> >>>> creates datastore, and creates VMDK file****
> >>>>
> >>>> preAttachVolume = adds iSCSI target to each host in cluster****
> >>>>
> >>>> (CS volume gets attached and later detached)****
> >>>>
> >>>> postDetachVolume = removes iSCSI target from each host in
> cluster****
> >>>>
> >>>> deleteAsync (can be called when volume is not attached) = deletes
> >>>> datastore and deletes the SAN volume****
> >>>>
> >>>> ************************
> >>>>
> >>>> So, the way I have it written up here, the driver is talking to the
> >>>> hypervisor. As John has pointed out, this may not be ideal.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Whatever we decide on, it needs to be able to support this kind of flow.
> >>>> :)****
> >>>>
> >>>> ** **
> >>>>
> >>>> We could go John's route, where the driver says if it's managed or
> not.**
> >>>> **
> >>>>
> >>>> ** **
> >>>>
> >>>> If it is, then the storage framework could send the necessary messages
> to
> >>>> the hypervisor in question at the right times (just migrate the
> hypervisor
> >>>> logic above from the driver to the right parts of the storage framework).
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> This is fine, except that it does not resolve Edison's use case of
> >>>> desiring to have the storage LUN potentially connected directly to a
> VM.*
> >>>> ***
> >>>>
> >>>> ** **
> >>>>
> >>>> I'm not sure we should design around that use case, though. Is it
> common
> >>>> enough? Has anyone asked for it? Is it even desirable? How are usage
> >>>> statistics gathered for such volumes? It's nice to offer the flexibility,
> >>>> but is it a good idea in this case?****
> >>>>
> >>>> ** **
> >>>>
> >>>> Thanks!****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>
> wrote:**
> >>>> **
> >>>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>> Sent: Friday, June 07, 2013 4:08 PM
> >>>>> To: dev@cloudstack.apache.org
> >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>
> >>>>> Hi John,
> >>>>>
> >>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
> >>>> volume
> >>>>> to use it. Same story for VMware: ESX(i) must layer a datastore on top
> >>>> of
> >>>>> the iSCSI volume to use it. I look at it like they are layering a
> >>>> clustered
> >>>>> file system on the SAN volume so the hypervisors can share access to
> >> the
> >>>>> contents of the volume.
> >>>>>
> >>>>> In 4.1, by the time the hypervisor-attach-volume logic is called, the
> >>>> SR or
> >>>>> datastore has already been created (usually manually by an admin).
> >>>>>
> >>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
> >>>>> environment where each CloudStack volume that a user creates is
> >> mapped
> >>>>> to a
> >>>>> single SAN volume (via the SR).
> >>>>>
> >>>>> The question comes down to who should allocate the SR dynamically.
> >>>>>
> >>>>> We could have the storage framework ask the storage plug-in if it is
> >>>>> managed. If it is, then the storage framework could send a message
> to
> >>>> the
> >>>>> hypervisor in question to create (let's talk Xen here) the SR ahead of
> >>>>> time. Then, when the storage framework next sends the attach-
> volume
> >>>>> command
> >>>>> to the hypervisor, it should work without changes to that attach-
> volume
> >>>>> logic (because - from the point of view of the attach logic - the SR is
> >>>>> already existent, as expected).
> >>>>>
> >>>>> Now, as Edison has pointed out, this limits the power of the storage
> >>>>> plug-in. A storage plug-in in this model cannot directly attach storage
> >>>> to
> >>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We
> need
> >> to
> >>>>> make a call on that.
> >>>>
> >>>>
> >>>> Yes, I think we need to address this usage case, and won't limit the
> >>>> power of the storage plug-in.
> >>>> John, what's your idea to fit this usage case into your abstraction?
> >>>>
> >>>>
> >>>>>
> >>>>> If we want to give more power to the storage vendor, we would have
> to
> >>>>> have
> >>>>> the storage framework call into the storage plug-in to send the
> >>>> appropriate
> >>>>> attach commands to the hypervisor.
> >>>>>
> >>>>> Let's discuss and come to a consensus (or at least agree on a path) as
> >>>> soon
> >>>>> as we can. :)
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> My understanding is the Xen expects an SR structure on all ISCSI
> >>>> devices
> >>>>>> -- at least that is how I read the code in your patch. Is my
> >>>> understanding
> >>>>>> correct? If so, the Xen plugin should be able to query the storage
> >>>> device
> >>>>>> to determine the presence of the SR structure and create it if it
> >>>> does not
> >>>>>> exist. Am I missing something in the implementation that makes
> that
> >>>> type
> >>>>>> of implementation impossible?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Yeah, if a storage vendor wanted to attach a volume directly to a
> >>>> VM in
> >>>>>> 4.2
> >>>>>>> today, it would probably fail because the attach-volume logic
> >>>> assumes
> >>>>> the
> >>>>>>> existence of the necessary hypervisor data structure (ex. SR on
> >>>> Xen).
> >>>>>>>
> >>>>>>> If we wanted to enable such an attach, we could do it the way
> >> Edison
> >>>>>>> suggests.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>
> >>>>>>>>> As we only have three weeks until feature freeze, we should
> >> come
> >>>> to
> >>>>> a
> >>>>>>>>> consensus on this design point as soon as possible.
> >>>>>>>>>
> >>>>>>>>> Right now, if the storage framework asks my driver it is is
> >>>> managed, it
> >>>>>>>>> will say 'yes.' This means the framework will tell the driver to
> >>>>>> perform
> >>>>>>>>> its management activities. This then means the driver will call
> >>>> into
> >>>>>> the
> >>>>>>>>> host (it doesn't know which hypervisor, by the way) to perform
> >> the
> >>>>>>>> activity
> >>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
> >>>>>>>>>
> >>>>>>>>> The driver doesn't know which hypervisor it's talking to, it just
> >>>>>> sends a
> >>>>>>>>> message to the host to perform the necessary pre-attach work.
> >>>>>>>>
> >>>>>>>> Could we just expose a method like
> >> "attachVolume/dettachVolume" on
> >>>>> the
> >>>>>>>> PrimaryDataStoreDriver driver?
> >>>>>>>> For most of cases, the implementation of each driver would just
> >>>> send
> >>>>>>>> attachvolumecommand/dettachvolumecmd to hypervisor(we can
> >> put
> >>>>> the
> >>>>>>>> implementation in a base class, so that can be shared by all of
> >>>> this
> >>>>>>>> drivers), and in each hypervisor resource code, which may just call
> >>>>>>>> hypervisor's api to attach the volume to VM, while for certain
> >>>> storage,
> >>>>>>>> like SolidFire, may need to create a SR first, and create volume
> >>>> on it,
> >>>>>>>> then call hypervisor's API to attach volume to VM.
> >>>>>>>> While for some other storage vendor, may want to bypass
> >> hypervisor
> >>>>>> during
> >>>>>>>> attaching volume, so inside driver's attachvolume implementation,
> >>>> the
> >>>>>>>> driver can do some magic, such as, directly talk to an agent
> >>>> inside VM
> >>>>>>>> instance, then create a disk inside VM.
> >>>>>>>>
> >>>>>>>> How do you guys think?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> Hi John,
> >>>>>>>>>>>
> >>>>>>>>>>> How's about this:
> >>>>>>>>>>>
> >>>>>>>>>>> The driver can implement an isManaged() method. The
> >>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>> can
> >>>>>>>>>>> call into the driver to see if its managed. If it is, the
> >>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>> (which is responsible for calling into the hypervisor to attach
> >>>> the
> >>>>>>>> disk)
> >>>>>>>>>>> can call into the hypervisor to create the necessary hypervisor
> >>>> data
> >>>>>>>>>>> structure (ex. for XenServer, a storage repository).
> >>>>>>>>>>
> >>>>>>>>>> The problem here is that storage vendor may work differently
> >> with
> >>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> >>>> maybe
> >>>>>>>> other
> >>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the LUN
> >>>>> directly
> >>>>>>>> to
> >>>>>>>>>> VM instance, see the discuss(
> >>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>>>>>>>>
> >>>>>
> >>
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>>>>>>>> %3E
> >>>>>>>>>> ).
> >>>>>>>>>> So I would let storage provider to implement attach disk to VM,
> >>>>>>>> instead of
> >>>>>>>>>> implemented by cloudstack itself.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> If that's what you're going for, that works for me. By the way,
> >>>>>>>> Edison's
> >>>>>>>>>>> default storage plug-in (which handles the default storage
> >>>> behavior
> >>>>>>>> in
> >>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> >>>> talks to
> >>>>>>>>>>> hypervisors. You might want to contact him and inform him of
> >>>> your
> >>>>>>>>>> concerns
> >>>>>>>>>>> or that logic (as is) will make it to production.
> >>>>>>>>>>>
> >>>>>>>>>>> Please let me know if what I wrote in above (for my solution)
> >>>> is OK
> >>>>>>>> with
> >>>>>>>>>>> you. :)
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >>>> jburwell@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please see my responses in-line below.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I still have a bit more testing I'd like to do before I build
> >>>> up
> >>>>>>>> a
> >>>>>>>>>> patch
> >>>>>>>>>>>>> file, but this is the gist of what I've done:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * During a volume-attach operation, after
> >> VolumeManagerImpl
> >>>>> tells
> >>>>>>>>>>>>> VolumeServiceImpl to have the driver create a volume, I
> >> have
> >>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> >> driver if
> >>>> it
> >>>>>>>>>>> managed.
> >>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> >>>>>>>> whatever
> >>>>>>>>>>>>> activity is required. In my case, this includes sending a
> >>>>>>>> message to
> >>>>>>>>>> the
> >>>>>>>>>>>>> host where the VM is running to have, say XenServer, add
> a
> >>>>>>>> storage
> >>>>>>>>>>>>> repository (based on the IP address of the SAN, the IQN of
> >> the
> >>>>>>>> SAN
> >>>>>>>>>>>> volume,
> >>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
> >>>> can
> >>>>>>>> on
> >>>>>>>>>> the
> >>>>>>>>>>>>> storage repository). After this, the normal attach-volume
> >>>>>>>> message is
> >>>>>>>>>> sent
> >>>>>>>>>>>>> to the host by VolumeManagerImpl.
> >>>>>>>>>>>>
> >>>>>>>>>>>> There should be **no** code from a storage driver to a
> >>>> hypervisor.
> >>>>>>>> I
> >>>>>>>>>>>> apologize for the repetition, but we simply can not have
> >>>> hypervisor
> >>>>>>>>>>>> specific code in the storage layer. The circular dependencies
> >>>>>>>> between
> >>>>>>>>>> the
> >>>>>>>>>>>> two layers are not sustainable in the long term. Either the
> >>>>>>>>>> VirtualManager
> >>>>>>>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >>>>>>>>> coordinate
> >>>>>>>>>>>> volume creation and then populating the SR. Ideally, we can
> >>>>>>>>>> generalize the
> >>>>>>>>>>>> process flow for attaching volumes such that the Xen
> >> hypervisor
> >>>>>>>> plugin
> >>>>>>>>>>>> would only implement callbacks to perform the attach action
> >> and
> >>>>>>>> create
> >>>>>>>>>>> the
> >>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver should
> >>>> only be
> >>>>>>>>>>>> allocating space and providing information about contents
> >> (e.g.
> >>>>>>>> space
> >>>>>>>>>>>> available, space consumed, streams to a URI, file handle for a
> >>>> URI,
> >>>>>>>>>> etc)
> >>>>>>>>>>>> and capabilities.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * The reverse is performed for a detach-volume command.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in my
> >>>> driver.
> >>>>>>>>>>>> Edison's
> >>>>>>>>>>>>> default driver simply does a "return false;". We could add a
> >>>> new
> >>>>>>>>>>>> parameter
> >>>>>>>>>>>>> to the createStoragePool API command, if we want, to
> >> remove
> >>>>> the
> >>>>>>>>>>>> hard-coded
> >>>>>>>>>>>>> return values in the drivers (although my driver will
> probably
> >>>>>>>> just
> >>>>>>>>>>>> ignore
> >>>>>>>>>>>>> this parameter and always return true since it wouldn't
> make
> >>>>>>>> sense
> >>>>>>>>>> for it
> >>>>>>>>>>>>> to ever return false). We'd need another column in the
> >>>>>>>> storage_pool
> >>>>>>>>>>> table
> >>>>>>>>>>>>> to store this value.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes, I think we should have a parameter added to the
> >>>>>>>>> createStoragePool
> >>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
> >>>> configured
> >>>>>>>> for
> >>>>>>>>>>>> management when their underlying drivers support it. To
> >>>> simplify
> >>>>>>>>>> things,
> >>>>>>>>>>>> this flag should only be mutable when the DataStore is
> >>>> created. It
> >>>>>>>>>> would be
> >>>>>>>>>>>> a bit crazy to take a DataStore from managed to unmanaged
> >> after
> >>>>>>>>>> creation.
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
> >> working
> >>>>>>>> GUI
> >>>>>>>>>> for
> >>>>>>>>>>>> you
> >>>>>>>>>>>>>> to review (in that document), if you'd like to check it out.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I would like the UIs of two features reviewed together
> to
> >>>>>>>> ensure
> >>>>>>>>>>>>>>> consistency across the concepts of hypervisor throttled
> >> IOPs
> >>>>>>>> and
> >>>>>>>>>>>>>>> storage device provisioned IOPs. I see the potential for
> >>>>>>>>>> confusion,
> >>>>>>>>>>>>>>> and I think a side-by-side Ui review of these features will
> >>>>>>>> help
> >>>>>>>>>>>>>>> minimize any potential confusion.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will work
> >>>> itself
> >>>>>>>> if
> >>>>>>>>>> it
> >>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize
> hypervisor
> >>>>>>>>>> throttled
> >>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
> >> sent
> >>>>> to
> >>>>>>>>> you
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>> other day to see how the GUI looks for provisioned
> >> storage
> >>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-mail
> list
> >>>>>>>> and we
> >>>>>>>>>>>>>>> decided
> >>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> >>>>> Offering
> >>>>>>>>>> dialog.
> >>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS, so
> they
> >>>>> should
> >>>>>>>>> be
> >>>>>>>>>>> able
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say, only
> >>>> use
> >>>>>>>> Max
> >>>>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>> These fields are optional and can be ignored for storage
> >>>> that
> >>>>>>>>>> does not
> >>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
> >>>> the
> >>>>>>>>>> admin can
> >>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> >> Burst
> >>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's
> feature
> >> or
> >>>>>>>> mine,
> >>>>>>>>>> as
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> decided by the admin).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm not sure what we can do about these two features
> >>>>>>>> expressing
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express the
> >> IOPS
> >>>> for
> >>>>>>>>>> QoS in
> >>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
> >>>> read/write
> >>>>>>>>>> IOPS).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
> >>>>>>>>> Storage/Hypervisor
> >>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>> separation, but we still need to reconcile the behavior
> >> of
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> >>>>>>>> following
> >>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>> outstanding:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >>>>>>>>> read/write
> >>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>> where as storage provisioned IOPS are expressed as
> >> total
> >>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS
> attached
> >> to
> >>>>>>>>> storage
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>>>>>>>> 3. How should usage data be captured for throttled
> and
> >>>>>>>>>> provisioned
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> that will permit providers to discriminate these
> >>>> guaranteed
> >>>>>>>>>>>> operations
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
> >>>> provisioned
> >>>>>>>>> IOPS
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
> >>>> hypervisor
> >>>>>>>>>> throttled
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> >>>> solve
> >>>>>>>>>> items 1
> >>>>>>>>>>>>>>> and 2.
> >>>>>>>>>>>>>>>>> Since the two facilities would not be permitted to
> >> operate
> >>>>>>>>>> together,
> >>>>>>>>>>>>>>> they
> >>>>>>>>>>>>>>>>> do not need to be semantically compatible. I think
> item
> >> 3
> >>>>>>>> can be
> >>>>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>> with an additional flag or two on the usage records.
> As
> >>>> for
> >>>>>>>>>> Item 4,
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>> not familiar with how these two enhancements are
> >>>>> depicted in
> >>>>>>>>> the
> >>>>>>>>>>> user
> >>>>>>>>>>>>>>>>> interface. I think we need to review the UI
> >> enhancements
> >>>>> for
> >>>>>>>>>> both
> >>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >>>>> <us...@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> >>>>> 2780/2729,
> >>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >>>>>>>>> CLOUDSTACK-
> >>>>>>>>>>> 2875 ) this
> >>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
> >> and
> >>>>>>>> ask
> >>>>>>>>>> for
> >>>>>>>>>>>>>>> second
> >>>>>>>>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> >> <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I believe I understand where you're going with this,
> >>>> John,
> >>>>>>>> and
> >>>>>>>>>>> have
> >>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> >>>>> whether or
> >>>>>>>>>>> not a
> >>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> >> but an
> >>>>>>>>> actual
> >>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >>>>>>>> resources
> >>>>>>>>>> for
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> >>>>> method
> >>>>>>>>>>> added to
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns
> true,
> >>>>>>>> then the
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> framework could call the manage() (or whatever
> >> name)
> >>>>>>>>> method
> >>>>>>>>>>> (which
> >>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
> >> well)
> >>>>> and
> >>>>>>>>> this
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to
> create,
> >> say
> >>>>> on
> >>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >>>>>>>>>>> DataStoreDriver.
> >>>>>>>>>>>>>>> Since
> >>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> >> storage,
> >>>>> I
> >>>>>>>>>> would
> >>>>>>>>>>>> add
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is
> a
> >>>>> straw
> >>>>>>>>> man
> >>>>>>>>>>> term
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> >> mutable
> >>>>>>>>> property
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>>>>>>>> which can be enabled if/when the underlying
> >>>>>>>>> DataStoreDriver
> >>>>>>>>>>> can be
> >>>>>>>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>>>>>>> This approach allows operators to override
> >>>>> manageability
> >>>>>>>> of
> >>>>>>>>>>>> devices.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
> >> Xen
> >>>>>>>> plugin
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> responsible for composing DataStore/Volume
> >> methods
> >>>>> to
> >>>>>>>>>>> create any
> >>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR. There
> >>>> should
> >>>>>>>> be no
> >>>>>>>>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> >>>>>>>> earlier,
> >>>>>>>>>> such
> >>>>>>>>>>>>>>>>>>> circular
> >>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
> >> unmaintainable
> >>>>> mess.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page with
> >> you.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>>>>>>>>> <jb...@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
> >> layer
> >>>>> that
> >>>>>>>>>>>> supports n
> >>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors of m
> >>>>>>>> hypervisors.
> >>>>>>>>>>>> Such
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
> >> untestable
> >>>>>>>>> beast.
> >>>>>>>>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven to
> >> evolve
> >>>>> the
> >>>>>>>>>>> Storage
> >>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> >>>>> whether
> >>>>>>>>> or
> >>>>>>>>>>> not a
> >>>>>>>>>>>>>>> Volume
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> >> but
> >>>>> an
> >>>>>>>>> actual
> >>>>>>>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >>>>>>>> resources
> >>>>>>>>>> for
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
> >>>> needs
> >>>>>>>> to
> >>>>>>>>>>>> indicate
> >>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
> >> Finally, I
> >>>>>>>> think
> >>>>>>>>>> we
> >>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to elect to
> >>>> have
> >>>>>>>>>>>> something
> >>>>>>>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> >> capable
> >>>>>>>>> managing
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
> >>>>> unmanaged).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume should be
> >> done
> >>>>> in
> >>>>>>>>> the
> >>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed by
> the
> >>>>> Storage
> >>>>>>>>>>> layer to
> >>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> >> create a
> >>>>>>>> file,
> >>>>>>>>>> etc).
> >>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >>>>>>>>>> hypervisor, as
> >>>>>>>>>>>>>>> such,
> >>>>>>>>>>>>>>>>>>>>> should be confined to its implementation. From
> >> my
> >>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>> nothing
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
> >> content.
> >>>>> From
> >>>>>>>>> its
> >>>>>>>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides the
> >> means
> >>>>> to
> >>>>>>>>>> query
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> support the interpretation of the content by
> >> higher-
> >>>>> level
> >>>>>>>>>>> layers
> >>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> >>>>> composition
> >>>>>>>>> of
> >>>>>>>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on the
> >>>>> Volume
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> protocol
> >>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> >>>>> implemented
> >>>>>>>> in
> >>>>>>>>>> my
> >>>>>>>>>>>>>>> patch,
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >>>>>>>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
> >> the
> >>>>>>>>>>> hypervisor.
> >>>>>>>>>>>> Before
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if we're
> >>>>> dealing
> >>>>>>>>> with
> >>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>> (or
> >>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> >>>> send a
> >>>>>>>>>> "Create
> >>>>>>>>>>>>>>> SR"
> >>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK,
> we
> >>>>> would
> >>>>>>>>> then
> >>>>>>>>>>>>>>> proceed
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same and
> >> we just
> >>>>>>>> add
> >>>>>>>>>>>> another
> >>>>>>>>>>>>>>>>>>>>> command to the agent code that is called for this
> >>>>>>>> particular
> >>>>>>>>>>> type
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I created a document for a customer today that
> >>>>> outlines
> >>>>>>>>> how
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will probably
> >> be
> >>>> of
> >>>>>>>> use
> >>>>>>>>>> to
> >>>>>>>>>>>>>>> you, as
> >>>>>>>>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I have shared this document with you (you
> >> should
> >>>>> have
> >>>>>>>>>>> received
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> >>>>> thoughts,
> >>>>>>>>> but
> >>>>>>>>>>>> want
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> >>>> solidfire,
> >>>>>>>>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect
> them
> >> in a
> >>>>>>>>>>> coherent
> >>>>>>>>>>>>>>> form
> >>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> >> introduces
> >>>>> the
> >>>>>>>>>> notion
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> >> introduces
> >>>>>>>> the
> >>>>>>>>>>> notion
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages the
> >> plug-in
> >>>>> was
> >>>>>>>>>>>>>>> incomplete,
> >>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed
> iSCSI
> >>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> >> spending on
> >>>>>>>> this.
> >>>>>>>>>> :)
> >>>>>>>>>>> Do
> >>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
> >> accomplish
> >>>>>>>>> what
> >>>>>>>>>>> you're
> >>>>>>>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John
> Burwell
> >> <
> >>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> >> introduces
> >>>>> the
> >>>>>>>>>> notion
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> >> management
> >>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>> (i.e.
> >>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator
> want
> >> it
> >>>>>>>>> managed
> >>>>>>>>>>> by
> >>>>>>>>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end
> up
> >> with
> >>>>> a
> >>>>>>>>>>> semantic
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> >> managed.
> >>>>> I
> >>>>>>>>> also
> >>>>>>>>>>> want
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
> >> from
> >>>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike
> Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> >> Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> >>>>> questions
> >>>>>>>>> for
> >>>>>>>>>>> me
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> >> LUNs
> >>>>> and
> >>>>>>>>> SAN
> >>>>>>>>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
> >> LUN
> >>>>> (LUN
> >>>>>>>>> 0).
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> >> Volumes
> >>>>> and
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> >> management
> >>>>>>>>> server or
> >>>>>>>>>>>>>>> externally
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
> >> SAN
> >>>>>>>>> volume
> >>>>>>>>>>> (same
> >>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server (via
> >> the
> >>>>> plug-in)
> >>>>>>>>>>> the
> >>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> >> hypervisor.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire
> plug-
> >> in,
> >>>>> but
> >>>>>>>>>> still
> >>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
> >> already
> >>>>>>>>> today
> >>>>>>>>>>>> (prior
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
> >> and -
> >>>>> in
> >>>>>>>>> this
> >>>>>>>>>>> case -
> >>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> >> While
> >>>>> you
> >>>>>>>>> can do
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
> >> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> >> management
> >>>>> server
> >>>>>>>>> or
> >>>>>>>>>>> externally
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> >> SAN
> >>>>>>>>> volumes
> >>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
> >> in).
> >>>>> There
> >>>>>>>>> is
> >>>>>>>>>>> no
> >>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
> >> SAN
> >>>>>>>>> volume
> >>>>>>>>>>> and a
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage
> vendor
> >> that
> >>>>>>>>>>> supports
> >>>>>>>>>>>> true
> >>>>>>>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
> >> are
> >>>>>>>> related
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> >>>>> multiple
> >>>>>>>>> VMs
> >>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> >> storage."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
> >> map a
> >>>>>>>>> single
> >>>>>>>>>>> SAN
> >>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
> >> have
> >>>>>>>>> multiple
> >>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> >>>>> guarantee
> >>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> >>>>> volume
> >>>>>>>>> house
> >>>>>>>>>>> more
> >>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
> >> today
> >>>>>>>>> without
> >>>>>>>>>>> any of
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> >>>>> volume
> >>>>>>>>> for
> >>>>>>>>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> >> created."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> >>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> >>>>> concerns me
> >>>>>>>>> --
> >>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
> >> layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI
> type
> >> is
> >>>>> today
> >>>>>>>>>> only
> >>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when
> you
> >> have
> >>>>> not
> >>>>>>>>>>> PreSetup an
> >>>>>>>>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >> volume
> >>>>> from
> >>>>>>>>>>> VMware, it
> >>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> >> volume
> >>>>> from
> >>>>>>>>> KVM,
> >>>>>>>>>>> it uses
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> >> thinking
> >>>>>>>>> here
> >>>>>>>>>>> was to
> >>>>>>>>>>>>>>> make a
> >>>>>>>>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
> >> ability
> >>>>>>>>> Edison
> >>>>>>>>>>> added
> >>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> >> more
> >>>>>>>>> specificy,
> >>>>>>>>>>> though:
> >>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> >> Mike
> >>>>>>>>> Tutkowski
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> >> shouldn't
> >>>>> know
> >>>>>>>>> or
> >>>>>>>>>>> care
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
> >> answer
> >>>>>>>>>>> questions
> >>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is
> that
> >> the
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
> >> the
> >>>>>>>>> hypervisor
> >>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> >>>>> constraint
> >>>>>>>>>>> and just
> >>>>>>>>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
> >> way
> >>>>> for
> >>>>>>>>> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> >>>>> perform
> >>>>>>>>> the
> >>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> >> John
> >>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM,
> Mike
> >>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
> >> be
> >>>>> able to
> >>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged
> by
> >>>>>>>>> CloudStack
> >>>>>>>>>>> (e.g. the
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> >> operator has
> >>>>>>>>> chosen
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> >> reasoning
> >>>>>>>>> make
> >>>>>>>>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> >> creating a
> >>>>> SAN
> >>>>>>>>>>> volume via
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> >> hypervisor)
> >>>>> would
> >>>>>>>>>>> work unless
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
> >> say,
> >>>>> an
> >>>>>>>>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that,
> independent
> >> of
> >>>>>>>>>>> CloudStack, an
> >>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
> >> and
> >>>>>>>>> exposes it
> >>>>>>>>>>>> through
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
> >> over
> >>>>> to
> >>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
> >> LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
> >> to my
> >>>>>>>>>>> changes, we
> >>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not
> have
> >> the
> >>>>>>>>>>> necessary
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> >> maybe
> >>>>> have
> >>>>>>>>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> >>>>> support
> >>>>>>>>> from
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> >> specifically
> >>>>> to
> >>>>>>>>> create
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> SR
> >>>>>>>>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> >> management
> >>>>> plugin
> >>>>>>>>>>> should
> >>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> >> determine
> >>>>>>>>> available
> >>>>>>>>>>> for
> >>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >>>>>>>>> interrogation of
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> >> device
> >>>>> itself
> >>>>>>>>>>> shouldn't
> >>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> >> simply be
> >>>>> able
> >>>>>>>>> to
> >>>>>>>>>>>> answer
> >>>>>>>>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
> >> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> >> SolidFire
> >>>>>>>>> driver
> >>>>>>>>>>> (which
> >>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this
> is
> >>>>> done, a
> >>>>>>>>> new
> >>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> >> Offering
> >>>>>>>>> that
> >>>>>>>>>>>> references
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> >> Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> >>>>> CloudStack
> >>>>>>>>>>> volume. This
> >>>>>>>>>>>>>>> leads
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> >>>>> CloudStack
> >>>>>>>>>>> volume to a
> >>>>>>>>>>>>>>> VM
> >>>>>>>>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> >> framework
> >>>>> calling
> >>>>>>>>> the
> >>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
> >> case, a
> >>>>>>>>> SAN).
> >>>>>>>>>>> The
> >>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> >> applicable
> >>>>> data
> >>>>>>>>>>> (like the
> >>>>>>>>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> >> invoked if
> >>>>> the
> >>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> >> attach,
> >>>>> the
> >>>>>>>>>>> volume
> >>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach
> disk)
> >> and
> >>>>> the
> >>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> >> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
> >> is run
> >>>>> and
> >>>>>>>>>>> detects
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
> >> the
> >>>>> form
> >>>>>>>>> of a
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
> >> PM,
> >>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> >> sync on
> >>>>>>>>>>> terminology,
> >>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> >> physical
> >>>>>>>>>>> allocation on
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
> >> try to
> >>>>>>>>> mean
> >>>>>>>>>>> 'SAN
> >>>>>>>>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
> >> end
> >>>>> user
> >>>>>>>>> can
> >>>>>>>>>>> make in
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
> >> PM,
> >>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> >> very
> >>>>> well
> >>>>>>>>> make
> >>>>>>>>>>> sense, but
> >>>>>>>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
> >> Edison
> >>>>> in on
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest
> the
> >>>>>>>>> approach I
> >>>>>>>>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at
> 12:42
> >> PM,
> >>>>>>>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> >> combining
> >>>>> two
> >>>>>>>>>>> distinct
> >>>>>>>>>>>>>>> concepts
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and
> storage
> >>>>> protocols.
> >>>>>>>>> In
> >>>>>>>>>>> both
> >>>>>>>>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> >>>>> system to
> >>>>>>>>>>>> create/delete
> >>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> >> requires
> >>>>> the
> >>>>>>>>>>> volume to
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the
> CloudStack
> >>>>> context.
> >>>>>>>>> To
> >>>>>>>>>>> ensure
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
> >> these
> >>>>>>>>>>> definitions,
> >>>>>>>>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> >>>>> correct?
> >>>>>>>>>>> Minimally,
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> >> storage
> >>>>> device
> >>>>>>>>> to
> >>>>>>>>>>> move bits
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >>>>>>>>> Optionally, a
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> >>>>> Therefore,
> >>>>>>>>>>> we can have
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a
> Xen
> >> SR,
> >>>>> and
> >>>>>>>>> we
> >>>>>>>>>>> can have a
> >>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> >> CloudStack is
> >>>>>>>>> capable
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> >>>>> CloudStack
> >>>>>>>>> may
> >>>>>>>>>>> be able
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> >> leave it
> >>>>>>>>>>> unmanaged by
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
> >> and
> >>>>> the
> >>>>>>>>>>> operator has
> >>>>>>>>>>>>>>> chosen
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> >> CloudStack).
> >>>>> Does
> >>>>>>>>> my
> >>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> >> above
> >>>>> are
> >>>>>>>>>>> reasonable, it
> >>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> >>>>> concerns
> >>>>>>>>> from
> >>>>>>>>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> >>>>> attached
> >>>>>>>>> driver
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
> >> and
> >>>>>>>>> establish
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> >> separate
> >>>>> that
> >>>>>>>>> is
> >>>>>>>>>>>>>>> associated
> >>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> >>>>> hypervisor
> >>>>>>>>>>> drivers can
> >>>>>>>>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> >>>>> encounter
> >>>>>>>>> a
> >>>>>>>>>>> device
> >>>>>>>>>>>> managed
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> >>>>> exposed
> >>>>>>>>> by
> >>>>>>>>>>> the driver to
> >>>>>>>>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
> >> make
> >>>>>>>>> sense,
> >>>>>>>>>>> then we can
> >>>>>>>>>>>>>>> sit
> >>>>>>>>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25
> PM,
> >>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> >>>>> between
> >>>>>>>>> the
> >>>>>>>>>>> current
> >>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> >> has to
> >>>>> go in
> >>>>>>>>> and
> >>>>>>>>>>> create
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> >> point in
> >>>>> time,
> >>>>>>>>>>> the
> >>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is
> pre-
> >>>>> allocated).
> >>>>>>>>>>> Future
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
> >> was
> >>>>> created
> >>>>>>>>>>> behind the
> >>>>>>>>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The
> admin
> >> has
> >>>>> to
> >>>>>>>>> go in
> >>>>>>>>>>> and
> >>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
> >> and
> >>>>> delete
> >>>>>>>>>>> volumes on
> >>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> >> enabled via
> >>>>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> >> volume
> >>>>> that
> >>>>>>>>> was
> >>>>>>>>>>> created,
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> >>>>> volume.
> >>>>>>>>> After
> >>>>>>>>>>> this is
> >>>>>>>>>>>>>>> done,
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> >> question is
> >>>>> called.
> >>>>>>>>>>> No
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because
> the
> >>>>> volume
> >>>>>>>>> was
> >>>>>>>>>>> just
> >>>>>>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> >> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 3:21
> >>>>> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
> >> so I
> >>>>> should
> >>>>>>>>>>> probably
> >>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
> >> that is
> >>>>>>>>> pre-
> >>>>>>>>>>> allocated
> >>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN),
> then
> >> a
> >>>>>>>>> hypervisor
> >>>>>>>>>>> data
> >>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> >> XenServer
> >>>>> SR),
> >>>>>>>>> then
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> >> CloudStack.
> >>>>> Disks
> >>>>>>>>> (VDI)
> >>>>>>>>>>> are later
> >>>>>>>>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> >> needed. In
> >>>>>>>>> these
> >>>>>>>>>>> cases, the
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >> structure is
> >>>>>>>>> already
> >>>>>>>>>>> in place
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> >>>>> structure
> >>>>>>>>> to
> >>>>>>>>>>> the VM in
> >>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> >> Storage
> >>>>> that is
> >>>>>>>>> not
> >>>>>>>>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
> >> this
> >>>>> could
> >>>>>>>>> be a
> >>>>>>>>>>> SAN
> >>>>>>>>>>>> (not
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> >> hypervisor
> >>>>> data
> >>>>>>>>>>> structure
> >>>>>>>>>>>>>>> must be
> >>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> >>>>> because
> >>>>>>>>> these
> >>>>>>>>>>> types of
> >>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a
> hypervisor
> >> data
> >>>>>>>>>>> structure by an
> >>>>>>>>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> >>>>> XenServer
> >>>>>>>>> for
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
> >> to the
> >>>>> VM
> >>>>>>>>> in
> >>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 3:13
> >>>>> PM,
> >>>>>>>>>>> John Burwell
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >>>>> implementation of
> >>>>>>>>> the
> >>>>>>>>>>> Dynamic
> >>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> >> which
> >>>>> why I
> >>>>>>>>> ask
> >>>>>>>>>>> about the
> >>>>>>>>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> >> what is
> >>>>> the
> >>>>>>>>>>> definition
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> 5:10
> >> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
> >> iSCSI
> >>>>> type
> >>>>>>>>> is
> >>>>>>>>>>>> uniquely
> >>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> >> Primary
> >>>>>>>>> Storage
> >>>>>>>>>>> that is
> >>>>>>>>>>>>>>> directly
> >>>>>>>>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> >> skip the
> >>>>>>>>> step of
> >>>>>>>>>>> going
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
> >> based
> >>>>> on
> >>>>>>>>> that
> >>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this
> is
> >> only
> >>>>>>>>>>> supported for
> >>>>>>>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> >> must
> >>>>> first go
> >>>>>>>>> to
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
> >> what
> >>>>> RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> >> at
> >>>>> 2:13
> >>>>>>>>> PM,
> >>>>>>>>>>> John
> >>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
> >> the
> >>>>> code,
> >>>>>>>>> what
> >>>>>>>>>>> is the
> >>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> >> Why
> >>>>> isn't
> >>>>>>>>> RBD
> >>>>>>>>>>> considered
> >>>>>>>>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> >> 3:46
> >>>>> PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> >>>>> storage is
> >>>>>>>>>>> defined in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static
> enum
> >>>>>>>>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
> >> //
> >>>>> local
> >>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> NetworkFilesystem(true),
> >>>>>>>>> //
> >>>>>>>>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> >>>>> shared
> >>>>>>>>> LUN,
> >>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
> >> e.g.,
> >>>>> ZFS
> >>>>>>>>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
> >> iso
> >>>>> image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> >>>>> XenServer
> >>>>>>>>> local
> >>>>>>>>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> >>>>> VMware
> >>>>>>>>> VMFS
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true),
> //
> >> for
> >>>>>>>>>>> XenServer, Storage
> >>>>>>>>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> >>>>> XenServer
> >>>>>>>>> local
> >>>>>>>>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> >>>>> dynamic,
> >>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> StoragePoolType(boolean
> >>>>>>>>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> >> shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> >>>>> isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013 at
> >>>>> 1:41
> >>>>>>>>> PM,
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> >> let's say
> >>>>>>>>> another
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> >> leverage its
> >>>>>>>>> Quality
> >>>>>>>>>>> of
> >>>>>>>>>>>> Service
> >>>>>>>>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >> storage, as
> >>>>> well.
> >>>>>>>>>>> They would
> >>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> >> made
> >>>>> the
> >>>>>>>>>>> necessary
> >>>>>>>>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
> >> plug-
> >>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>>> at
> >>>>>>>>> 1:39
> >>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> >> imply
> >>>>> the
> >>>>>>>>>>> XenServer code
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> >>>>> attach
> >>>>>>>>> logic is
> >>>>>>>>>>> now aware
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire
> is
> >> an
> >>>>>>>>>>> implementation of
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
> >> to 4.2
> >>>>>>>>> with
> >>>>>>>>>>> Edison's
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
> >> a
> >>>>> new
> >>>>>>>>>>> framework that
> >>>>>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> >>>>> dynamically.
> >>>>>>>>>>> However, when I
> >>>>>>>>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> >> realized
> >>>>> that
> >>>>>>>>> it
> >>>>>>>>>>> was not
> >>>>>>>>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> >> CloudStack
> >>>>> could
> >>>>>>>>> do
> >>>>>>>>>>> with these
> >>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> >>>>> recognize
> >>>>>>>>> this
> >>>>>>>>>>> new type of
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> data
> >>>>> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>>> at
> >>>>>>>>> 1:28
> >>>>>>>>>>> PM, John
> >>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
> >> odd
> >>>>> to me
> >>>>>>>>>>> that any
> >>>>>>>>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> or
> >> care
> >>>>>>>>> about
> >>>>>>>>>>> details. I
> >>>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
> >> of
> >>>>>>>>> operations
> >>>>>>>>>>> that can be
> >>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> implementations to
> >>>>>>>>>>> allocate
> >>>>>>>>>>>> space/create
> >>>>>>>>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
> >> this
> >>>>>>>>>>> dependency, we are
> >>>>>>>>>>>>>>> going
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
> >> will
> >>>>> make
> >>>>>>>>> the
> >>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance.
> Am
> >> I
> >>>>>>>>>>> understanding that the
> >>>>>>>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >>>>>>>>>>> CitrixResourceBase
> >>>>>>>>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> 3,
> >>>>> 2013 at
> >>>>>>>>> 3:12
> >>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve
> into
> >> this
> >>>>> in a
> >>>>>>>>> bit
> >>>>>>>>>>> more
> >>>>>>>>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
> >> and
> >>>>> aside
> >>>>>>>>> from
> >>>>>>>>>>> one setup
> >>>>>>>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first
> create
> >> a
> >>>>>>>>> volume on
> >>>>>>>>>>> the
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
> >> data
> >>>>>>>>> structure
> >>>>>>>>>>> to make
> >>>>>>>>>>>> use
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> on
> >>>>>>>>> XenServer
> >>>>>>>>>>> or a
> >>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> >>>>> storage
> >>>>>>>>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> Edison's
> >> new
> >>>>>>>>> storage
> >>>>>>>>>>> framework,
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
> >> easily
> >>>>>>>>> create
> >>>>>>>>>>> a 1:1
> >>>>>>>>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
> >> the
> >>>>> VM's
> >>>>>>>>> data
> >>>>>>>>>>> disk
> >>>>>>>>>>>>>>> (necessary
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can
> now
> >> write
> >>>>> a
> >>>>>>>>> plug-
> >>>>>>>>>>> in that is
> >>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> >>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> problem
> >> that
> >>>>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>> framework
> >>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >>>>>>>>> hypervisor-
> >>>>>>>>>>> specific data
> >>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
> >> the
> >>>>> case,
> >>>>>>>>> I've
> >>>>>>>>>>> been
> >>>>>>>>>>>>>>> enhancing
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
> >> and
> >>>>>>>>>>> submitted. I've got
> >>>>>>>>>>>>>>> ESX(i)
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we
> extend
> >> the
> >>>>> 4.2
> >>>>>>>>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> >> help a
> >>>>> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun
> >> 3,
> >>>>> 2013 at
> >>>>>>>>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> storage
> >>>>> plug-in -
> >>>>>>>>> by
> >>>>>>>>>>> itself - is
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue
> is
> >> with
> >>>>> the
> >>>>>>>>>>> volume-attach
> >>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> framework
> >> calls
> >>>>> into
> >>>>>>>>> the
> >>>>>>>>>>> plug-in to
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
> >> time
> >>>>>>>>> comes to
> >>>>>>>>>>> attach the
> >>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
> >> smart
> >>>>>>>>> enough
> >>>>>>>>>>> to recognize
> >>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where
> the
> >>>>> volume
> >>>>>>>>> has
> >>>>>>>>>>> just been
> >>>>>>>>>>>>>>> created)
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> >> (for
> >>>>>>>>>>> XenServer) or a
> >>>>>>>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> that
> >> was
> >>>>> just
> >>>>>>>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> >>>>> spending
> >>>>>>>>> most
> >>>>>>>>>>> of my time
> >>>>>>>>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the
> agent
> >>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> >> clear it
> >>>>> up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun 3,
> >>>>> 2013
> >>>>>>>>> at
> >>>>>>>>>>> 12:48 PM,
> >>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> >> explain
> >>>>>>>>> why
> >>>>>>>>>>> the the storage
> >>>>>>>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> >> 2013,
> >>>>> at
> >>>>>>>>> 1:21
> >>>>>>>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> >>>>> ultimately I
> >>>>>>>>>>> would like to
> >>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> supports.
> >> I
> >>>>> think
> >>>>>>>>> I'm
> >>>>>>>>>>> just out of
> >>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> >> now this
> >>>>>>>>> plug-in
> >>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> >> to 4.2
> >>>>>>>>> feature
> >>>>>>>>>>> freeze, I
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> just
> >> to
> >>>>> be
> >>>>>>>>> clear,
> >>>>>>>>>>> this is
> >>>>>>>>>>>> all
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>> Compute
> >>>>>>>>>>> Offerings post
> >>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> >> Jun 3,
> >>>>>>>>> 2013
> >>>>>>>>>>> at 11:14 AM,
> >>>>>>>>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is
> there
> >> any
> >>>>> plan
> >>>>>>>>> on
> >>>>>>>>>>> supporting
> >>>>>>>>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> >> Original
> >>>>>>>>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
> >> "Mike
> >>>>>>>>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >>>>>>>>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> >>>>> Monday,
> >>>>>>>>> June
> >>>>>>>>>>> 3, 2013
> >>>>>>>>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Subject:
> >> Re:
> >>>>>>>>> [MERGE]
> >>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
> >> on
> >>>>>>>>> merging
> >>>>>>>>>>> Wei's feature
> >>>>>>>>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
> >> feature
> >>>>> is
> >>>>>>>>> for
> >>>>>>>>>>> KVM only,
> >>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> in
> >> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Mon,
> >> Jun
> >>>>> 3,
> >>>>>>>>> 2013
> >>>>>>>>>>> at 8:55 AM,
> >>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> >> the
> >>>>> billing,
> >>>>>>>>> as
> >>>>>>>>>>> no one works
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> billing
> >>>>> by
> >>>>>>>>>>> themselves. They
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> disk_offering of
> >>>>>>>>> a
> >>>>>>>>>>> VMs and
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
> >> user
> >>>>> the
> >>>>>>>>>>> exact
> >>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>> limitation
> >>>>>>>>> for
> >>>>>>>>>>> nics as
> >>>>>>>>>>>> well.
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> >> part of
> >>>>> I/O
> >>>>>>>>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
> >> my
> >>>>>>>>>>> misunstand the second
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Agree
> >> with
> >>>>>>>>> what
> >>>>>>>>>>> you said about
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 2013/6/3
> >>>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >> Jun 3,
> >>>>> 2013,
> >>>>>>>>> at
> >>>>>>>>>>> 2:13 AM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> >> John,
> >>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> hope
> >>>>> Mike's
> >>>>>>>>>>> aswer helps you.
> >>>>>>>>>>>>>>> I am
> >>>>>>>>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1)
> I
> >>>>> think
> >>>>>>>>> billing
> >>>>>>>>>>> should
> >>>>>>>>>>>>>>> depend
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> limitation.
> >>>>>>>>> Please
> >>>>>>>>>>> review
> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> get
> >> the
> >>>>> IO
> >>>>>>>>>>> statistics
> >>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> virtual
> >>>>>>>>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Going by
> >>>>> the
> >>>>>>>>> AWS
> >>>>>>>>>>> model,
> >>>>>>>>>>>> customers
> >>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> provisioned
> >>>>>>>>> IOPS,
> >>>>>>>>>>> as well as,
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://aws.amazon.com/ebs/).
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> option to
> >>>>>>>>> employ
> >>>>>>>>>>> similar cost
> >>>>>>>>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> billing
> >>>>> model
> >>>>>>>>> in
> >>>>>>>>>>> the current
> >>>>>>>>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> >> Do
> >>>>> you
> >>>>>>>>> mean
> >>>>>>>>>>> IOPS runtime
> >>>>>>>>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> limitation for
> >>>>>>>>> a
> >>>>>>>>>>> running
> >>>>>>>>>>>> virtual
> >>>>>>>>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> CloudStack
> >>>>>>>>> does
> >>>>>>>>>>> not support
> >>>>>>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> (computer
> >>>>>>>>>>> offering or disk
> >>>>>>>>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> meant at
> >>>>> the
> >>>>>>>>>>> Java interface
> >>>>>>>>>>>>>>> level.
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >> more
> >>>>>>>>>>> generalize allocation
> >>>>>>>>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> describe
> >>>>> the
> >>>>>>>>>>> service guarantees
> >>>>>>>>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> >> It is a
> >>>>>>>>> good
> >>>>>>>>>>> question.
> >>>>>>>>>>>>>>> Maybe it
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> disk_io_throttling as Mike
> >>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> type, I
> >>>>> think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> >>>>> expand
> >>>>>>>>> on
> >>>>>>>>>>> my thoughts
> >>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> touch
> >>>>> points
> >>>>>>>>>>> between these two
> >>>>>>>>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >> need
> >>>>> to be
> >>>>>>>>>>> merged before
> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> coordination
> >>>>>>>>>>> between the
> >>>>>>>>>>>> branches
> >>>>>>>>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> disk_io_throttling)
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 2013/6/3
> >>>>>>>>> John
> >>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> The
> >>>>> things I
> >>>>>>>>>>> want to
> >>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
> >> Is
> >>>>> there
> >>>>>>>>>>> value in
> >>>>>>>>>>>> capturing
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> data
> >>>>> model
> >>>>>>>>> (e.g.
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> >>>>> Should
> >>>>>>>>> there
> >>>>>>>>>>> be a common
> >>>>>>>>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> provisioning
> >>>>>>>>> at
> >>>>>>>>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
> >> How
> >>>>> are
> >>>>>>>>>>> conflicting
> >>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> hypervisor
> >>>>>>>>> and
> >>>>>>>>>>> storage device
> >>>>>>>>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> lead
> >>>>> to
> >>>>>>>>>>> believe (and
> >>>>>>>>>>>> billed)
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> storage
> >>>>>>>>> device
> >>>>>>>>>>> has been
> >>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> consistent
> >>>>>>>>>>> configuration
> >>>>>>>>>>>>>>> between a
> >>>>>>>>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> time,
> >>>>> but a
> >>>>>>>>>>> later
> >>>>>>>>>>>> modification
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> Jun
> >>>>> 2,
> >>>>>>>>> 2013,
> >>>>>>>>>>> at 8:38 PM,
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi
> >> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>> believe
> >>>>>>>>> Wei's
> >>>>>>>>>>> feature deals
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>>>>>> hypervisor
> >>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> My
> >>>>> feature
> >>>>>>>>> is
> >>>>>>>>>>> focused on
> >>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> hope
> >>>>> that
> >>>>>>>>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> Sun,
> >>>>> Jun
> >>>>>>>>> 2,
> >>>>>>>>>>> 2013 at 6:35
> >>>>>>>>>>>> PM,
> >>>>>>>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> My
> >>>>>>>>> opinion is
> >>>>>>>>>>> that no
> >>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> issues
> >>>>>>>>> have
> >>>>>>>>>>> been resolved
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> total
> >>>>> Ops
> >>>>>>>>> vs
> >>>>>>>>>>> discrete
> >>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> >>>>> reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>> I
> >>>>>>>>> don't
> >>>>>>>>>>> think this
> >>>>>>>>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Also,
> >>>>> how
> >>>>>>>>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> also
> >>>>>>>>> involves
> >>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> scenario
> >>>>>>>>>>> where provisioned
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>> Jun 1,
> >>>>>>>>> 2013,
> >>>>>>>>>>> at 6:47 AM,
> >>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Sure. I
> >>>>> will
> >>>>>>>>>>> change it next
> >>>>>>>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> 2013/6/1
> >>>>>>>>>>> Wido den Hollander
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Exactly. I
> >>>>>>>>> have
> >>>>>>>>>>> pushed the
> >>>>>>>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> If
> >>>>> anyone
> >>>>>>>>>>> object thems for
> >>>>>>>>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> For
> >>>>> the
> >>>>>>>>> sake
> >>>>>>>>>>> of clarity I
> >>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> the
> >>>>> total
> >>>>>>>>> IOps
> >>>>>>>>>>> to R/W IOps
> >>>>>>>>>>>>>>> asap
> >>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> only
> >>>>> total
> >>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> You
> >>>>> laid
> >>>>>>>>> the
> >>>>>>>>>>> groundwork for
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> however
> >>>>>>>>>>> prevent that we
> >>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> 2013/5/31
> >>>>>>>>>>> Wido den
> >>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> +1 --
> >>>>> this
> >>>>>>>>>>> enhancement must
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> don't
> >>>>> see
> >>>>>>>>> how
> >>>>>>>>>>> it could be
> >>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> split
> >>>>> total
> >>>>>>>>> IOPS
> >>>>>>>>>>> into read
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> total
> >>>>>>>>>>> unless/until we
> >>>>>>>>>>>> decided
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> What
> >>>>> Wei
> >>>>>>>>>>> meant was merging
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 4.2
> >>>>> branch
> >>>>>>>>>>> and add Read /
> >>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> will be
> >>>>>>>>>>> released with Read
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> This is
> >>>>> to
> >>>>>>>>>>> make the May 31st
> >>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (see
> >>>>> other
> >>>>>>>>>>> threads) then it
> >>>>>>>>>>>>>>> won't
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >> also
> >>>>>>>>>>> completely agree that
> >>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> disk
> >>>>> I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >> John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>> May 31,
> >>>>>>>>>>> 2013, at 9:51 AM,
> >>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi
> >>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Thanks.
> >>>>>>>>> Good
> >>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>>> thought
> >>>>>>>>>>> about at the
> >>>>>>>>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> difference
> >>>>>>>>> of
> >>>>>>>>>>> read and write
> >>>>>>>>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> care
> >>>>> the
> >>>>>>>>>>> difference of sent
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> That
> >>>>>>>>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Disk
> >>>>> I/O is
> >>>>>>>>>>> much more
> >>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> Implementing
> >>>>>>>>>>> it will be some
> >>>>>>>>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> implemented
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> few
> >>>>> days.
> >>>>>>>>> For
> >>>>>>>>>>> the deadline
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> that ,
> >>>>> if
> >>>>>>>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> It
> >>>>> think it's
> >>>>>>>>> a
> >>>>>>>>>>> feature we
> >>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> window
> >>>>>>>>> we
> >>>>>>>>>>> have to make sure
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> it
> >> in
> >>>>> 4.3,
> >>>>>>>>> that
> >>>>>>>>>>> will confuse
> >>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> 2013/5/31
> >>>>>>>>>>> Wido den
> >>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> On
> >>>>>>>>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> I
> >>>>> would
> >>>>>>>>> like to
> >>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> If
> >>>>> nobody
> >>>>>>>>>>> object, I will
> >>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> The
> >>>>>>>>> purpose
> >>>>>>>>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Virtual
> >>>>>>>>>>> machines are running
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> share
> >>>>>>>>> strage).
> >>>>>>>>>>> Because of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> iops),
> >>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> one
> >>>>> VM
> >>>>>>>>> has
> >>>>>>>>>>> large disk
> >>>>>>>>>>>>>>> operation,
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> other
> >>>>> VMs
> >>>>>>>>>>> running on the
> >>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> It
> >> is
> >>>>>>>>> neccesary
> >>>>>>>>>>> to set the
> >>>>>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Looking at
> >>>>>>>>> the
> >>>>>>>>>>> code I see
> >>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Qemu
> >>>>> and
> >>>>>>>>>>> libvirt support
> >>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps
> >>>>>>>>> which
> >>>>>>>>>>> could benefit a
> >>>>>>>>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> It's
> >>>>> also
> >>>>>>>>>>> strange, in the
> >>>>>>>>>>>>>>> polling
> >>>>>>>>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps,
> >>>>> but
> >>>>>>>>> on
> >>>>>>>>>>> the throttling
> >>>>>>>>>>>>>>> side
> >>>>>>>>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Write
> >>>>> IOps
> >>>>>>>>> are
> >>>>>>>>>>> usually much
> >>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> like a
> >>>>> valid
> >>>>>>>>>>> use-case where
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> IOps
> >>>>> vs
> >>>>>>>>> Read
> >>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Since
> >>>>> this
> >>>>>>>>> only
> >>>>>>>>>>> supports KVM
> >>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> value
> >>>>> to at
> >>>>>>>>>>> least have the
> >>>>>>>>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> this
> >>>>> later
> >>>>>>>>>>> would be a lot of
> >>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> If
> >> a
> >>>>>>>>> hypervisor
> >>>>>>>>>>> doesn't
> >>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> write
> >>>>> you
> >>>>>>>>> can
> >>>>>>>>>>> always sum
> >>>>>>>>>>>> both
> >>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Can
> >>>>> you
> >>>>>>>>>>> explain why you
> >>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> The
> >>>>>>>>> feature
> >>>>>>>>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (1) set
> >>>>> the
> >>>>>>>>>>> maximum rate of
> >>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (2)
> >>>>> change
> >>>>>>>>> the
> >>>>>>>>>>> maximum rate
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (3)
> >>>>> limit
> >>>>>>>>> the
> >>>>>>>>>>> disk rate
> >>>>>>>>>>>> (total
> >>>>>>>>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> JIRA
> >>>>> ticket:
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>> 1192
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> FS (I
> >>>>> will
> >>>>>>>>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Merge
> >>>>>>>>> check
> >>>>>>>>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *
> >> Did
> >>>>> you
> >>>>>>>>>>> check the branch's
> >>>>>>>>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *
> >> Are
> >>>>>>>>> there
> >>>>>>>>>>> new dependencies
> >>>>>>>>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *
> >>>>> What
> >>>>>>>>>>> automated testing
> >>>>>>>>>>>> (unit
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Unit
> >>>>> tests
> >>>>>>>>> are
> >>>>>>>>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *
> >>>>> What
> >>>>>>>>>>> testing has been done
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (1) set
> >>>>> the
> >>>>>>>>>>> bytes rate and
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (2)
> >>>>> VM
> >>>>>>>>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> deploy,
> >>>>>>>>> stop,
> >>>>>>>>>>> start, reboot,
> >>>>>>>>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> (3)
> >>>>>>>>> Volume
> >>>>>>>>>>> operations,
> >>>>>>>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Attach,
> >>>>>>>>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> To
> >>>>> review
> >>>>>>>>> the
> >>>>>>>>>>> code, you can
> >>>>>>>>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Best
> >>>>>>>>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>> <https://%0b/>> > >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>>>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> [2]
> >>>>>>>>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> >> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>>>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> -
> >>>>> VM
> >>>>>>>>> Disk
> >>>>>>>>>>> I/O
> >>>>>>>>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Mike
> >>>>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *Senior
> >>>>>>>>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>>>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> Advancing
> >>>>>>>>> the
> >>>>>>>>>>> way the world
> >>>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >>>>>>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Advancing
> >>>>> the
> >>>>>>>>> way
> >>>>>>>>>>> the world uses
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Advancing the
> >>>>>>>>> way
> >>>>>>>>>>> the world uses
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >> the
> >>>>> way
> >>>>>>>>> the
> >>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>> CloudStack
> >>>>>>>>>>> Developer,
> >>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >> the
> >>>>> way
> >>>>>>>>> the
> >>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>>>> Developer, SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>>> the
> >>>>>>>>>>> world uses the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>>> the
> >>>>>>>>> world
> >>>>>>>>>>> uses the
> >>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>>>>>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way the
> >>>>>>>>> world
> >>>>>>>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >> the
> >>>>> world
> >>>>>>>>> uses
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>> Developer,
> >>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>>> world
> >>>>>>>>> uses
> >>>>>>>>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>>> uses
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>>> uses
> >>>>>>>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >> uses
> >>>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>>> the
> >>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> the
> >>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *(tm)*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *(tm)*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *(tm)*****
> >>>>
> >>>>
> >>>>
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*****
> >>>>
> >>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>
> >>>> e: mike.tutkowski@solidfire.com****
> >>>>
> >>>> o: 303.746.7302****
> >>>>
> >>>> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*****
> >>>>
> >>>>
> >>>>
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*****
> >>>>
> >>>> *Senior CloudStack Developer, SolidFire Inc.*****
> >>>>
> >>>> e: mike.tutkowski@solidfire.com****
> >>>>
> >>>> o: 303.746.7302****
> >>>>
> >>>> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*****
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *(tm)*
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Edison,
TL;DR Always give operators/system administrators the ability to manage things themselves, if they like.
As more manageable drivers emerge, I think it will be import to allow operators to opt-out of CloudStack device management. Therefore, a driver declares whether or not it can support device management (many, in fact most, won't). For those drivers that do support management, allow operators to opt-in/out of CloudStack management on a per device basis. To support this notion, we need a flag on the storage_pool table and associated classes to indicate whether or not the operator wants CloudStack to manage the device.
Does this explanation make more sense?
Thanks,
-John
On Jun 10, 2013, at 2:58 PM, Edison Su <Ed...@citrix.com> wrote:
> I don't understand why we need to add a "managed" in the UI and in "storage_pool" table?
> AFAIK, what you guys trying to do is to enable attachVolume/DettachVolume if the volume is created on Solidfire, right?
> If the volume is created on Solidfire already, by createAsync, then why we can't just send out all the information about the volume(it's volumeTO which will be returned by each storage driver) to hypervisor, then at the hypervisor side, check the volumeTO, if the volumeTO is managed, then doing whatever necessary operations(create SR, create VDI etc), then attach the VDI to volume.
>
>
>> -----Original Message-----
>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> Sent: Monday, June 10, 2013 11:10 AM
>> To: John Burwell
>> Cc: Edison Su; dev@cloudstack.apache.org
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>> I believe we're on the same page, John.
>>
>> I would add a new parameter to the createStoragePool API command (let's
>> call it "managed" and it can be true or false). If you pass in "true", but
>> the driver does not support being managed by CloudStack, an exception is
>> thrown by the driver. If you pass in "false", but the driver does not
>> support being managed outside of CloudStack, an exception is thrown by the
>> driver. It is possible one could write a driver that can be managed both
>> inside or outside of CloudStack (the SolidFire driver requires being
>> managed only by CloudStack).
>>
>> This managed value would end up in the storage_pool table in a new column
>> ("managed").
>>
>> An example usage: After the storage framework calls on the driver's
>> createAsync method, it will check to see if the driver is managed. If the
>> driver is managed, the storage framework will invoke the proper (new)
>> command of the hypervisor in use to set up, say, the SR for the SAN volume.
>>
>> There may be other cases when the isManaged() method is invoked on the
>> driver and - if true is returned - the storage framework sends a particular
>> command to the hypervisor in use.
>>
>> ********************
>>
>> As far as reworking the default storage plug-in (which sends commands to
>> hypervisors) for 4.2, I would check with Edison on his schedule.
>>
>> Thanks!
>>
>>
>> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
>> wrote:
>>
>>> Mike,
>>>
>>> I want to make sure we are on the same page regarding isManaged. I see
>>> the addition of the following operations:
>>>
>>> * DataStoreDriver#supportsManagement() : boolean -- indicates whether
>> or
>>> not the driver support/implements the management functions
>>> * DataStore#isManaged() : boolean -- indicates whether or not a device is
>>> managed by CloudStack
>>>
>>> Only DataStores associated with DataStoreDrivers that support
>> management
>>> would be able to have the management flag enabled. This behavior allows
>>> operators to declare their intention to allow CloudStack to manage a
>>> device. This behavior would need to be exposed as flag on the HTTP API.
>>> Additionally, this flag would only be settable when a device is
>>> defined/created in the system (i.e. once a device is managed by CloudStack,
>>> you can't unmanaged it without dissociating it from CloudStack). In the
>>> future, we may consider relaxing that rule, but, in the short term, it
>>> greatly decreases the alternative flows.
>>>
>>> In terms of removing the dependencies from Storage to Hypervisor, I
>> would
>>> really like to see this accomplished in 4.2. It will be much more
>>> difficult to address this issue post 4.2 because we will have more plugins
>>> to address. I also see this issue as a critical architectural issues that
>>> for the management server. I recommend we identify the dependencies
>> that
>>> need to be inverted, and see if we can divide the work across a few more
>>> resources.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
>> <mi...@solidfire.com>
>>> wrote:
>>>
>>> Sounds good, Edison :)
>>>
>>> I can implement John's isManaged() logic and resubmit my code for review.
>>>
>>> Perhaps in 4.3 I could address the direct-attach scenario. If it requires
>>> some refactoring of the storage framework, then I could do that.
>>>
>>> Also, the default storage plug-in does have hypervisor logic in it in 4.2.
>>> I recommend we leave this as is for 4.2 and it can be refactored in 4.3
>>> with the above work I mentioned.
>>>
>>> Are you OK with that, guys?
>>>
>>>
>>> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com> wrote:
>>>
>>>> Ok, I am Ok with the way you are talking about in 4.2.****
>>>>
>>>> ** **
>>>>
>>>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>> *Sent:* Sunday, June 09, 2013 8:59 PM
>>>> *To:* dev@cloudstack.apache.org
>>>> *Cc:* Edison Su; John Burwell
>>>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>>>>
>>>> ** **
>>>>
>>>> Another thing we might want to consider for 4.2 is that the way the code
>>>> is currently implemented in master, I don't think it's even possible to
>>>> write a plug-in that connects storage directly to a VM because the
>>>> CloudStack storage framework will call the "attach" method of the
>>>> hypervisor in use and this method will fail because it assumes there is
>>>> (talking Xen here) an SR existent and in this situation there isn't one.*
>>>> ***
>>>>
>>>> ** **
>>>>
>>>> My thinking is that we should ignore the direct-attach-of-volume-to-VM
>>>> use case for 4.2. I think it will require too much of a rewrite at this
>>>> point in development (fewer than three weeks left until code
>> freeze).****
>>>>
>>>> ** **
>>>>
>>>> If the plug-in implements an isManaged() method (as John suggested),
>> the
>>>> storage framework can invoke the necessary commands on the
>> hypervisor in
>>>> use.****
>>>>
>>>> ** **
>>>>
>>>> For example, the createAsync method is called. The storage plug-in
>>>> creates a SAN volume and updates the CS DB. The storage framework
>> then asks
>>>> the driver if it's managed. If it is, the storage framework executes a
>>>> command against the hypervisor in use to have it create, say, an SR (with
>> a
>>>> single VDI that takes up the entire SR). Now, when the storage
>> framework
>>>> executes the "attach" command against the hypervisor, this command will
>>>> work, as well (because that logic assumes the prior existence of an SR and
>>>> it will, in fact, be present).****
>>>>
>>>> ** **
>>>>
>>>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:****
>>>>
>>>> Hi Edison and John,****
>>>>
>>>> ** **
>>>>
>>>> I wanted to point out something I thought was relevant to this
>>>> conversation.****
>>>>
>>>> ** **
>>>>
>>>> My plug-in supports both XenServer and ESX(i).****
>>>>
>>>> ** **
>>>>
>>>> These two hypervisors handle their respective hypervisor data structures
>>>> (XenServer SR or ESX(i) datastore) quite differently.****
>>>>
>>>> ** **
>>>>
>>>> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
>>>> when you issue a remove-SR command to XenServer.****
>>>>
>>>> ** **
>>>>
>>>> When you later want to access the contents of that data, it is still
>>>> safely stored on the LUN and can be accessed from XenServer (from the
>> same
>>>> or different cluster (what XenServer calls a resource pool)) by creating a
>>>> new SR that is based on the same IQN and LUN.****
>>>>
>>>> ** **
>>>>
>>>> XenServer SRs exist at the cluster level.****
>>>>
>>>> ** **
>>>>
>>>> In ESX(i), the datastore that resides on the SAN LUN is actually
>>>> destroyed when you issue a remove-datastore command to ESX(i). In
>> other
>>>> words, the contents of the SAN LUN are destroyed.****
>>>>
>>>> ** **
>>>>
>>>> ESX(i) datastores exist at the datacenter level.****
>>>>
>>>> ** **
>>>>
>>>> The XenServer behavior (not destroying the contents of the LUN when
>> the
>>>> SR is removed) is desirable; the ESX(i) behavior (destroying the contents
>>>> of the LUN when the datastore is removed) is not desirable.****
>>>>
>>>> ** **
>>>>
>>>> I can get around the ESX(i) behavior by removing the referenced IQN
>> from
>>>> each host in the cluster when the CS storage framework notifies my plug-
>> in
>>>> of a detach-volume event. When the CS volume is later attached again
>>>> (either to the same or a different ESX(i) cluster), I can have each host in
>>>> the relevant ESX(i) cluster reference the IQN and the datastore will be
>>>> back again.****
>>>>
>>>> ** **
>>>>
>>>> This sequence of events is critical for my plug-in to support both
>>>> environments:****
>>>>
>>>> *XenServer flow (events for Driver class):*****
>>>>
>>>> createAsync (called right before first attach) = creates SAN volume****
>>>>
>>>> preAttachVolume = creates SR and creates VDI or just introduces SR****
>>>>
>>>> (CS volume gets attached and later detached)****
>>>>
>>>> postDetachVolume = removes SR (this does not delete data on SAN
>> volume)**
>>>> **
>>>>
>>>> deleteAsync (can be called when volume is not attached) = deletes the
>> SAN
>>>> volume****
>>>>
>>>> *VMware flow (events for Driver class):*****
>>>>
>>>> createAsync (called right before first attach) = creates SAN volume,
>>>> creates datastore, and creates VMDK file****
>>>>
>>>> preAttachVolume = adds iSCSI target to each host in cluster****
>>>>
>>>> (CS volume gets attached and later detached)****
>>>>
>>>> postDetachVolume = removes iSCSI target from each host in cluster****
>>>>
>>>> deleteAsync (can be called when volume is not attached) = deletes
>>>> datastore and deletes the SAN volume****
>>>>
>>>> ************************
>>>>
>>>> So, the way I have it written up here, the driver is talking to the
>>>> hypervisor. As John has pointed out, this may not be ideal.****
>>>>
>>>> ** **
>>>>
>>>> Whatever we decide on, it needs to be able to support this kind of flow.
>>>> :)****
>>>>
>>>> ** **
>>>>
>>>> We could go John's route, where the driver says if it's managed or not.**
>>>> **
>>>>
>>>> ** **
>>>>
>>>> If it is, then the storage framework could send the necessary messages to
>>>> the hypervisor in question at the right times (just migrate the hypervisor
>>>> logic above from the driver to the right parts of the storage framework).
>>>> ****
>>>>
>>>> ** **
>>>>
>>>> This is fine, except that it does not resolve Edison's use case of
>>>> desiring to have the storage LUN potentially connected directly to a VM.*
>>>> ***
>>>>
>>>> ** **
>>>>
>>>> I'm not sure we should design around that use case, though. Is it common
>>>> enough? Has anyone asked for it? Is it even desirable? How are usage
>>>> statistics gathered for such volumes? It's nice to offer the flexibility,
>>>> but is it a good idea in this case?****
>>>>
>>>> ** **
>>>>
>>>> Thanks!****
>>>>
>>>> ** **
>>>>
>>>> ** **
>>>>
>>>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:**
>>>> **
>>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>> Sent: Friday, June 07, 2013 4:08 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>
>>>>> Hi John,
>>>>>
>>>>> Yes, you are correct that Xen must layer an SR on top of the iSCSI
>>>> volume
>>>>> to use it. Same story for VMware: ESX(i) must layer a datastore on top
>>>> of
>>>>> the iSCSI volume to use it. I look at it like they are layering a
>>>> clustered
>>>>> file system on the SAN volume so the hypervisors can share access to
>> the
>>>>> contents of the volume.
>>>>>
>>>>> In 4.1, by the time the hypervisor-attach-volume logic is called, the
>>>> SR or
>>>>> datastore has already been created (usually manually by an admin).
>>>>>
>>>>> This pre-setup of the, say, SR is, of course, not acceptable in an
>>>>> environment where each CloudStack volume that a user creates is
>> mapped
>>>>> to a
>>>>> single SAN volume (via the SR).
>>>>>
>>>>> The question comes down to who should allocate the SR dynamically.
>>>>>
>>>>> We could have the storage framework ask the storage plug-in if it is
>>>>> managed. If it is, then the storage framework could send a message to
>>>> the
>>>>> hypervisor in question to create (let's talk Xen here) the SR ahead of
>>>>> time. Then, when the storage framework next sends the attach-volume
>>>>> command
>>>>> to the hypervisor, it should work without changes to that attach-volume
>>>>> logic (because - from the point of view of the attach logic - the SR is
>>>>> already existent, as expected).
>>>>>
>>>>> Now, as Edison has pointed out, this limits the power of the storage
>>>>> plug-in. A storage plug-in in this model cannot directly attach storage
>>>> to
>>>>> a VM (it must go through the hypervisor). Perhaps that is OK. We need
>> to
>>>>> make a call on that.
>>>>
>>>>
>>>> Yes, I think we need to address this usage case, and won't limit the
>>>> power of the storage plug-in.
>>>> John, what's your idea to fit this usage case into your abstraction?
>>>>
>>>>
>>>>>
>>>>> If we want to give more power to the storage vendor, we would have to
>>>>> have
>>>>> the storage framework call into the storage plug-in to send the
>>>> appropriate
>>>>> attach commands to the hypervisor.
>>>>>
>>>>> Let's discuss and come to a consensus (or at least agree on a path) as
>>>> soon
>>>>> as we can. :)
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> My understanding is the Xen expects an SR structure on all ISCSI
>>>> devices
>>>>>> -- at least that is how I read the code in your patch. Is my
>>>> understanding
>>>>>> correct? If so, the Xen plugin should be able to query the storage
>>>> device
>>>>>> to determine the presence of the SR structure and create it if it
>>>> does not
>>>>>> exist. Am I missing something in the implementation that makes that
>>>> type
>>>>>> of implementation impossible?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yeah, if a storage vendor wanted to attach a volume directly to a
>>>> VM in
>>>>>> 4.2
>>>>>>> today, it would probably fail because the attach-volume logic
>>>> assumes
>>>>> the
>>>>>>> existence of the necessary hypervisor data structure (ex. SR on
>>>> Xen).
>>>>>>>
>>>>>>> If we wanted to enable such an attach, we could do it the way
>> Edison
>>>>>>> suggests.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>> Sent: Friday, June 07, 2013 2:37 PM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>
>>>>>>>>> As we only have three weeks until feature freeze, we should
>> come
>>>> to
>>>>> a
>>>>>>>>> consensus on this design point as soon as possible.
>>>>>>>>>
>>>>>>>>> Right now, if the storage framework asks my driver it is is
>>>> managed, it
>>>>>>>>> will say 'yes.' This means the framework will tell the driver to
>>>>>> perform
>>>>>>>>> its management activities. This then means the driver will call
>>>> into
>>>>>> the
>>>>>>>>> host (it doesn't know which hypervisor, by the way) to perform
>> the
>>>>>>>> activity
>>>>>>>>> of, say, creating an SR on XenServer or a datastore on ESX.
>>>>>>>>>
>>>>>>>>> The driver doesn't know which hypervisor it's talking to, it just
>>>>>> sends a
>>>>>>>>> message to the host to perform the necessary pre-attach work.
>>>>>>>>
>>>>>>>> Could we just expose a method like
>> "attachVolume/dettachVolume" on
>>>>> the
>>>>>>>> PrimaryDataStoreDriver driver?
>>>>>>>> For most of cases, the implementation of each driver would just
>>>> send
>>>>>>>> attachvolumecommand/dettachvolumecmd to hypervisor(we can
>> put
>>>>> the
>>>>>>>> implementation in a base class, so that can be shared by all of
>>>> this
>>>>>>>> drivers), and in each hypervisor resource code, which may just call
>>>>>>>> hypervisor's api to attach the volume to VM, while for certain
>>>> storage,
>>>>>>>> like SolidFire, may need to create a SR first, and create volume
>>>> on it,
>>>>>>>> then call hypervisor's API to attach volume to VM.
>>>>>>>> While for some other storage vendor, may want to bypass
>> hypervisor
>>>>>> during
>>>>>>>> attaching volume, so inside driver's attachvolume implementation,
>>>> the
>>>>>>>> driver can do some magic, such as, directly talk to an agent
>>>> inside VM
>>>>>>>> instance, then create a disk inside VM.
>>>>>>>>
>>>>>>>> How do you guys think?
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>>>>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>
>>>>>>>>>>> Hi John,
>>>>>>>>>>>
>>>>>>>>>>> How's about this:
>>>>>>>>>>>
>>>>>>>>>>> The driver can implement an isManaged() method. The
>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>> can
>>>>>>>>>>> call into the driver to see if its managed. If it is, the
>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>> (which is responsible for calling into the hypervisor to attach
>>>> the
>>>>>>>> disk)
>>>>>>>>>>> can call into the hypervisor to create the necessary hypervisor
>>>> data
>>>>>>>>>>> structure (ex. for XenServer, a storage repository).
>>>>>>>>>>
>>>>>>>>>> The problem here is that storage vendor may work differently
>> with
>>>>>>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while
>>>> maybe
>>>>>>>> other
>>>>>>>>>> vendor wants to totally bypass hypervisor, and assign the LUN
>>>>> directly
>>>>>>>> to
>>>>>>>>>> VM instance, see the discuss(
>>>>>>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>>>>>>>>>
>>>>>
>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>>>>>>>>> %3E
>>>>>>>>>> ).
>>>>>>>>>> So I would let storage provider to implement attach disk to VM,
>>>>>>>> instead of
>>>>>>>>>> implemented by cloudstack itself.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> If that's what you're going for, that works for me. By the way,
>>>>>>>> Edison's
>>>>>>>>>>> default storage plug-in (which handles the default storage
>>>> behavior
>>>>>>>> in
>>>>>>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that
>>>> talks to
>>>>>>>>>>> hypervisors. You might want to contact him and inform him of
>>>> your
>>>>>>>>>> concerns
>>>>>>>>>>> or that logic (as is) will make it to production.
>>>>>>>>>>>
>>>>>>>>>>> Please let me know if what I wrote in above (for my solution)
>>>> is OK
>>>>>>>> with
>>>>>>>>>>> you. :)
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
>>>> jburwell@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Please see my responses in-line below.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I still have a bit more testing I'd like to do before I build
>>>> up
>>>>>>>> a
>>>>>>>>>> patch
>>>>>>>>>>>>> file, but this is the gist of what I've done:
>>>>>>>>>>>>>
>>>>>>>>>>>>> * During a volume-attach operation, after
>> VolumeManagerImpl
>>>>> tells
>>>>>>>>>>>>> VolumeServiceImpl to have the driver create a volume, I
>> have
>>>>>>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
>> driver if
>>>> it
>>>>>>>>>>> managed.
>>>>>>>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
>>>>>>>> whatever
>>>>>>>>>>>>> activity is required. In my case, this includes sending a
>>>>>>>> message to
>>>>>>>>>> the
>>>>>>>>>>>>> host where the VM is running to have, say XenServer, add a
>>>>>>>> storage
>>>>>>>>>>>>> repository (based on the IP address of the SAN, the IQN of
>> the
>>>>>>>> SAN
>>>>>>>>>>>> volume,
>>>>>>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
>>>> can
>>>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>>>> storage repository). After this, the normal attach-volume
>>>>>>>> message is
>>>>>>>>>> sent
>>>>>>>>>>>>> to the host by VolumeManagerImpl.
>>>>>>>>>>>>
>>>>>>>>>>>> There should be **no** code from a storage driver to a
>>>> hypervisor.
>>>>>>>> I
>>>>>>>>>>>> apologize for the repetition, but we simply can not have
>>>> hypervisor
>>>>>>>>>>>> specific code in the storage layer. The circular dependencies
>>>>>>>> between
>>>>>>>>>> the
>>>>>>>>>>>> two layers are not sustainable in the long term. Either the
>>>>>>>>>> VirtualManager
>>>>>>>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>>>>>>>>> coordinate
>>>>>>>>>>>> volume creation and then populating the SR. Ideally, we can
>>>>>>>>>> generalize the
>>>>>>>>>>>> process flow for attaching volumes such that the Xen
>> hypervisor
>>>>>>>> plugin
>>>>>>>>>>>> would only implement callbacks to perform the attach action
>> and
>>>>>>>> create
>>>>>>>>>>> the
>>>>>>>>>>>> structure and SR. To my mind, the SolidFire driver should
>>>> only be
>>>>>>>>>>>> allocating space and providing information about contents
>> (e.g.
>>>>>>>> space
>>>>>>>>>>>> available, space consumed, streams to a URI, file handle for a
>>>> URI,
>>>>>>>>>> etc)
>>>>>>>>>>>> and capabilities.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> * The reverse is performed for a detach-volume command.
>>>>>>>>>>>>>
>>>>>>>>>>>>> * Right now I simply "return true;" for isManaged() in my
>>>> driver.
>>>>>>>>>>>> Edison's
>>>>>>>>>>>>> default driver simply does a "return false;". We could add a
>>>> new
>>>>>>>>>>>> parameter
>>>>>>>>>>>>> to the createStoragePool API command, if we want, to
>> remove
>>>>> the
>>>>>>>>>>>> hard-coded
>>>>>>>>>>>>> return values in the drivers (although my driver will probably
>>>>>>>> just
>>>>>>>>>>>> ignore
>>>>>>>>>>>>> this parameter and always return true since it wouldn't make
>>>>>>>> sense
>>>>>>>>>> for it
>>>>>>>>>>>>> to ever return false). We'd need another column in the
>>>>>>>> storage_pool
>>>>>>>>>>> table
>>>>>>>>>>>>> to store this value.
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, I think we should have a parameter added to the
>>>>>>>>> createStoragePool
>>>>>>>>>>>> surfaced to the HTTP API that allows DataStores to be
>>>> configured
>>>>>>>> for
>>>>>>>>>>>> management when their underlying drivers support it. To
>>>> simplify
>>>>>>>>>> things,
>>>>>>>>>>>> this flag should only be mutable when the DataStore is
>>>> created. It
>>>>>>>>>> would be
>>>>>>>>>>>> a bit crazy to take a DataStore from managed to unmanaged
>> after
>>>>>>>>>> creation.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I agree, John. Just wanted to point out that I have a
>> working
>>>>>>>> GUI
>>>>>>>>>> for
>>>>>>>>>>>> you
>>>>>>>>>>>>>> to review (in that document), if you'd like to check it out.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would like the UIs of two features reviewed together to
>>>>>>>> ensure
>>>>>>>>>>>>>>> consistency across the concepts of hypervisor throttled
>> IOPs
>>>>>>>> and
>>>>>>>>>>>>>>> storage device provisioned IOPs. I see the potential for
>>>>>>>>>> confusion,
>>>>>>>>>>>>>>> and I think a side-by-side Ui review of these features will
>>>>>>>> help
>>>>>>>>>>>>>>> minimize any potential confusion.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As I mentioned, the term reconciliation issue will work
>>>> itself
>>>>>>>> if
>>>>>>>>>> it
>>>>>>>>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
>>>>>>>>>> throttled
>>>>>>>>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
>> sent
>>>>> to
>>>>>>>>> you
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> other day to see how the GUI looks for provisioned
>> storage
>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Several months ago, I put this topic out on the e-mail list
>>>>>>>> and we
>>>>>>>>>>>>>>> decided
>>>>>>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
>>>>> Offering
>>>>>>>>>> dialog.
>>>>>>>>>>>>>>>> Other storage vendors are coming out with QoS, so they
>>>>> should
>>>>>>>>> be
>>>>>>>>>>> able
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> leverage this GUI going forward (even if they, say, only
>>>> use
>>>>>>>> Max
>>>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>> These fields are optional and can be ignored for storage
>>>> that
>>>>>>>>>> does not
>>>>>>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
>>>> the
>>>>>>>>>> admin can
>>>>>>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and
>> Burst
>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature
>> or
>>>>>>>> mine,
>>>>>>>>>> as
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> decided by the admin).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm not sure what we can do about these two features
>>>>>>>> expressing
>>>>>>>>>> the
>>>>>>>>>>>>>>> speed
>>>>>>>>>>>>>>>> in different terms. I've never seen a SAN express the
>> IOPS
>>>> for
>>>>>>>>>> QoS in
>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>> way other than total IOPS (i.e. not broken in into
>>>> read/write
>>>>>>>>>> IOPS).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We have been down the rabbit hole a bit on the
>>>>>>>>> Storage/Hypervisor
>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>> separation, but we still need to reconcile the behavior
>> of
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
>>>>>>>> following
>>>>>>>>>>>> issues
>>>>>>>>>>>>>>>>> outstanding:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>>>>>>>>> read/write
>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>> where as storage provisioned IOPS are expressed as
>> total
>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
>> to
>>>>>>>>> storage
>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>>>>>>>>> 3. How should usage data be captured for throttled and
>>>>>>>>>> provisioned
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>> that will permit providers to discriminate these
>>>> guaranteed
>>>>>>>>>>>> operations
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>>>>>>>>> 4. What is the user experience for throttled and
>>>> provisioned
>>>>>>>>> IOPS
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> My thinking is that a VM can have either utilize
>>>> hypervisor
>>>>>>>>>> throttled
>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
>>>> solve
>>>>>>>>>> items 1
>>>>>>>>>>>>>>> and 2.
>>>>>>>>>>>>>>>>> Since the two facilities would not be permitted to
>> operate
>>>>>>>>>> together,
>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>> do not need to be semantically compatible. I think item
>> 3
>>>>>>>> can be
>>>>>>>>>>>>>>> resolved
>>>>>>>>>>>>>>>>> with an additional flag or two on the usage records. As
>>>> for
>>>>>>>>>> Item 4,
>>>>>>>>>>>> I
>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>> not familiar with how these two enhancements are
>>>>> depicted in
>>>>>>>>> the
>>>>>>>>>>> user
>>>>>>>>>>>>>>>>> interface. I think we need to review the UI
>> enhancements
>>>>> for
>>>>>>>>>> both
>>>>>>>>>>>>>>>>> enhancements and ensure they are consistent.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>>>>> <us...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> John and Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
>>>>> 2780/2729,
>>>>>>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>>>>>>>>> CLOUDSTACK-
>>>>>>>>>>> 2875 ) this
>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
>> and
>>>>>>>> ask
>>>>>>>>>> for
>>>>>>>>>>>>>>> second
>>>>>>>>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski
>> <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I believe I understand where you're going with this,
>>>> John,
>>>>>>>> and
>>>>>>>>>>> have
>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> "I am fine with the VolumeManager determining
>>>>> whether or
>>>>>>>>>>> not a
>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
>> but an
>>>>>>>>> actual
>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
>>>>>>>> resources
>>>>>>>>>> for
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
>>>>> method
>>>>>>>>>>> added to
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
>>>>>>>> then the
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> framework could call the manage() (or whatever
>> name)
>>>>>>>>> method
>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
>> well)
>>>>> and
>>>>>>>>> this
>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>> into a new method in the hypervisor code to create,
>> say
>>>>> on
>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> SR?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>>>>>>>>>>> DataStoreDriver.
>>>>>>>>>>>>>>> Since
>>>>>>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary
>> storage,
>>>>> I
>>>>>>>>>> would
>>>>>>>>>>>> add
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
>>>>> straw
>>>>>>>>> man
>>>>>>>>>>> term
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> something other than volume) methods to
>>>>>>>>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a
>> mutable
>>>>>>>>> property
>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> DataStore
>>>>>>>>>>>>>>>>>>>> which can be enabled if/when the underlying
>>>>>>>>> DataStoreDriver
>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>>>>>>> This approach allows operators to override
>>>>> manageability
>>>>>>>> of
>>>>>>>>>>>> devices.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
>> Xen
>>>>>>>> plugin
>>>>>>>>>>>> would
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> responsible for composing DataStore/Volume
>> methods
>>>>> to
>>>>>>>>>>> create any
>>>>>>>>>>>>>>>>>>>> directories or files necessary for the SR. There
>>>> should
>>>>>>>> be no
>>>>>>>>>>>>>>>>>>> dependencies
>>>>>>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
>>>>>>>> earlier,
>>>>>>>>>> such
>>>>>>>>>>>>>>>>>>> circular
>>>>>>>>>>>>>>>>>>>> dependencies will lead to a tangled,
>> unmaintainable
>>>>> mess.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Just want to make sure I'm on the same page with
>> you.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>>>>>>>>>>> <jb...@basho.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
>> layer
>>>>> that
>>>>>>>>>>>> supports n
>>>>>>>>>>>>>>>>>>>>> devices types with each specific behaviors of m
>>>>>>>> hypervisors.
>>>>>>>>>>>> Such
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> scenario will create an unmaintainable and
>> untestable
>>>>>>>>> beast.
>>>>>>>>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>>>>>>>>> thoughts and recommendations are driven to
>> evolve
>>>>> the
>>>>>>>>>>> Storage
>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I am fine with the VolumeManager determining
>>>>> whether
>>>>>>>>> or
>>>>>>>>>>> not a
>>>>>>>>>>>>>>> Volume
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
>> but
>>>>> an
>>>>>>>>> actual
>>>>>>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
>>>>>>>> resources
>>>>>>>>>> for
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
>>>> needs
>>>>>>>> to
>>>>>>>>>>>> indicate
>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>> or not it supports management operations.
>> Finally, I
>>>>>>>> think
>>>>>>>>>> we
>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>>>>>>> provide the ability for an administrator to elect to
>>>> have
>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
>> capable
>>>>>>>>> managing
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>> but the administrator has elected to leave it
>>>>> unmanaged).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Creation of a structure on the volume should be
>> done
>>>>> in
>>>>>>>>> the
>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>> hypervisor module using methods exposed by the
>>>>> Storage
>>>>>>>>>>> layer to
>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>> low-level operations (e.g. make directories,
>> create a
>>>>>>>> file,
>>>>>>>>>> etc).
>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>>>>>>>>>> hypervisor, as
>>>>>>>>>>>>>>> such,
>>>>>>>>>>>>>>>>>>>>> should be confined to its implementation. From
>> my
>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>> nothing
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> the Storage layer should be concerned with
>> content.
>>>>> From
>>>>>>>>> its
>>>>>>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>>>>>>> structure and data are opaque. It provides the
>> means
>>>>> to
>>>>>>>>>> query
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> support the interpretation of the content by
>> higher-
>>>>> level
>>>>>>>>>>> layers
>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
>>>>> composition
>>>>>>>>> of
>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>>>> from the Storage layer that varies based on the
>>>>> Volume
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> protocol
>>>>>>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is
>>>>> implemented
>>>>>>>> in
>>>>>>>>>> my
>>>>>>>>>>>>>>> patch,
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>>>>>>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
>> the
>>>>>>>>>>> hypervisor.
>>>>>>>>>>>> Before
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> command is sent, we could check to see if we're
>>>>> dealing
>>>>>>>>> with
>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>> (or
>>>>>>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
>>>> send a
>>>>>>>>>> "Create
>>>>>>>>>>>>>>> SR"
>>>>>>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
>>>>> would
>>>>>>>>> then
>>>>>>>>>>>>>>> proceed
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This way the attach logic remains the same and
>> we just
>>>>>>>> add
>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>>>> command to the agent code that is called for this
>>>>>>>> particular
>>>>>>>>>>> type
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I created a document for a customer today that
>>>>> outlines
>>>>>>>>> how
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>> works from a user standpoint. This will probably
>> be
>>>> of
>>>>>>>> use
>>>>>>>>>> to
>>>>>>>>>>>>>>> you, as
>>>>>>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I have shared this document with you (you
>> should
>>>>> have
>>>>>>>>>>> received
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
>>>>> thoughts,
>>>>>>>>> but
>>>>>>>>>>>> want
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
>>>> solidfire,
>>>>>>>>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
>> in a
>>>>>>>>>>> coherent
>>>>>>>>>>>>>>> form
>>>>>>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
>> introduces
>>>>> the
>>>>>>>>>> notion
>>>>>>>>>>>> of
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
>> introduces
>>>>>>>> the
>>>>>>>>>>> notion
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The storage framework that leverages the
>> plug-in
>>>>> was
>>>>>>>>>>>>>>> incomplete,
>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
>> spending on
>>>>>>>> this.
>>>>>>>>>> :)
>>>>>>>>>>> Do
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>> recommendation as to how we should
>> accomplish
>>>>>>>>> what
>>>>>>>>>>> you're
>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
>> <
>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
>> introduces
>>>>> the
>>>>>>>>>> notion
>>>>>>>>>>>> of
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the
>> management
>>>>>>>>>>> capabilities
>>>>>>>>>>>>>>> (i.e.
>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
>> it
>>>>>>>>> managed
>>>>>>>>>>> by
>>>>>>>>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
>> with
>>>>> a
>>>>>>>>>>> semantic
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
>> managed.
>>>>> I
>>>>>>>>> also
>>>>>>>>>>> want
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
>> from
>>>>> the
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
>> Burwell <
>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
>>>>> questions
>>>>>>>>> for
>>>>>>>>>>> me
>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
>> LUNs
>>>>> and
>>>>>>>>> SAN
>>>>>>>>>>>> volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
>> LUN
>>>>> (LUN
>>>>>>>>> 0).
>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
>> Volumes
>>>>> and
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
>> management
>>>>>>>>> server or
>>>>>>>>>>>>>>> externally
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
>> SAN
>>>>>>>>> volume
>>>>>>>>>>> (same
>>>>>>>>>>>> as a
>>>>>>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>>>>>>>>> created by the management server (via
>> the
>>>>> plug-in)
>>>>>>>>>>> the
>>>>>>>>>>>> first
>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
>> hypervisor.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
>> in,
>>>>> but
>>>>>>>>>> still
>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
>> already
>>>>>>>>> today
>>>>>>>>>>>> (prior
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
>> and -
>>>>> in
>>>>>>>>> this
>>>>>>>>>>> case -
>>>>>>>>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
>> While
>>>>> you
>>>>>>>>> can do
>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
>> management
>>>>> server
>>>>>>>>> or
>>>>>>>>>>> externally
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
>> SAN
>>>>>>>>> volumes
>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
>> in).
>>>>> There
>>>>>>>>> is
>>>>>>>>>>> no
>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
>> SAN
>>>>>>>>> volume
>>>>>>>>>>> and a
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
>> that
>>>>>>>>>>> supports
>>>>>>>>>>>> true
>>>>>>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
>> are
>>>>>>>> related
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
>>>>> multiple
>>>>>>>>> VMs
>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
>> storage."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
>> map a
>>>>>>>>> single
>>>>>>>>>>> SAN
>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
>> have
>>>>>>>>> multiple
>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
>>>>> guarantee
>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
>>>>> volume
>>>>>>>>> house
>>>>>>>>>>> more
>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
>> today
>>>>>>>>> without
>>>>>>>>>>> any of
>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
>>>>> volume
>>>>>>>>> for
>>>>>>>>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
>> created."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
>>>>> concerns me
>>>>>>>>> --
>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
>> layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
>> is
>>>>> today
>>>>>>>>>> only
>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>> (as
>>>>>>>>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
>> have
>>>>> not
>>>>>>>>>>> PreSetup an
>>>>>>>>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>> volume
>>>>> from
>>>>>>>>>>> VMware, it
>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
>> volume
>>>>> from
>>>>>>>>> KVM,
>>>>>>>>>>> it uses
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
>> thinking
>>>>>>>>> here
>>>>>>>>>>> was to
>>>>>>>>>>>>>>> make a
>>>>>>>>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
>> ability
>>>>>>>>> Edison
>>>>>>>>>>> added
>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
>> more
>>>>>>>>> specificy,
>>>>>>>>>>> though:
>>>>>>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
>> Mike
>>>>>>>>> Tutkowski
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
>> shouldn't
>>>>> know
>>>>>>>>> or
>>>>>>>>>>> care
>>>>>>>>>>>> that
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
>> answer
>>>>>>>>>>> questions
>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
>> the
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
>> the
>>>>>>>>> hypervisor
>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>> knows
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
>>>>> constraint
>>>>>>>>>>> and just
>>>>>>>>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
>> way
>>>>> for
>>>>>>>>> the
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
>>>>> perform
>>>>>>>>> the
>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
>> John
>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
>> be
>>>>> able to
>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>>>>>>>>> CloudStack
>>>>>>>>>>> (e.g. the
>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
>> operator has
>>>>>>>>> chosen
>>>>>>>>>>> to
>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
>> reasoning
>>>>>>>>> make
>>>>>>>>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
>> creating a
>>>>> SAN
>>>>>>>>>>> volume via
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
>> hypervisor)
>>>>> would
>>>>>>>>>>> work unless
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
>> say,
>>>>> an
>>>>>>>>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
>> of
>>>>>>>>>>> CloudStack, an
>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
>> and
>>>>>>>>> exposes it
>>>>>>>>>>>> through
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
>> over
>>>>> to
>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
>> LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
>> to my
>>>>>>>>>>> changes, we
>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
>> the
>>>>>>>>>>> necessary
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
>> maybe
>>>>> have
>>>>>>>>> the
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
>>>>> support
>>>>>>>>> from
>>>>>>>>>>> the
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
>> specifically
>>>>> to
>>>>>>>>> create
>>>>>>>>>>> the
>>>>>>>>>>>>>>> SR
>>>>>>>>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
>> management
>>>>> plugin
>>>>>>>>>>> should
>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
>> determine
>>>>>>>>> available
>>>>>>>>>>> for
>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>>>>>>>>> interrogation of
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
>> device
>>>>> itself
>>>>>>>>>>> shouldn't
>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
>> simply be
>>>>> able
>>>>>>>>> to
>>>>>>>>>>>> answer
>>>>>>>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
>> Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
>> SolidFire
>>>>>>>>> driver
>>>>>>>>>>> (which
>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
>>>>> done, a
>>>>>>>>> new
>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
>> Offering
>>>>>>>>> that
>>>>>>>>>>>> references
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
>> Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
>>>>> CloudStack
>>>>>>>>>>> volume. This
>>>>>>>>>>>>>>> leads
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
>>>>> CloudStack
>>>>>>>>>>> volume to a
>>>>>>>>>>>>>>> VM
>>>>>>>>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
>> framework
>>>>> calling
>>>>>>>>> the
>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
>> case, a
>>>>>>>>> SAN).
>>>>>>>>>>> The
>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
>> applicable
>>>>> data
>>>>>>>>>>> (like the
>>>>>>>>>>>>>>> IQN of
>>>>>>>>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
>> invoked if
>>>>> the
>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
>> attach,
>>>>> the
>>>>>>>>>>> volume
>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
>> and
>>>>> the
>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
>> is run
>>>>> and
>>>>>>>>>>> detects
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
>> the
>>>>> form
>>>>>>>>> of a
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
>> PM,
>>>>> Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
>> sync on
>>>>>>>>>>> terminology,
>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
>> physical
>>>>>>>>>>> allocation on
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
>> try to
>>>>>>>>> mean
>>>>>>>>>>> 'SAN
>>>>>>>>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
>> end
>>>>> user
>>>>>>>>> can
>>>>>>>>>>> make in
>>>>>>>>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
>> PM,
>>>>> Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
>> very
>>>>> well
>>>>>>>>> make
>>>>>>>>>>> sense, but
>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
>> Edison
>>>>> in on
>>>>>>>>>>> this
>>>>>>>>>>>>>>> conversation
>>>>>>>>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>>>>>>>>> approach I
>>>>>>>>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
>> PM,
>>>>>>>>> John
>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
>> combining
>>>>> two
>>>>>>>>>>> distinct
>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
>>>>> protocols.
>>>>>>>>> In
>>>>>>>>>>> both
>>>>>>>>>>>>>>> cases, we
>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
>>>>> system to
>>>>>>>>>>>> create/delete
>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
>> requires
>>>>> the
>>>>>>>>>>> volume to
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
>>>>> context.
>>>>>>>>> To
>>>>>>>>>>> ensure
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
>> these
>>>>>>>>>>> definitions,
>>>>>>>>>>>>>>> refers to
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
>>>>> correct?
>>>>>>>>>>> Minimally,
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
>> storage
>>>>> device
>>>>>>>>> to
>>>>>>>>>>> move bits
>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>>>>>>>>> Optionally, a
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
>>>>> Therefore,
>>>>>>>>>>> we can have
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
>> SR,
>>>>> and
>>>>>>>>> we
>>>>>>>>>>> can have a
>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
>> CloudStack is
>>>>>>>>> capable
>>>>>>>>>>> of
>>>>>>>>>>>>>>> allocating
>>>>>>>>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
>>>>> CloudStack
>>>>>>>>> may
>>>>>>>>>>> be able
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
>> leave it
>>>>>>>>>>> unmanaged by
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
>> and
>>>>> the
>>>>>>>>>>> operator has
>>>>>>>>>>>>>>> chosen
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
>> CloudStack).
>>>>> Does
>>>>>>>>> my
>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
>> above
>>>>> are
>>>>>>>>>>> reasonable, it
>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
>>>>> concerns
>>>>>>>>> from
>>>>>>>>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
>>>>> attached
>>>>>>>>> driver
>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
>> and
>>>>>>>>> establish
>>>>>>>>>>> a
>>>>>>>>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
>> separate
>>>>> that
>>>>>>>>> is
>>>>>>>>>>>>>>> associated
>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
>>>>> hypervisor
>>>>>>>>>>> drivers can
>>>>>>>>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
>>>>> encounter
>>>>>>>>> a
>>>>>>>>>>> device
>>>>>>>>>>>> managed
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
>>>>> exposed
>>>>>>>>> by
>>>>>>>>>>> the driver to
>>>>>>>>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
>> make
>>>>>>>>> sense,
>>>>>>>>>>> then we can
>>>>>>>>>>>>>>> sit
>>>>>>>>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
>>>>> Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
>>>>> between
>>>>>>>>> the
>>>>>>>>>>> current
>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
>> has to
>>>>> go in
>>>>>>>>> and
>>>>>>>>>>> create
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
>> point in
>>>>> time,
>>>>>>>>>>> the
>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
>>>>> allocated).
>>>>>>>>>>> Future
>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
>> was
>>>>> created
>>>>>>>>>>> behind the
>>>>>>>>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
>> has
>>>>> to
>>>>>>>>> go in
>>>>>>>>>>> and
>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
>> and
>>>>> delete
>>>>>>>>>>> volumes on
>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
>> enabled via
>>>>> the
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
>> volume
>>>>> that
>>>>>>>>> was
>>>>>>>>>>> created,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
>>>>> volume.
>>>>>>>>> After
>>>>>>>>>>> this is
>>>>>>>>>>>>>>> done,
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
>> question is
>>>>> called.
>>>>>>>>>>> No
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
>>>>> volume
>>>>>>>>> was
>>>>>>>>>>> just
>>>>>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> 3:21
>>>>> PM,
>>>>>>>>> Mike
>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
>> so I
>>>>> should
>>>>>>>>>>> probably
>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
>> that is
>>>>>>>>> pre-
>>>>>>>>>>> allocated
>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
>> a
>>>>>>>>> hypervisor
>>>>>>>>>>> data
>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
>> XenServer
>>>>> SR),
>>>>>>>>> then
>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
>> CloudStack.
>>>>> Disks
>>>>>>>>> (VDI)
>>>>>>>>>>> are later
>>>>>>>>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
>> needed. In
>>>>>>>>> these
>>>>>>>>>>> cases, the
>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>> structure is
>>>>>>>>> already
>>>>>>>>>>> in place
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
>>>>> structure
>>>>>>>>> to
>>>>>>>>>>> the VM in
>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
>> Storage
>>>>> that is
>>>>>>>>> not
>>>>>>>>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
>> this
>>>>> could
>>>>>>>>> be a
>>>>>>>>>>> SAN
>>>>>>>>>>>> (not
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
>> hypervisor
>>>>> data
>>>>>>>>>>> structure
>>>>>>>>>>>>>>> must be
>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
>>>>> because
>>>>>>>>> these
>>>>>>>>>>> types of
>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
>> data
>>>>>>>>>>> structure by an
>>>>>>>>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
>>>>> XenServer
>>>>>>>>> for
>>>>>>>>>>> this
>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
>> to the
>>>>> VM
>>>>>>>>> in
>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> 3:13
>>>>> PM,
>>>>>>>>>>> John Burwell
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>>>>> implementation of
>>>>>>>>> the
>>>>>>>>>>> Dynamic
>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
>> which
>>>>> why I
>>>>>>>>> ask
>>>>>>>>>>> about the
>>>>>>>>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
>> what is
>>>>> the
>>>>>>>>>>> definition
>>>>>>>>>>>> of
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
>> PM,
>>>>>>>>> Mike
>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
>> iSCSI
>>>>> type
>>>>>>>>> is
>>>>>>>>>>>> uniquely
>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
>> Primary
>>>>>>>>> Storage
>>>>>>>>>>> that is
>>>>>>>>>>>>>>> directly
>>>>>>>>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
>> skip the
>>>>>>>>> step of
>>>>>>>>>>> going
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
>> based
>>>>> on
>>>>>>>>> that
>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>> target as
>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
>> only
>>>>>>>>>>> supported for
>>>>>>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
>> must
>>>>> first go
>>>>>>>>> to
>>>>>>>>>>> the
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
>> what
>>>>> RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> at
>>>>> 2:13
>>>>>>>>> PM,
>>>>>>>>>>> John
>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
>> the
>>>>> code,
>>>>>>>>> what
>>>>>>>>>>> is the
>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
>> Why
>>>>> isn't
>>>>>>>>> RBD
>>>>>>>>>>> considered
>>>>>>>>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>> 3:46
>>>>> PM,
>>>>>>>>> Mike
>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
>>>>> storage is
>>>>>>>>>>> defined in
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>>>>>>>>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
>> //
>>>>> local
>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> NetworkFilesystem(true),
>>>>>>>>> //
>>>>>>>>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
>>>>> shared
>>>>>>>>> LUN,
>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
>> e.g.,
>>>>> ZFS
>>>>>>>>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
>> iso
>>>>> image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
>>>>> XenServer
>>>>>>>>> local
>>>>>>>>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
>>>>> VMware
>>>>>>>>> VMFS
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
>> for
>>>>>>>>>>> XenServer, Storage
>>>>>>>>>>>>>>> Pool
>>>>>>>>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
>>>>> XenServer
>>>>>>>>> local
>>>>>>>>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
>>>>> dynamic,
>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> StoragePoolType(boolean
>>>>>>>>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
>> shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
>>>>> isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013 at
>>>>> 1:41
>>>>>>>>> PM,
>>>>>>>>>>> Mike
>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
>> let's say
>>>>>>>>> another
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> company
>>>>>>>>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
>> leverage its
>>>>>>>>> Quality
>>>>>>>>>>> of
>>>>>>>>>>>> Service
>>>>>>>>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> storage, as
>>>>> well.
>>>>>>>>>>> They would
>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
>> made
>>>>> the
>>>>>>>>>>> necessary
>>>>>>>>>>>>>>> changes to
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
>> plug-
>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013
>>>>> at
>>>>>>>>> 1:39
>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
>> imply
>>>>> the
>>>>>>>>>>> XenServer code
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
>>>>> attach
>>>>>>>>> logic is
>>>>>>>>>>> now aware
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
>> an
>>>>>>>>>>> implementation of
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
>> to 4.2
>>>>>>>>> with
>>>>>>>>>>> Edison's
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
>> a
>>>>> new
>>>>>>>>>>> framework that
>>>>>>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
>>>>> dynamically.
>>>>>>>>>>> However, when I
>>>>>>>>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
>> realized
>>>>> that
>>>>>>>>> it
>>>>>>>>>>> was not
>>>>>>>>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
>> CloudStack
>>>>> could
>>>>>>>>> do
>>>>>>>>>>> with these
>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
>>>>> recognize
>>>>>>>>> this
>>>>>>>>>>> new type of
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>>>>> structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013
>>>>> at
>>>>>>>>> 1:28
>>>>>>>>>>> PM, John
>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
>> odd
>>>>> to me
>>>>>>>>>>> that any
>>>>>>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
>> care
>>>>>>>>> about
>>>>>>>>>>> details. I
>>>>>>>>>>>>>>> expect
>>>>>>>>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
>> of
>>>>>>>>> operations
>>>>>>>>>>> that can be
>>>>>>>>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> implementations to
>>>>>>>>>>> allocate
>>>>>>>>>>>> space/create
>>>>>>>>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
>> this
>>>>>>>>>>> dependency, we are
>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
>> will
>>>>> make
>>>>>>>>> the
>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
>> I
>>>>>>>>>>> understanding that the
>>>>>>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>>>>>>>>>>> CitrixResourceBase
>>>>>>>>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>>> 2013 at
>>>>>>>>> 3:12
>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
>> this
>>>>> in a
>>>>>>>>> bit
>>>>>>>>>>> more
>>>>>>>>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
>> and
>>>>> aside
>>>>>>>>> from
>>>>>>>>>>> one setup
>>>>>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
>> a
>>>>>>>>> volume on
>>>>>>>>>>> the
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
>> data
>>>>>>>>> structure
>>>>>>>>>>> to make
>>>>>>>>>>>> use
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>>>>>>>>> XenServer
>>>>>>>>>>> or a
>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
>>>>> storage
>>>>>>>>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
>> new
>>>>>>>>> storage
>>>>>>>>>>> framework,
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
>> easily
>>>>>>>>> create
>>>>>>>>>>> a 1:1
>>>>>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
>> the
>>>>> VM's
>>>>>>>>> data
>>>>>>>>>>> disk
>>>>>>>>>>>>>>> (necessary
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
>> write
>>>>> a
>>>>>>>>> plug-
>>>>>>>>>>> in that is
>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
>> that
>>>>> the
>>>>>>>>>>> storage
>>>>>>>>>>>> framework
>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>>>>>>>>> hypervisor-
>>>>>>>>>>> specific data
>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
>> the
>>>>> case,
>>>>>>>>> I've
>>>>>>>>>>> been
>>>>>>>>>>>>>>> enhancing
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
>> and
>>>>>>>>>>> submitted. I've got
>>>>>>>>>>>>>>> ESX(i)
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
>> the
>>>>> 4.2
>>>>>>>>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
>> help a
>>>>> bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
>> 3,
>>>>> 2013 at
>>>>>>>>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
>>>>> plug-in -
>>>>>>>>> by
>>>>>>>>>>> itself - is
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
>> with
>>>>> the
>>>>>>>>>>> volume-attach
>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
>> calls
>>>>> into
>>>>>>>>> the
>>>>>>>>>>> plug-in to
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
>> time
>>>>>>>>> comes to
>>>>>>>>>>> attach the
>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
>> smart
>>>>>>>>> enough
>>>>>>>>>>> to recognize
>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
>>>>> volume
>>>>>>>>> has
>>>>>>>>>>> just been
>>>>>>>>>>>>>>> created)
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
>> (for
>>>>>>>>>>> XenServer) or a
>>>>>>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
>> was
>>>>> just
>>>>>>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
>>>>> spending
>>>>>>>>> most
>>>>>>>>>>> of my time
>>>>>>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
>> clear it
>>>>> up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun 3,
>>>>> 2013
>>>>>>>>> at
>>>>>>>>>>> 12:48 PM,
>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
>> explain
>>>>>>>>> why
>>>>>>>>>>> the the storage
>>>>>>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>> 2013,
>>>>> at
>>>>>>>>> 1:21
>>>>>>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
>>>>> ultimately I
>>>>>>>>>>> would like to
>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
>> I
>>>>> think
>>>>>>>>> I'm
>>>>>>>>>>> just out of
>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
>> now this
>>>>>>>>> plug-in
>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
>> to 4.2
>>>>>>>>> feature
>>>>>>>>>>> freeze, I
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
>> to
>>>>> be
>>>>>>>>> clear,
>>>>>>>>>>> this is
>>>>>>>>>>>> all
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>> Compute
>>>>>>>>>>> Offerings post
>>>>>>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun 3,
>>>>>>>>> 2013
>>>>>>>>>>> at 11:14 AM,
>>>>>>>>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
>> any
>>>>> plan
>>>>>>>>> on
>>>>>>>>>>> supporting
>>>>>>>>>>>>>>> KVM in
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
>> Original
>>>>>>>>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
>> "Mike
>>>>>>>>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>>>>>>>>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
>>>>> Monday,
>>>>>>>>> June
>>>>>>>>>>> 3, 2013
>>>>>>>>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
>> Re:
>>>>>>>>> [MERGE]
>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
>> on
>>>>>>>>> merging
>>>>>>>>>>> Wei's feature
>>>>>>>>>>>>>>>>> first,
>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
>> feature
>>>>> is
>>>>>>>>> for
>>>>>>>>>>> KVM only,
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
>> Jun
>>>>> 3,
>>>>>>>>> 2013
>>>>>>>>>>> at 8:55 AM,
>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>> the
>>>>> billing,
>>>>>>>>> as
>>>>>>>>>>> no one works
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> billing
>>>>> by
>>>>>>>>>>> themselves. They
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> disk_offering of
>>>>>>>>> a
>>>>>>>>>>> VMs and
>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
>> user
>>>>> the
>>>>>>>>>>> exact
>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>> limitation
>>>>>>>>> for
>>>>>>>>>>> nics as
>>>>>>>>>>>> well.
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
>> part of
>>>>> I/O
>>>>>>>>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
>> my
>>>>>>>>>>> misunstand the second
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
>> with
>>>>>>>>> what
>>>>>>>>>>> you said about
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 2013/6/3
>>>>> John
>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Jun 3,
>>>>> 2013,
>>>>>>>>> at
>>>>>>>>>>> 2:13 AM, Wei
>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>> John,
>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> hope
>>>>> Mike's
>>>>>>>>>>> aswer helps you.
>>>>>>>>>>>>>>> I am
>>>>>>>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
>>>>> think
>>>>>>>>> billing
>>>>>>>>>>> should
>>>>>>>>>>>>>>> depend
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> limitation.
>>>>>>>>> Please
>>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>> the
>>>>> IO
>>>>>>>>>>> statistics
>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> virtual
>>>>>>>>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Going by
>>>>> the
>>>>>>>>> AWS
>>>>>>>>>>> model,
>>>>>>>>>>>> customers
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> provisioned
>>>>>>>>> IOPS,
>>>>>>>>>>> as well as,
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://aws.amazon.com/ebs/).
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> option to
>>>>>>>>> employ
>>>>>>>>>>> similar cost
>>>>>>>>>>>>>>>>> models.
>>>>>>>>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> billing
>>>>> model
>>>>>>>>> in
>>>>>>>>>>> the current
>>>>>>>>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> Do
>>>>> you
>>>>>>>>> mean
>>>>>>>>>>> IOPS runtime
>>>>>>>>>>>>>>>>> change?
>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> limitation for
>>>>>>>>> a
>>>>>>>>>>> running
>>>>>>>>>>>> virtual
>>>>>>>>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> CloudStack
>>>>>>>>> does
>>>>>>>>>>> not support
>>>>>>>>>>>>>>>>> changing
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> (computer
>>>>>>>>>>> offering or disk
>>>>>>>>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> meant at
>>>>> the
>>>>>>>>>>> Java interface
>>>>>>>>>>>>>>> level.
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> more
>>>>>>>>>>> generalize allocation
>>>>>>>>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> describe
>>>>> the
>>>>>>>>>>> service guarantees
>>>>>>>>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> It is a
>>>>>>>>> good
>>>>>>>>>>> question.
>>>>>>>>>>>>>>> Maybe it
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> disk_io_throttling as Mike
>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> type, I
>>>>> think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
>>>>> expand
>>>>>>>>> on
>>>>>>>>>>> my thoughts
>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> touch
>>>>> points
>>>>>>>>>>> between these two
>>>>>>>>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> need
>>>>> to be
>>>>>>>>>>> merged before
>>>>>>>>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> coordination
>>>>>>>>>>> between the
>>>>>>>>>>>> branches
>>>>>>>>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> disk_io_throttling)
>>>>>>>>>>> to
>>>>>>>>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 2013/6/3
>>>>>>>>> John
>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> The
>>>>> things I
>>>>>>>>>>> want to
>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
>> Is
>>>>> there
>>>>>>>>>>> value in
>>>>>>>>>>>> capturing
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> data
>>>>> model
>>>>>>>>> (e.g.
>>>>>>>>>>> for
>>>>>>>>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
>>>>> Should
>>>>>>>>> there
>>>>>>>>>>> be a common
>>>>>>>>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> provisioning
>>>>>>>>> at
>>>>>>>>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
>> How
>>>>> are
>>>>>>>>>>> conflicting
>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> hypervisor
>>>>>>>>> and
>>>>>>>>>>> storage device
>>>>>>>>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> lead
>>>>> to
>>>>>>>>>>> believe (and
>>>>>>>>>>>> billed)
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> storage
>>>>>>>>> device
>>>>>>>>>>> has been
>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> consistent
>>>>>>>>>>> configuration
>>>>>>>>>>>>>>> between a
>>>>>>>>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> time,
>>>>> but a
>>>>>>>>>>> later
>>>>>>>>>>>> modification
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Jun
>>>>> 2,
>>>>>>>>> 2013,
>>>>>>>>>>> at 8:38 PM,
>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>> believe
>>>>>>>>> Wei's
>>>>>>>>>>> feature deals
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> the
>>>>>>>>> hypervisor
>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> My
>>>>> feature
>>>>>>>>> is
>>>>>>>>>>> focused on
>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> hope
>>>>> that
>>>>>>>>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> Sun,
>>>>> Jun
>>>>>>>>> 2,
>>>>>>>>>>> 2013 at 6:35
>>>>>>>>>>>> PM,
>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> My
>>>>>>>>> opinion is
>>>>>>>>>>> that no
>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> issues
>>>>>>>>> have
>>>>>>>>>>> been resolved
>>>>>>>>>>>> and
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> total
>>>>> Ops
>>>>>>>>> vs
>>>>>>>>>>> discrete
>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
>>>>> reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido,
>>>>> I
>>>>>>>>> don't
>>>>>>>>>>> think this
>>>>>>>>>>>>>>> criteria
>>>>>>>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Also,
>>>>> how
>>>>>>>>>>> does this work
>>>>>>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> also
>>>>>>>>> involves
>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> scenario
>>>>>>>>>>> where provisioned
>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>> Jun 1,
>>>>>>>>> 2013,
>>>>>>>>>>> at 6:47 AM,
>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Sure. I
>>>>> will
>>>>>>>>>>> change it next
>>>>>>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> 2013/6/1
>>>>>>>>>>> Wido den Hollander
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>>>>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> Exactly. I
>>>>>>>>> have
>>>>>>>>>>> pushed the
>>>>>>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>> anyone
>>>>>>>>>>> object thems for
>>>>>>>>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> For
>>>>> the
>>>>>>>>> sake
>>>>>>>>>>> of clarity I
>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> the
>>>>> total
>>>>>>>>> IOps
>>>>>>>>>>> to R/W IOps
>>>>>>>>>>>>>>> asap
>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> only
>>>>> total
>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> You
>>>>> laid
>>>>>>>>> the
>>>>>>>>>>> groundwork for
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> however
>>>>>>>>>>> prevent that we
>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> 2013/5/31
>>>>>>>>>>> Wido den
>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>>>>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> +1 --
>>>>> this
>>>>>>>>>>> enhancement must
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> don't
>>>>> see
>>>>>>>>> how
>>>>>>>>>>> it could be
>>>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> split
>>>>> total
>>>>>>>>> IOPS
>>>>>>>>>>> into read
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> total
>>>>>>>>>>> unless/until we
>>>>>>>>>>>> decided
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> What
>>>>> Wei
>>>>>>>>>>> meant was merging
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 4.2
>>>>> branch
>>>>>>>>>>> and add Read /
>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> will be
>>>>>>>>>>> released with Read
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> This is
>>>>> to
>>>>>>>>>>> make the May 31st
>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (see
>>>>> other
>>>>>>>>>>> threads) then it
>>>>>>>>>>>>>>> won't
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> also
>>>>>>>>>>> completely agree that
>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> disk
>>>>> I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>> May 31,
>>>>>>>>>>> 2013, at 9:51 AM,
>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>>>>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Hi
>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> Thanks.
>>>>>>>>> Good
>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>> thought
>>>>>>>>>>> about at the
>>>>>>>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> difference
>>>>>>>>> of
>>>>>>>>>>> read and write
>>>>>>>>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> care
>>>>> the
>>>>>>>>>>> difference of sent
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> That
>>>>>>>>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Disk
>>>>> I/O is
>>>>>>>>>>> much more
>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> Implementing
>>>>>>>>>>> it will be some
>>>>>>>>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> implemented
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> few
>>>>> days.
>>>>>>>>> For
>>>>>>>>>>> the deadline
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> that ,
>>>>> if
>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>> think it's
>>>>>>>>> a
>>>>>>>>>>> feature we
>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> window
>>>>>>>>> we
>>>>>>>>>>> have to make sure
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> in
>>>>> 4.3,
>>>>>>>>> that
>>>>>>>>>>> will confuse
>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> 2013/5/31
>>>>>>>>>>> Wido den
>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> On
>>>>>>>>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>> would
>>>>>>>>> like to
>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>> nobody
>>>>>>>>>>> object, I will
>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> The
>>>>>>>>> purpose
>>>>>>>>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Virtual
>>>>>>>>>>> machines are running
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> share
>>>>>>>>> strage).
>>>>>>>>>>> Because of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> iops),
>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> one
>>>>> VM
>>>>>>>>> has
>>>>>>>>>>> large disk
>>>>>>>>>>>>>>> operation,
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> other
>>>>> VMs
>>>>>>>>>>> running on the
>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>> is
>>>>>>>>> neccesary
>>>>>>>>>>> to set the
>>>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> Looking at
>>>>>>>>> the
>>>>>>>>>>> code I see
>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Qemu
>>>>> and
>>>>>>>>>>> libvirt support
>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOps
>>>>>>>>> which
>>>>>>>>>>> could benefit a
>>>>>>>>>>>>>>> lot of
>>>>>>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> It's
>>>>> also
>>>>>>>>>>> strange, in the
>>>>>>>>>>>>>>> polling
>>>>>>>>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOps,
>>>>> but
>>>>>>>>> on
>>>>>>>>>>> the throttling
>>>>>>>>>>>>>>> side
>>>>>>>>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Write
>>>>> IOps
>>>>>>>>> are
>>>>>>>>>>> usually much
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> like a
>>>>> valid
>>>>>>>>>>> use-case where
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> IOps
>>>>> vs
>>>>>>>>> Read
>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Since
>>>>> this
>>>>>>>>> only
>>>>>>>>>>> supports KVM
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> value
>>>>> to at
>>>>>>>>>>> least have the
>>>>>>>>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> this
>>>>> later
>>>>>>>>>>> would be a lot of
>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> a
>>>>>>>>> hypervisor
>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> write
>>>>> you
>>>>>>>>> can
>>>>>>>>>>> always sum
>>>>>>>>>>>> both
>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Can
>>>>> you
>>>>>>>>>>> explain why you
>>>>>>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> The
>>>>>>>>> feature
>>>>>>>>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (1) set
>>>>> the
>>>>>>>>>>> maximum rate of
>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (2)
>>>>> change
>>>>>>>>> the
>>>>>>>>>>> maximum rate
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (3)
>>>>> limit
>>>>>>>>> the
>>>>>>>>>>> disk rate
>>>>>>>>>>>> (total
>>>>>>>>>>>>>>>>> bps
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> JIRA
>>>>> ticket:
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>> 1192
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> FS (I
>>>>> will
>>>>>>>>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> Merge
>>>>>>>>> check
>>>>>>>>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> Did
>>>>> you
>>>>>>>>>>> check the branch's
>>>>>>>>>>>>>>> RAT
>>>>>>>>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> Are
>>>>>>>>> there
>>>>>>>>>>> new dependencies
>>>>>>>>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>>>>> What
>>>>>>>>>>> automated testing
>>>>>>>>>>>> (unit
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Unit
>>>>> tests
>>>>>>>>> are
>>>>>>>>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>>>>> What
>>>>>>>>>>> testing has been done
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (1) set
>>>>> the
>>>>>>>>>>> bytes rate and
>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (2)
>>>>> VM
>>>>>>>>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> deploy,
>>>>>>>>> stop,
>>>>>>>>>>> start, reboot,
>>>>>>>>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (3)
>>>>>>>>> Volume
>>>>>>>>>>> operations,
>>>>>>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Attach,
>>>>>>>>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> To
>>>>> review
>>>>>>>>> the
>>>>>>>>>>> code, you can
>>>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Best
>>>>>>>>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>> <https://%0b/>> > >>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> [2]
>>>>>>>>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>>>>> VM
>>>>>>>>> Disk
>>>>>>>>>>> I/O
>>>>>>>>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Mike
>>>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *Senior
>>>>>>>>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> Advancing
>>>>>>>>> the
>>>>>>>>>>> way the world
>>>>>>>>>>>>>>> uses
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Advancing
>>>>> the
>>>>>>>>> way
>>>>>>>>>>> the world uses
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>>>>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Advancing the
>>>>>>>>> way
>>>>>>>>>>> the world uses
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> the
>>>>> way
>>>>>>>>> the
>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>> CloudStack
>>>>>>>>>>> Developer,
>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> the
>>>>> way
>>>>>>>>> the
>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>>>>>>>>> Developer, SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way
>>>>> the
>>>>>>>>>>> world uses the
>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>>>>>>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way
>>>>> the
>>>>>>>>> world
>>>>>>>>>>> uses the
>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>>>>>>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way the
>>>>>>>>> world
>>>>>>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>>> world
>>>>>>>>> uses
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>> Developer,
>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>>> world
>>>>>>>>> uses
>>>>>>>>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>>> uses
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>>> uses
>>>>>>>>> the
>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>>> the
>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>>> the
>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *(tm)*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *(tm)*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *(tm)*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *(tm)*****
>>>>
>>>>
>>>>
>>>> ****
>>>>
>>>> ** **
>>>>
>>>> --
>>>> *Mike Tutkowski*****
>>>>
>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>
>>>> e: mike.tutkowski@solidfire.com****
>>>>
>>>> o: 303.746.7302****
>>>>
>>>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*****
>>>>
>>>>
>>>>
>>>> ****
>>>>
>>>> ** **
>>>>
>>>> --
>>>> *Mike Tutkowski*****
>>>>
>>>> *Senior CloudStack Developer, SolidFire Inc.*****
>>>>
>>>> e: mike.tutkowski@solidfire.com****
>>>>
>>>> o: 303.746.7302****
>>>>
>>>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*****
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *(tm)*
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *(tm)*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
I don't understand why we need to add a "managed" in the UI and in "storage_pool" table?
AFAIK, what you guys trying to do is to enable attachVolume/DettachVolume if the volume is created on Solidfire, right?
If the volume is created on Solidfire already, by createAsync, then why we can't just send out all the information about the volume(it's volumeTO which will be returned by each storage driver) to hypervisor, then at the hypervisor side, check the volumeTO, if the volumeTO is managed, then doing whatever necessary operations(create SR, create VDI etc), then attach the VDI to volume.
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Monday, June 10, 2013 11:10 AM
> To: John Burwell
> Cc: Edison Su; dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> I believe we're on the same page, John.
>
> I would add a new parameter to the createStoragePool API command (let's
> call it "managed" and it can be true or false). If you pass in "true", but
> the driver does not support being managed by CloudStack, an exception is
> thrown by the driver. If you pass in "false", but the driver does not
> support being managed outside of CloudStack, an exception is thrown by the
> driver. It is possible one could write a driver that can be managed both
> inside or outside of CloudStack (the SolidFire driver requires being
> managed only by CloudStack).
>
> This managed value would end up in the storage_pool table in a new column
> ("managed").
>
> An example usage: After the storage framework calls on the driver's
> createAsync method, it will check to see if the driver is managed. If the
> driver is managed, the storage framework will invoke the proper (new)
> command of the hypervisor in use to set up, say, the SR for the SAN volume.
>
> There may be other cases when the isManaged() method is invoked on the
> driver and - if true is returned - the storage framework sends a particular
> command to the hypervisor in use.
>
> ********************
>
> As far as reworking the default storage plug-in (which sends commands to
> hypervisors) for 4.2, I would check with Edison on his schedule.
>
> Thanks!
>
>
> On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com>
> wrote:
>
> > Mike,
> >
> > I want to make sure we are on the same page regarding isManaged. I see
> > the addition of the following operations:
> >
> > * DataStoreDriver#supportsManagement() : boolean -- indicates whether
> or
> > not the driver support/implements the management functions
> > * DataStore#isManaged() : boolean -- indicates whether or not a device is
> > managed by CloudStack
> >
> > Only DataStores associated with DataStoreDrivers that support
> management
> > would be able to have the management flag enabled. This behavior allows
> > operators to declare their intention to allow CloudStack to manage a
> > device. This behavior would need to be exposed as flag on the HTTP API.
> > Additionally, this flag would only be settable when a device is
> > defined/created in the system (i.e. once a device is managed by CloudStack,
> > you can't unmanaged it without dissociating it from CloudStack). In the
> > future, we may consider relaxing that rule, but, in the short term, it
> > greatly decreases the alternative flows.
> >
> > In terms of removing the dependencies from Storage to Hypervisor, I
> would
> > really like to see this accomplished in 4.2. It will be much more
> > difficult to address this issue post 4.2 because we will have more plugins
> > to address. I also see this issue as a critical architectural issues that
> > for the management server. I recommend we identify the dependencies
> that
> > need to be inverted, and see if we can divide the work across a few more
> > resources.
> >
> > Thanks,
> > -John
> >
> > On Jun 10, 2013, at 1:36 PM, Mike Tutkowski
> <mi...@solidfire.com>
> > wrote:
> >
> > Sounds good, Edison :)
> >
> > I can implement John's isManaged() logic and resubmit my code for review.
> >
> > Perhaps in 4.3 I could address the direct-attach scenario. If it requires
> > some refactoring of the storage framework, then I could do that.
> >
> > Also, the default storage plug-in does have hypervisor logic in it in 4.2.
> > I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> > with the above work I mentioned.
> >
> > Are you OK with that, guys?
> >
> >
> > On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com> wrote:
> >
> >> Ok, I am Ok with the way you are talking about in 4.2.****
> >>
> >> ** **
> >>
> >> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >> *Sent:* Sunday, June 09, 2013 8:59 PM
> >> *To:* dev@cloudstack.apache.org
> >> *Cc:* Edison Su; John Burwell
> >> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
> >>
> >> ** **
> >>
> >> Another thing we might want to consider for 4.2 is that the way the code
> >> is currently implemented in master, I don't think it's even possible to
> >> write a plug-in that connects storage directly to a VM because the
> >> CloudStack storage framework will call the "attach" method of the
> >> hypervisor in use and this method will fail because it assumes there is
> >> (talking Xen here) an SR existent and in this situation there isn't one.*
> >> ***
> >>
> >> ** **
> >>
> >> My thinking is that we should ignore the direct-attach-of-volume-to-VM
> >> use case for 4.2. I think it will require too much of a rewrite at this
> >> point in development (fewer than three weeks left until code
> freeze).****
> >>
> >> ** **
> >>
> >> If the plug-in implements an isManaged() method (as John suggested),
> the
> >> storage framework can invoke the necessary commands on the
> hypervisor in
> >> use.****
> >>
> >> ** **
> >>
> >> For example, the createAsync method is called. The storage plug-in
> >> creates a SAN volume and updates the CS DB. The storage framework
> then asks
> >> the driver if it's managed. If it is, the storage framework executes a
> >> command against the hypervisor in use to have it create, say, an SR (with
> a
> >> single VDI that takes up the entire SR). Now, when the storage
> framework
> >> executes the "attach" command against the hypervisor, this command will
> >> work, as well (because that logic assumes the prior existence of an SR and
> >> it will, in fact, be present).****
> >>
> >> ** **
> >>
> >> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:****
> >>
> >> Hi Edison and John,****
> >>
> >> ** **
> >>
> >> I wanted to point out something I thought was relevant to this
> >> conversation.****
> >>
> >> ** **
> >>
> >> My plug-in supports both XenServer and ESX(i).****
> >>
> >> ** **
> >>
> >> These two hypervisors handle their respective hypervisor data structures
> >> (XenServer SR or ESX(i) datastore) quite differently.****
> >>
> >> ** **
> >>
> >> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> >> when you issue a remove-SR command to XenServer.****
> >>
> >> ** **
> >>
> >> When you later want to access the contents of that data, it is still
> >> safely stored on the LUN and can be accessed from XenServer (from the
> same
> >> or different cluster (what XenServer calls a resource pool)) by creating a
> >> new SR that is based on the same IQN and LUN.****
> >>
> >> ** **
> >>
> >> XenServer SRs exist at the cluster level.****
> >>
> >> ** **
> >>
> >> In ESX(i), the datastore that resides on the SAN LUN is actually
> >> destroyed when you issue a remove-datastore command to ESX(i). In
> other
> >> words, the contents of the SAN LUN are destroyed.****
> >>
> >> ** **
> >>
> >> ESX(i) datastores exist at the datacenter level.****
> >>
> >> ** **
> >>
> >> The XenServer behavior (not destroying the contents of the LUN when
> the
> >> SR is removed) is desirable; the ESX(i) behavior (destroying the contents
> >> of the LUN when the datastore is removed) is not desirable.****
> >>
> >> ** **
> >>
> >> I can get around the ESX(i) behavior by removing the referenced IQN
> from
> >> each host in the cluster when the CS storage framework notifies my plug-
> in
> >> of a detach-volume event. When the CS volume is later attached again
> >> (either to the same or a different ESX(i) cluster), I can have each host in
> >> the relevant ESX(i) cluster reference the IQN and the datastore will be
> >> back again.****
> >>
> >> ** **
> >>
> >> This sequence of events is critical for my plug-in to support both
> >> environments:****
> >>
> >> *XenServer flow (events for Driver class):*****
> >>
> >> createAsync (called right before first attach) = creates SAN volume****
> >>
> >> preAttachVolume = creates SR and creates VDI or just introduces SR****
> >>
> >> (CS volume gets attached and later detached)****
> >>
> >> postDetachVolume = removes SR (this does not delete data on SAN
> volume)**
> >> **
> >>
> >> deleteAsync (can be called when volume is not attached) = deletes the
> SAN
> >> volume****
> >>
> >> *VMware flow (events for Driver class):*****
> >>
> >> createAsync (called right before first attach) = creates SAN volume,
> >> creates datastore, and creates VMDK file****
> >>
> >> preAttachVolume = adds iSCSI target to each host in cluster****
> >>
> >> (CS volume gets attached and later detached)****
> >>
> >> postDetachVolume = removes iSCSI target from each host in cluster****
> >>
> >> deleteAsync (can be called when volume is not attached) = deletes
> >> datastore and deletes the SAN volume****
> >>
> >> ************************
> >>
> >> So, the way I have it written up here, the driver is talking to the
> >> hypervisor. As John has pointed out, this may not be ideal.****
> >>
> >> ** **
> >>
> >> Whatever we decide on, it needs to be able to support this kind of flow.
> >> :)****
> >>
> >> ** **
> >>
> >> We could go John's route, where the driver says if it's managed or not.**
> >> **
> >>
> >> ** **
> >>
> >> If it is, then the storage framework could send the necessary messages to
> >> the hypervisor in question at the right times (just migrate the hypervisor
> >> logic above from the driver to the right parts of the storage framework).
> >> ****
> >>
> >> ** **
> >>
> >> This is fine, except that it does not resolve Edison's use case of
> >> desiring to have the storage LUN potentially connected directly to a VM.*
> >> ***
> >>
> >> ** **
> >>
> >> I'm not sure we should design around that use case, though. Is it common
> >> enough? Has anyone asked for it? Is it even desirable? How are usage
> >> statistics gathered for such volumes? It's nice to offer the flexibility,
> >> but is it a good idea in this case?****
> >>
> >> ** **
> >>
> >> Thanks!****
> >>
> >> ** **
> >>
> >> ** **
> >>
> >> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:**
> >> **
> >>
> >>
> >>
> >> > -----Original Message-----
> >> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >> > Sent: Friday, June 07, 2013 4:08 PM
> >> > To: dev@cloudstack.apache.org
> >> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >> >
> >> > Hi John,
> >> >
> >> > Yes, you are correct that Xen must layer an SR on top of the iSCSI
> >> volume
> >> > to use it. Same story for VMware: ESX(i) must layer a datastore on top
> >> of
> >> > the iSCSI volume to use it. I look at it like they are layering a
> >> clustered
> >> > file system on the SAN volume so the hypervisors can share access to
> the
> >> > contents of the volume.
> >> >
> >> > In 4.1, by the time the hypervisor-attach-volume logic is called, the
> >> SR or
> >> > datastore has already been created (usually manually by an admin).
> >> >
> >> > This pre-setup of the, say, SR is, of course, not acceptable in an
> >> > environment where each CloudStack volume that a user creates is
> mapped
> >> > to a
> >> > single SAN volume (via the SR).
> >> >
> >> > The question comes down to who should allocate the SR dynamically.
> >> >
> >> > We could have the storage framework ask the storage plug-in if it is
> >> > managed. If it is, then the storage framework could send a message to
> >> the
> >> > hypervisor in question to create (let's talk Xen here) the SR ahead of
> >> > time. Then, when the storage framework next sends the attach-volume
> >> > command
> >> > to the hypervisor, it should work without changes to that attach-volume
> >> > logic (because - from the point of view of the attach logic - the SR is
> >> > already existent, as expected).
> >> >
> >> > Now, as Edison has pointed out, this limits the power of the storage
> >> > plug-in. A storage plug-in in this model cannot directly attach storage
> >> to
> >> > a VM (it must go through the hypervisor). Perhaps that is OK. We need
> to
> >> > make a call on that.
> >>
> >>
> >> Yes, I think we need to address this usage case, and won't limit the
> >> power of the storage plug-in.
> >> John, what's your idea to fit this usage case into your abstraction?
> >>
> >>
> >> >
> >> > If we want to give more power to the storage vendor, we would have to
> >> > have
> >> > the storage framework call into the storage plug-in to send the
> >> appropriate
> >> > attach commands to the hypervisor.
> >> >
> >> > Let's discuss and come to a consensus (or at least agree on a path) as
> >> soon
> >> > as we can. :)
> >> >
> >> > Thanks!
> >> >
> >> >
> >> >
> >> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >> >
> >> > > Mike,
> >> > >
> >> > > My understanding is the Xen expects an SR structure on all ISCSI
> >> devices
> >> > > -- at least that is how I read the code in your patch. Is my
> >> understanding
> >> > > correct? If so, the Xen plugin should be able to query the storage
> >> device
> >> > > to determine the presence of the SR structure and create it if it
> >> does not
> >> > > exist. Am I missing something in the implementation that makes that
> >> type
> >> > > of implementation impossible?
> >> > >
> >> > > Thanks,
> >> > > -John
> >> > >
> >> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >> > > wrote:
> >> > >
> >> > > > Yeah, if a storage vendor wanted to attach a volume directly to a
> >> VM in
> >> > > 4.2
> >> > > > today, it would probably fail because the attach-volume logic
> >> assumes
> >> > the
> >> > > > existence of the necessary hypervisor data structure (ex. SR on
> >> Xen).
> >> > > >
> >> > > > If we wanted to enable such an attach, we could do it the way
> Edison
> >> > > > suggests.
> >> > > >
> >> > > >
> >> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
> >> wrote:
> >> > > >
> >> > > >>
> >> > > >>
> >> > > >>> -----Original Message-----
> >> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >> > > >>> Sent: Friday, June 07, 2013 2:37 PM
> >> > > >>> To: dev@cloudstack.apache.org
> >> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >> > > >>>
> >> > > >>> As we only have three weeks until feature freeze, we should
> come
> >> to
> >> > a
> >> > > >>> consensus on this design point as soon as possible.
> >> > > >>>
> >> > > >>> Right now, if the storage framework asks my driver it is is
> >> managed, it
> >> > > >>> will say 'yes.' This means the framework will tell the driver to
> >> > > perform
> >> > > >>> its management activities. This then means the driver will call
> >> into
> >> > > the
> >> > > >>> host (it doesn't know which hypervisor, by the way) to perform
> the
> >> > > >> activity
> >> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> >> > > >>>
> >> > > >>> The driver doesn't know which hypervisor it's talking to, it just
> >> > > sends a
> >> > > >>> message to the host to perform the necessary pre-attach work.
> >> > > >>
> >> > > >> Could we just expose a method like
> "attachVolume/dettachVolume" on
> >> > the
> >> > > >> PrimaryDataStoreDriver driver?
> >> > > >> For most of cases, the implementation of each driver would just
> >> send
> >> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can
> put
> >> > the
> >> > > >> implementation in a base class, so that can be shared by all of
> >> this
> >> > > >> drivers), and in each hypervisor resource code, which may just call
> >> > > >> hypervisor's api to attach the volume to VM, while for certain
> >> storage,
> >> > > >> like SolidFire, may need to create a SR first, and create volume
> >> on it,
> >> > > >> then call hypervisor's API to attach volume to VM.
> >> > > >> While for some other storage vendor, may want to bypass
> hypervisor
> >> > > during
> >> > > >> attaching volume, so inside driver's attachvolume implementation,
> >> the
> >> > > >> driver can do some magic, such as, directly talk to an agent
> >> inside VM
> >> > > >> instance, then create a disk inside VM.
> >> > > >>
> >> > > >> How do you guys think?
> >> > > >>
> >> > > >>
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> >> > > wrote:
> >> > > >>>
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>> -----Original Message-----
> >> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> >> > > >>>>> To: dev@cloudstack.apache.org
> >> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >> > > >>>>>
> >> > > >>>>> Hi John,
> >> > > >>>>>
> >> > > >>>>> How's about this:
> >> > > >>>>>
> >> > > >>>>> The driver can implement an isManaged() method. The
> >> > > >>> VolumeManagerImpl
> >> > > >>>>> can
> >> > > >>>>> call into the driver to see if its managed. If it is, the
> >> > > >>>> VolumeManagerImpl
> >> > > >>>>> (which is responsible for calling into the hypervisor to attach
> >> the
> >> > > >> disk)
> >> > > >>>>> can call into the hypervisor to create the necessary hypervisor
> >> data
> >> > > >>>>> structure (ex. for XenServer, a storage repository).
> >> > > >>>>
> >> > > >>>> The problem here is that storage vendor may work differently
> with
> >> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while
> >> maybe
> >> > > >> other
> >> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> >> > directly
> >> > > >> to
> >> > > >>>> VM instance, see the discuss(
> >> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >> > > >>>
> >> >
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >> > > >>> %3E
> >> > > >>>> ).
> >> > > >>>> So I would let storage provider to implement attach disk to VM,
> >> > > >> instead of
> >> > > >>>> implemented by cloudstack itself.
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>>
> >> > > >>>>> If that's what you're going for, that works for me. By the way,
> >> > > >> Edison's
> >> > > >>>>> default storage plug-in (which handles the default storage
> >> behavior
> >> > > >> in
> >> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that
> >> talks to
> >> > > >>>>> hypervisors. You might want to contact him and inform him of
> >> your
> >> > > >>>> concerns
> >> > > >>>>> or that logic (as is) will make it to production.
> >> > > >>>>>
> >> > > >>>>> Please let me know if what I wrote in above (for my solution)
> >> is OK
> >> > > >> with
> >> > > >>>>> you. :)
> >> > > >>>>>
> >> > > >>>>> Thanks!
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
> >> jburwell@basho.com>
> >> > > >>> wrote:
> >> > > >>>>>
> >> > > >>>>>> Mike,
> >> > > >>>>>>
> >> > > >>>>>> Please see my responses in-line below.
> >> > > >>>>>>
> >> > > >>>>>> Thanks,
> >> > > >>>>>> -John
> >> > > >>>>>>
> >> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >> > > >>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>> wrote:
> >> > > >>>>>>
> >> > > >>>>>>> Hey John,
> >> > > >>>>>>>
> >> > > >>>>>>> I still have a bit more testing I'd like to do before I build
> >> up
> >> > > >> a
> >> > > >>>> patch
> >> > > >>>>>>> file, but this is the gist of what I've done:
> >> > > >>>>>>>
> >> > > >>>>>>> * During a volume-attach operation, after
> VolumeManagerImpl
> >> > tells
> >> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I
> have
> >> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the
> driver if
> >> it
> >> > > >>>>> managed.
> >> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> >> > > >> whatever
> >> > > >>>>>>> activity is required. In my case, this includes sending a
> >> > > >> message to
> >> > > >>>> the
> >> > > >>>>>>> host where the VM is running to have, say XenServer, add a
> >> > > >> storage
> >> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of
> the
> >> > > >> SAN
> >> > > >>>>>> volume,
> >> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
> >> can
> >> > > >> on
> >> > > >>>> the
> >> > > >>>>>>> storage repository). After this, the normal attach-volume
> >> > > >> message is
> >> > > >>>> sent
> >> > > >>>>>>> to the host by VolumeManagerImpl.
> >> > > >>>>>>
> >> > > >>>>>> There should be **no** code from a storage driver to a
> >> hypervisor.
> >> > > >> I
> >> > > >>>>>> apologize for the repetition, but we simply can not have
> >> hypervisor
> >> > > >>>>>> specific code in the storage layer. The circular dependencies
> >> > > >> between
> >> > > >>>> the
> >> > > >>>>>> two layers are not sustainable in the long term. Either the
> >> > > >>>> VirtualManager
> >> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >> > > >>> coordinate
> >> > > >>>>>> volume creation and then populating the SR. Ideally, we can
> >> > > >>>> generalize the
> >> > > >>>>>> process flow for attaching volumes such that the Xen
> hypervisor
> >> > > >> plugin
> >> > > >>>>>> would only implement callbacks to perform the attach action
> and
> >> > > >> create
> >> > > >>>>> the
> >> > > >>>>>> structure and SR. To my mind, the SolidFire driver should
> >> only be
> >> > > >>>>>> allocating space and providing information about contents
> (e.g.
> >> > > >> space
> >> > > >>>>>> available, space consumed, streams to a URI, file handle for a
> >> URI,
> >> > > >>>> etc)
> >> > > >>>>>> and capabilities.
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> * The reverse is performed for a detach-volume command.
> >> > > >>>>>>>
> >> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
> >> driver.
> >> > > >>>>>> Edison's
> >> > > >>>>>>> default driver simply does a "return false;". We could add a
> >> new
> >> > > >>>>>> parameter
> >> > > >>>>>>> to the createStoragePool API command, if we want, to
> remove
> >> > the
> >> > > >>>>>> hard-coded
> >> > > >>>>>>> return values in the drivers (although my driver will probably
> >> > > >> just
> >> > > >>>>>> ignore
> >> > > >>>>>>> this parameter and always return true since it wouldn't make
> >> > > >> sense
> >> > > >>>> for it
> >> > > >>>>>>> to ever return false). We'd need another column in the
> >> > > >> storage_pool
> >> > > >>>>> table
> >> > > >>>>>>> to store this value.
> >> > > >>>>>>
> >> > > >>>>>> Yes, I think we should have a parameter added to the
> >> > > >>> createStoragePool
> >> > > >>>>>> surfaced to the HTTP API that allows DataStores to be
> >> configured
> >> > > >> for
> >> > > >>>>>> management when their underlying drivers support it. To
> >> simplify
> >> > > >>>> things,
> >> > > >>>>>> this flag should only be mutable when the DataStore is
> >> created. It
> >> > > >>>> would be
> >> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged
> after
> >> > > >>>> creation.
> >> > > >>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> Sound like I'm in sync with what you were thinking?
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >> > > >>>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>> wrote:
> >> > > >>>>>>>
> >> > > >>>>>>>> I agree, John. Just wanted to point out that I have a
> working
> >> > > >> GUI
> >> > > >>>> for
> >> > > >>>>>> you
> >> > > >>>>>>>> to review (in that document), if you'd like to check it out.
> >> > > >>>>>>>>
> >> > > >>>>>>>> Thanks!
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >> > > >> jburwell@basho.com>
> >> > > >>>>>> wrote:
> >> > > >>>>>>>>
> >> > > >>>>>>>>> Mike,
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> I would like the UIs of two features reviewed together to
> >> > > >> ensure
> >> > > >>>>>>>>> consistency across the concepts of hypervisor throttled
> IOPs
> >> > > >> and
> >> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
> >> > > >>>> confusion,
> >> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
> >> > > >> help
> >> > > >>>>>>>>> minimize any potential confusion.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
> >> itself
> >> > > >> if
> >> > > >>>> it
> >> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> >> > > >>>> throttled
> >> > > >>>>>>>>> IOPs or storage provisioned IOPs.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> Thanks,
> >> > > >>>>>>>>> -John
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >> > > >>>>>>>>> <mi...@solidfire.com> wrote:
> >> > > >>>>>>>>>
> >> > > >>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I
> sent
> >> > to
> >> > > >>> you
> >> > > >>>>> the
> >> > > >>>>>>>>>> other day to see how the GUI looks for provisioned
> storage
> >> > > >> IOPS.
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> >> > > >> and we
> >> > > >>>>>>>>> decided
> >> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> >> > Offering
> >> > > >>>> dialog.
> >> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> >> > should
> >> > > >>> be
> >> > > >>>>> able
> >> > > >>>>>> to
> >> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only
> >> use
> >> > > >> Max
> >> > > >>>>>> IOPS).
> >> > > >>>>>>>>>> These fields are optional and can be ignored for storage
> >> that
> >> > > >>>> does not
> >> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
> >> the
> >> > > >>>> admin can
> >> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and
> Burst
> >> > > >> IOPS.
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature
> or
> >> > > >> mine,
> >> > > >>>> as
> >> > > >>>>>> is
> >> > > >>>>>>>>>> decided by the admin).
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> I'm not sure what we can do about these two features
> >> > > >> expressing
> >> > > >>>> the
> >> > > >>>>>>>>> speed
> >> > > >>>>>>>>>> in different terms. I've never seen a SAN express the
> IOPS
> >> for
> >> > > >>>> QoS in
> >> > > >>>>>>>>> any
> >> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
> >> read/write
> >> > > >>>> IOPS).
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> Thanks
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >> > > >>> <jb...@basho.com>
> >> > > >>>>>>>>> wrote:
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>> Wei,
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> >> > > >>> Storage/Hypervisor
> >> > > >>>>>> layer
> >> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior
> of
> >> > > >>>> hypervisor
> >> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> >> > > >> following
> >> > > >>>>>> issues
> >> > > >>>>>>>>>>> outstanding:
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >> > > >>> read/write
> >> > > >>>>>>>>> values
> >> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as
> total
> >> > > >> IOPS.
> >> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached
> to
> >> > > >>> storage
> >> > > >>>>>> volumes
> >> > > >>>>>>>>>>> with provisioned IOPS?
> >> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> >> > > >>>> provisioned
> >> > > >>>>>> IOPS
> >> > > >>>>>>>>>>> that will permit providers to discriminate these
> >> guaranteed
> >> > > >>>>>> operations
> >> > > >>>>>>>>> in
> >> > > >>>>>>>>>>> the event they want to bill for it differently?
> >> > > >>>>>>>>>>> 4. What is the user experience for throttled and
> >> provisioned
> >> > > >>> IOPS
> >> > > >>>>>> that
> >> > > >>>>>>>>>>> minimizes confusion of these concepts?
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> My thinking is that a VM can have either utilize
> >> hypervisor
> >> > > >>>> throttled
> >> > > >>>>>>>>> IOPS
> >> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> >> solve
> >> > > >>>> items 1
> >> > > >>>>>>>>> and 2.
> >> > > >>>>>>>>>>> Since the two facilities would not be permitted to
> operate
> >> > > >>>> together,
> >> > > >>>>>>>>> they
> >> > > >>>>>>>>>>> do not need to be semantically compatible. I think item
> 3
> >> > > >> can be
> >> > > >>>>>>>>> resolved
> >> > > >>>>>>>>>>> with an additional flag or two on the usage records. As
> >> for
> >> > > >>>> Item 4,
> >> > > >>>>>> I
> >> > > >>>>>>>>> am
> >> > > >>>>>>>>>>> not familiar with how these two enhancements are
> >> > depicted in
> >> > > >>> the
> >> > > >>>>> user
> >> > > >>>>>>>>>>> interface. I think we need to review the UI
> enhancements
> >> > for
> >> > > >>>> both
> >> > > >>>>>>>>>>> enhancements and ensure they are consistent.
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> Do these solutions make sense?
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>> -John
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >> > <us...@gmail.com>
> >> > > >>>>> wrote:
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>>> John and Mike,
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> >> > 2780/2729,
> >> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >> > > >>> CLOUDSTACK-
> >> > > >>>>> 2875 ) this
> >> > > >>>>>>>>> week.
> >> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow,
> and
> >> > > >> ask
> >> > > >>>> for
> >> > > >>>>>>>>> second
> >> > > >>>>>>>>>>>> merge request after testing.
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>> -Wei
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski
> <mi...@solidfire.com>
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
> >> John,
> >> > > >> and
> >> > > >>>>> have
> >> > > >>>>>>>>> been
> >> > > >>>>>>>>>>>>> re-working this section of code today.
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> Thanks for the comments,
> >> > > >>>>>>>>>>>>> Mike
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >> > > >>>>> <jb...@basho.com>
> >> > > >>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> See my responses in-line below.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Thanks for the response.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> >> > whether or
> >> > > >>>>> not a
> >> > > >>>>>>>>> Volume
> >> > > >>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> but an
> >> > > >>> actual
> >> > > >>>>>>>>> isManaged
> >> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> >> > > >> resources
> >> > > >>>> for
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>> if it is managed."
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> >> > method
> >> > > >>>>> added to
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> >> > > >> then the
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever
> name)
> >> > > >>> method
> >> > > >>>>> (which
> >> > > >>>>>>>>>>> would
> >> > > >>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as
> well)
> >> > and
> >> > > >>> this
> >> > > >>>>>> would
> >> > > >>>>>>>>>>> call
> >> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create,
> say
> >> > on
> >> > > >>>>>>>>> XenServer,
> >> > > >>>>>>>>>>> an
> >> > > >>>>>>>>>>>>> SR?
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >> > > >>>>> DataStoreDriver.
> >> > > >>>>>>>>> Since
> >> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary
> storage,
> >> > I
> >> > > >>>> would
> >> > > >>>>>> add
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> >> > straw
> >> > > >>> man
> >> > > >>>>> term
> >> > > >>>>>>>>> --
> >> > > >>>>>>>>>>>>>> something other than volume) methods to
> >> > > >>>>>>>>>>> allocate/create/deallocate/delete
> >> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a
> mutable
> >> > > >>> property
> >> > > >>>>> of
> >> > > >>>>>>>>>>>>> DataStore
> >> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> >> > > >>> DataStoreDriver
> >> > > >>>>> can be
> >> > > >>>>>>>>>>>>> managed.
> >> > > >>>>>>>>>>>>>> This approach allows operators to override
> >> > manageability
> >> > > >> of
> >> > > >>>>>> devices.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the
> Xen
> >> > > >> plugin
> >> > > >>>>>> would
> >> > > >>>>>>>>> be
> >> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume
> methods
> >> > to
> >> > > >>>>> create any
> >> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There
> >> should
> >> > > >> be no
> >> > > >>>>>>>>>>>>> dependencies
> >> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> >> > > >> earlier,
> >> > > >>>> such
> >> > > >>>>>>>>>>>>> circular
> >> > > >>>>>>>>>>>>>> dependencies will lead to a tangled,
> unmaintainable
> >> > mess.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with
> you.
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Thanks again, John
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >> > > >>>>> <jb...@basho.com>
> >> > > >>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage
> layer
> >> > that
> >> > > >>>>>> supports n
> >> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> >> > > >> hypervisors.
> >> > > >>>>>> Such
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and
> untestable
> >> > > >>> beast.
> >> > > >>>>>>>>>>>>> Therefore, my
> >> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to
> evolve
> >> > the
> >> > > >>>>> Storage
> >> > > >>>>>> layer
> >> > > >>>>>>>>>>>>> towards
> >> > > >>>>>>>>>>>>>>> this separation.
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> >> > whether
> >> > > >>> or
> >> > > >>>>> not a
> >> > > >>>>>>>>> Volume
> >> > > >>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType,
> but
> >> > an
> >> > > >>> actual
> >> > > >>>>>>>>>>> isManaged
> >> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >> > > >> resources
> >> > > >>>> for
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
> >> needs
> >> > > >> to
> >> > > >>>>>> indicate
> >> > > >>>>>>>>>>>>> whether
> >> > > >>>>>>>>>>>>>>> or not it supports management operations.
> Finally, I
> >> > > >> think
> >> > > >>>> we
> >> > > >>>>>>>>> need to
> >> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to
> >> have
> >> > > >>>>>> something
> >> > > >>>>>>>>>>>>> that is
> >> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is
> capable
> >> > > >>> managing
> >> > > >>>>> the
> >> > > >>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> >> > unmanaged).
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be
> done
> >> > in
> >> > > >>> the
> >> > > >>>>> Xen
> >> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> >> > Storage
> >> > > >>>>> layer to
> >> > > >>>>>>>>>>> perform
> >> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories,
> create a
> >> > > >> file,
> >> > > >>>> etc).
> >> > > >>>>>>>>>>> This
> >> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >> > > >>>> hypervisor, as
> >> > > >>>>>>>>> such,
> >> > > >>>>>>>>>>>>>>> should be confined to its implementation. From
> my
> >> > > >>>>> perspective,
> >> > > >>>>>>>>>>> nothing
> >> > > >>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with
> content.
> >> > From
> >> > > >>> its
> >> > > >>>>>>>>>>>>> perspective,
> >> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the
> means
> >> > to
> >> > > >>>> query
> >> > > >>>>>> the
> >> > > >>>>>>>>>>> data
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>> support the interpretation of the content by
> higher-
> >> > level
> >> > > >>>>> layers
> >> > > >>>>>>>>> (e.g.
> >> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> >> > composition
> >> > > >>> of
> >> > > >>>>>>>>>>> operations
> >> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> >> > Volume
> >> > > >>>>> storage
> >> > > >>>>>>>>>>> protocol
> >> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> >> > implemented
> >> > > >> in
> >> > > >>>> my
> >> > > >>>>>>>>> patch,
> >> > > >>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>> could do the following:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >> > > >>>>>>>>> VolumeManagerImpl
> >> > > >>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to
> the
> >> > > >>>>> hypervisor.
> >> > > >>>>>> Before
> >> > > >>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> >> > dealing
> >> > > >>> with
> >> > > >>>>>>>>> Dynamic
> >> > > >>>>>>>>>>> (or
> >> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> >> send a
> >> > > >>>> "Create
> >> > > >>>>>>>>> SR"
> >> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> >> > would
> >> > > >>> then
> >> > > >>>>>>>>> proceed
> >> > > >>>>>>>>>>> to
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and
> we just
> >> > > >> add
> >> > > >>>>>> another
> >> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
> >> > > >> particular
> >> > > >>>>> type
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>>>> storage.
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> What do you think?
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> Hey John,
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
> >> > outlines
> >> > > >>> how
> >> > > >>>>> the
> >> > > >>>>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably
> be
> >> of
> >> > > >> use
> >> > > >>>> to
> >> > > >>>>>>>>> you, as
> >> > > >>>>>>>>>>>>> well,
> >> > > >>>>>>>>>>>>>>>> as you perform the code review.
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> I have shared this document with you (you
> should
> >> > have
> >> > > >>>>> received
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> Talk to you later!
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >> > > >>>>>> jburwell@basho.com
> >> > > >>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> >> > thoughts,
> >> > > >>> but
> >> > > >>>>>> want
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
> >> solidfire,
> >> > > >>>>>>>>>>>>> disk_io_throttle,
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them
> in a
> >> > > >>>>> coherent
> >> > > >>>>>>>>> form
> >> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin
> introduces
> >> > the
> >> > > >>>> notion
> >> > > >>>>>> of
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just
> introduces
> >> > > >> the
> >> > > >>>>> notion
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the
> plug-in
> >> > was
> >> > > >>>>>>>>> incomplete,
> >> > > >>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>> I had
> >> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> >> > > >> device.
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been
> spending on
> >> > > >> this.
> >> > > >>>> :)
> >> > > >>>>> Do
> >> > > >>>>>>>>> you
> >> > > >>>>>>>>>>>>>>>>>> have a
> >> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should
> accomplish
> >> > > >>> what
> >> > > >>>>> you're
> >> > > >>>>>>>>> looking
> >> > > >>>>>>>>>>>>>>>>>> for?
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell
> <
> >> > > >>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin
> introduces
> >> > the
> >> > > >>>> notion
> >> > > >>>>>> of
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the
> management
> >> > > >>>>> capabilities
> >> > > >>>>>>>>> (i.e.
> >> > > >>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want
> it
> >> > > >>> managed
> >> > > >>>>> by
> >> > > >>>>>>>>> CloudStack)
> >> > > >>>>>>>>>>>>>>>>>> from the
> >> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up
> with
> >> > a
> >> > > >>>>> semantic
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>> will
> >> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be
> managed.
> >> > I
> >> > > >>> also
> >> > > >>>>> want
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types
> from
> >> > the
> >> > > >>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> definitions.
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski
> <
> >> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John
> Burwell <
> >> > > >>>>>>>>>>> jburwell@basho.com
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> >> > questions
> >> > > >>> for
> >> > > >>>>> me
> >> > > >>>>>> with
> >> > > >>>>>>>>>>>>>>>>>> regards to
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI
> LUNs
> >> > and
> >> > > >>> SAN
> >> > > >>>>>> volumes?
> >> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single
> LUN
> >> > (LUN
> >> > > >>> 0).
> >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN
> Volumes
> >> > and
> >> > > >>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> Volumes?
> >> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the
> management
> >> > > >>> server or
> >> > > >>>>>>>>> externally
> >> > > >>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>> operator?
> >> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a
> SAN
> >> > > >>> volume
> >> > > >>>>> (same
> >> > > >>>>>> as a
> >> > > >>>>>>>>>>> SAN
> >> > > >>>>>>>>>>>>>>>>>> LUN) is
> >> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via
> the
> >> > plug-in)
> >> > > >>>>> the
> >> > > >>>>>> first
> >> > > >>>>>>>>>>> time
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a
> hypervisor.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-
> in,
> >> > but
> >> > > >>>> still
> >> > > >>>>>>>>> want
> >> > > >>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> use a
> >> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this
> already
> >> > > >>> today
> >> > > >>>>>> (prior
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> 4.2). The
> >> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume
> and -
> >> > in
> >> > > >>> this
> >> > > >>>>> case -
> >> > > >>>>>>>>>>>>>>>>>> multiple VMs
> >> > > >>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume.
> While
> >> > you
> >> > > >>> can do
> >> > > >>>>> this
> >> > > >>>>>>>>>>>>> today,
> >> > > >>>>>>>>>>>>>>>>>> it is
> >> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage
> QoS.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the
> management
> >> > server
> >> > > >>> or
> >> > > >>>>> externally
> >> > > >>>>>>>>> by
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> operator?
> >> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the
> SAN
> >> > > >>> volumes
> >> > > >>>>> are
> >> > > >>>>>>>>>>>>> completely
> >> > > >>>>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-
> in).
> >> > There
> >> > > >>> is
> >> > > >>>>> no
> >> > > >>>>>> admin
> >> > > >>>>>>>>>>>>>>>>>>>> interaction.
> >> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a
> SAN
> >> > > >>> volume
> >> > > >>>>> and a
> >> > > >>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>> volume,
> >> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor
> that
> >> > > >>>>> supports
> >> > > >>>>>> true
> >> > > >>>>>>>>>>> QoS.
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces
> are
> >> > > >> related
> >> > > >>>>> and
> >> > > >>>>>>>>>>>>> expected
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> operate.
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike
> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> >> > multiple
> >> > > >>> VMs
> >> > > >>>>> with
> >> > > >>>>>>>>>>>>>>>>>> provisioned
> >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying
> storage."
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to
> map a
> >> > > >>> single
> >> > > >>>>> SAN
> >> > > >>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>> (LUN) to a
> >> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot
> have
> >> > > >>> multiple
> >> > > >>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> >> > guarantee
> >> > > >>> QoS.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> >> > volume
> >> > > >>> house
> >> > > >>>>> more
> >> > > >>>>>>>>> than
> >> > > >>>>>>>>>>>>> one
> >> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that
> today
> >> > > >>> without
> >> > > >>>>> any of
> >> > > >>>>>> my
> >> > > >>>>>>>>>>>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>>>> code.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> >> > Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> >> > volume
> >> > > >>> for
> >> > > >>>>>>>>>>> CloudStack's
> >> > > >>>>>>>>>>>>>>>>>> use
> >> > > >>>>>>>>>>>>>>>>>>>> onto
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be
> created."
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> >> > Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> >> > concerns me
> >> > > >>> --
> >> > > >>>>>> concepts
> >> > > >>>>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage
> layer.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> >> > Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type
> is
> >> > today
> >> > > >>>> only
> >> > > >>>>>> used
> >> > > >>>>>>>>>>> (as
> >> > > >>>>>>>>>>>>>>>>>> far as
> >> > > >>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you
> have
> >> > not
> >> > > >>>>> PreSetup an
> >> > > >>>>>>>>> SR).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> volume
> >> > from
> >> > > >>>>> VMware, it
> >> > > >>>>>>>>> uses
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> vmfs
> >> > > >>>>>>>>>>>>>>>>>>>>>> type.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI
> volume
> >> > from
> >> > > >>> KVM,
> >> > > >>>>> it uses
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's
> thinking
> >> > > >>> here
> >> > > >>>>> was to
> >> > > >>>>>>>>> make a
> >> > > >>>>>>>>>>>>>>>>>> new type
> >> > > >>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic
> ability
> >> > > >>> Edison
> >> > > >>>>> added
> >> > > >>>>>>>>> into
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be
> more
> >> > > >>> specificy,
> >> > > >>>>> though:
> >> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM,
> Mike
> >> > > >>> Tutkowski
> >> > > >>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself
> shouldn't
> >> > know
> >> > > >>> or
> >> > > >>>>> care
> >> > > >>>>>> that
> >> > > >>>>>>>>> it
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>> being
> >> > > >>>>>>>>>>>>>>>>>>>>>> used
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to
> answer
> >> > > >>>>> questions
> >> > > >>>>>>>>> about
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that
> the
> >> > > >>>>> SolidFire
> >> > > >>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>> needs to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that
> the
> >> > > >>> hypervisor
> >> > > >>>>> logic
> >> > > >>>>>>>>>>> knows
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>> treat
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> >> > constraint
> >> > > >>>>> and just
> >> > > >>>>>>>>>>>>>>>>>> calling it
> >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a
> way
> >> > for
> >> > > >>> the
> >> > > >>>>> hypervisor
> >> > > >>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> >> > perform
> >> > > >>> the
> >> > > >>>>> necessary
> >> > > >>>>>>>>>>>>>>>>>> activities.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM,
> John
> >> > > >>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> >> > > >>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may
> be
> >> > able to
> >> > > >>>>> manage a
> >> > > >>>>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> >> > > >>> CloudStack
> >> > > >>>>> (e.g. the
> >> > > >>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>> shared
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the
> operator has
> >> > > >>> chosen
> >> > > >>>>> to
> >> > > >>>>>>>>> dedicate
> >> > > >>>>>>>>>>>>>>>>>> only a
> >> > > >>>>>>>>>>>>>>>>>>>>>> portion
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my
> reasoning
> >> > > >>> make
> >> > > >>>>> sense?"
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how
> creating a
> >> > SAN
> >> > > >>>>> volume via
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the
> hypervisor)
> >> > would
> >> > > >>>>> work unless
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of,
> say,
> >> > an
> >> > > >>> SR.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent
> of
> >> > > >>>>> CloudStack, an
> >> > > >>>>>>>>>>>>> operator
> >> > > >>>>>>>>>>>>>>>>>>>>>> allocates
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack,
> and
> >> > > >>> exposes it
> >> > > >>>>>> through
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> LUN. They
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN
> over
> >> > to
> >> > > >>>>> CloudStack,
> >> > > >>>>>>>>> but
> >> > > >>>>>>>>>>>>>>>>>> not allow
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore
> LUNs.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior
> to my
> >> > > >>>>> changes, we
> >> > > >>>>>>>>> would
> >> > > >>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>> passing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have
> the
> >> > > >>>>> necessary
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>> (like
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should
> maybe
> >> > have
> >> > > >>> the
> >> > > >>>>> storage
> >> > > >>>>>>>>>>>>> framework
> >> > > >>>>>>>>>>>>>>>>>>>> itself
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> >> > support
> >> > > >>> from
> >> > > >>>>> the
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> side and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code
> specifically
> >> > to
> >> > > >>> create
> >> > > >>>>> the
> >> > > >>>>>>>>> SR
> >> > > >>>>>>>>>>>>>>>>>> before the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor
> management
> >> > plugin
> >> > > >>>>> should
> >> > > >>>>>> have a
> >> > > >>>>>>>>>>>>> rich
> >> > > >>>>>>>>>>>>>>>>>> enough
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to
> determine
> >> > > >>> available
> >> > > >>>>> for
> >> > > >>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>> storage.
> >> > > >>>>>>>>>>>>>>>>>>>> For
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >> > > >>> interrogation of
> >> > > >>>>> the
> >> > > >>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage
> device
> >> > itself
> >> > > >>>>> shouldn't
> >> > > >>>>>>>>> know
> >> > > >>>>>>>>>>>>>>>>>> or care
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR --
> simply be
> >> > able
> >> > > >>> to
> >> > > >>>>>> answer
> >> > > >>>>>>>>>>>>>>>>>> questions
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM,
> Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the
> SolidFire
> >> > > >>> driver
> >> > > >>>>> (which
> >> > > >>>>>>>>> is a
> >> > > >>>>>>>>>>>>>>>>>> type of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> >> > done, a
> >> > > >>> new
> >> > > >>>>> Primary
> >> > > >>>>>>>>>>>>>>>>>> Storage shows
> >> > > >>>>>>>>>>>>>>>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk
> Offering
> >> > > >>> that
> >> > > >>>>>> references
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>> tag
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary
> Storage.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> >> > CloudStack
> >> > > >>>>> volume. This
> >> > > >>>>>>>>> leads
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> a new
> >> > > >>>>>>>>>>>>>>>>>>>>>> row
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> >> > CloudStack
> >> > > >>>>> volume to a
> >> > > >>>>>>>>> VM
> >> > > >>>>>>>>>>>>>>>>>> (attach
> >> > > >>>>>>>>>>>>>>>>>>>>>> disk).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage
> framework
> >> > calling
> >> > > >>> the
> >> > > >>>>> plug-in
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> create a
> >> > > >>>>>>>>>>>>>>>>>>>> new
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my
> case, a
> >> > > >>> SAN).
> >> > > >>>>> The
> >> > > >>>>>>>>> plug-in
> >> > > >>>>>>>>>>>>>>>>>> also
> >> > > >>>>>>>>>>>>>>>>>>>>>> updates
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with
> applicable
> >> > data
> >> > > >>>>> (like the
> >> > > >>>>>>>>> IQN of
> >> > > >>>>>>>>>>>>>>>>>> the SAN
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only
> invoked if
> >> > the
> >> > > >>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> volume is in
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the
> attach,
> >> > the
> >> > > >>>>> volume
> >> > > >>>>>>>>> will be
> >> > > >>>>>>>>>>>>>>>>>> in the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk)
> and
> >> > the
> >> > > >>>>> plug-in
> >> > > >>>>>>>>> code
> >> > > >>>>>>>>>>>>>>>>>> will not
> >> > > >>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN
> volume.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic
> is run
> >> > and
> >> > > >>>>> detects
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in
> the
> >> > form
> >> > > >>> of a
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54
> PM,
> >> > Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in
> sync on
> >> > > >>>>> terminology,
> >> > > >>>>>>>>> volume,
> >> > > >>>>>>>>>>>>>>>>>> in these
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the
> physical
> >> > > >>>>> allocation on
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I
> try to
> >> > > >>> mean
> >> > > >>>>> 'SAN
> >> > > >>>>>>>>> volume'.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the
> end
> >> > user
> >> > > >>> can
> >> > > >>>>> make in
> >> > > >>>>>>>>>>>>>>>>>> CloudStack, I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50
> PM,
> >> > Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> >> > wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may
> very
> >> > well
> >> > > >>> make
> >> > > >>>>> sense, but
> >> > > >>>>>>>>> I'm
> >> > > >>>>>>>>>>>>>>>>>> having a
> >> > > >>>>>>>>>>>>>>>>>>>>>> hard
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw
> Edison
> >> > in on
> >> > > >>>>> this
> >> > > >>>>>>>>> conversation
> >> > > >>>>>>>>>>>>>>>>>> as he
> >> > > >>>>>>>>>>>>>>>>>>>> was
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> >> > > >>> approach I
> >> > > >>>>> took.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42
> PM,
> >> > > >>> John
> >> > > >>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are
> combining
> >> > two
> >> > > >>>>> distinct
> >> > > >>>>>>>>> concepts
> >> > > >>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> >> > protocols.
> >> > > >>> In
> >> > > >>>>> both
> >> > > >>>>>>>>> cases, we
> >> > > >>>>>>>>>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> >> > system to
> >> > > >>>>>> create/delete
> >> > > >>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other
> requires
> >> > the
> >> > > >>>>> volume to
> >> > > >>>>>> be
> >> > > >>>>>>>>>>>>>>>>>> volume to be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> >> > context.
> >> > > >>> To
> >> > > >>>>> ensure
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>>>>> are in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in
> these
> >> > > >>>>> definitions,
> >> > > >>>>>>>>> refers to
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>> physical
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> >> > correct?
> >> > > >>>>> Minimally,
> >> > > >>>>>> we
> >> > > >>>>>>>>>>>>> must
> >> > > >>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>> able
> >> > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a
> storage
> >> > device
> >> > > >>> to
> >> > > >>>>> move bits
> >> > > >>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>> one place
> >> > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >> > > >>> Optionally, a
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>> may
> >> > > >>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> >> > Therefore,
> >> > > >>>>> we can have
> >> > > >>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> unmanaged
> >> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen
> SR,
> >> > and
> >> > > >>> we
> >> > > >>>>> can have a
> >> > > >>>>>>>>>>>>> managed
> >> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which
> CloudStack is
> >> > > >>> capable
> >> > > >>>>> of
> >> > > >>>>>>>>> allocating
> >> > > >>>>>>>>>>>>>>>>>> LUNs and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> >> > CloudStack
> >> > > >>> may
> >> > > >>>>> be able
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> manage a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to
> leave it
> >> > > >>>>> unmanaged by
> >> > > >>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>> (e.g.
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services,
> and
> >> > the
> >> > > >>>>> operator has
> >> > > >>>>>>>>> chosen
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to
> CloudStack).
> >> > Does
> >> > > >>> my
> >> > > >>>>>> reasoning
> >> > > >>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>> sense?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts
> above
> >> > are
> >> > > >>>>> reasonable, it
> >> > > >>>>>>>>> seems
> >> > > >>>>>>>>>>>>>>>>>>>> appropriate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> >> > concerns
> >> > > >>> from
> >> > > >>>>>>>>> StoragePoolType,
> >> > > >>>>>>>>>>>>>>>>>> add the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> >> > attached
> >> > > >>> driver
> >> > > >>>>> that
> >> > > >>>>>>>>>>> indicates
> >> > > >>>>>>>>>>>>>>>>>> whether
> >> > > >>>>>>>>>>>>>>>>>>>>>> or
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack,
> and
> >> > > >>> establish
> >> > > >>>>> a
> >> > > >>>>>>>>> abstraction
> >> > > >>>>>>>>>>>>>>>>>>>>>> representing a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device
> separate
> >> > that
> >> > > >>> is
> >> > > >>>>>>>>> associated
> >> > > >>>>>>>>>>>>>>>>>> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>> volume.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> >> > hypervisor
> >> > > >>>>> drivers can
> >> > > >>>>>>>>>>>>> declare
> >> > > >>>>>>>>>>>>>>>>>> which
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> >> > encounter
> >> > > >>> a
> >> > > >>>>> device
> >> > > >>>>>> managed
> >> > > >>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> >> > exposed
> >> > > >>> by
> >> > > >>>>> the driver to
> >> > > >>>>>>>>>>>>> automate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts
> make
> >> > > >>> sense,
> >> > > >>>>> then we can
> >> > > >>>>>>>>> sit
> >> > > >>>>>>>>>>>>>>>>>> down and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> >> > Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> >> > > >>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> >> > between
> >> > > >>> the
> >> > > >>>>> current
> >> > > >>>>>> iSCSI
> >> > > >>>>>>>>>>>>> type
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin
> has to
> >> > go in
> >> > > >>> and
> >> > > >>>>> create
> >> > > >>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> Primary
> >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this
> point in
> >> > time,
> >> > > >>>>> the
> >> > > >>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>> volume must
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> >> > allocated).
> >> > > >>>>> Future
> >> > > >>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that
> was
> >> > created
> >> > > >>>>> behind the
> >> > > >>>>>>>>>>>>> scenes.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin
> has
> >> > to
> >> > > >>> go in
> >> > > >>>>> and
> >> > > >>>>>> create
> >> > > >>>>>>>>>>>>>>>>>> Primary
> >> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create
> and
> >> > delete
> >> > > >>>>> volumes on
> >> > > >>>>>>>>> its
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is
> enabled via
> >> > the
> >> > > >>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> framework). When
> >> > > >>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack
> volume
> >> > that
> >> > > >>> was
> >> > > >>>>> created,
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> framework
> >> > > >>>>>>>>>>>>>>>>>>>>>> tells
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> >> > volume.
> >> > > >>> After
> >> > > >>>>> this is
> >> > > >>>>>>>>> done,
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in
> question is
> >> > called.
> >> > > >>>>> No
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> >> > volume
> >> > > >>> was
> >> > > >>>>> just
> >> > > >>>>>>>>> created.
> >> > > >>>>>>>>>>>>> The
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be
> created.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 3:21
> >> > PM,
> >> > > >>> Mike
> >> > > >>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms,
> so I
> >> > should
> >> > > >>>>> probably
> >> > > >>>>>> have
> >> > > >>>>>>>>>>>>>>>>>> defined them
> >> > > >>>>>>>>>>>>>>>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage
> that is
> >> > > >>> pre-
> >> > > >>>>> allocated
> >> > > >>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>> an admin
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then
> a
> >> > > >>> hypervisor
> >> > > >>>>> data
> >> > > >>>>>>>>> structure
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex.
> XenServer
> >> > SR),
> >> > > >>> then
> >> > > >>>>> that
> >> > > >>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by
> CloudStack.
> >> > Disks
> >> > > >>> (VDI)
> >> > > >>>>> are later
> >> > > >>>>>>>>>>>>> placed
> >> > > >>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as
> needed. In
> >> > > >>> these
> >> > > >>>>> cases, the
> >> > > >>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> structure is
> >> > > >>> already
> >> > > >>>>> in place
> >> > > >>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> simply
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> >> > structure
> >> > > >>> to
> >> > > >>>>> the VM in
> >> > > >>>>>>>>>>>>>>>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage:
> Storage
> >> > that is
> >> > > >>> not
> >> > > >>>>>>>>>>> pre-allocated.
> >> > > >>>>>>>>>>>>>>>>>> Instead
> >> > > >>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage,
> this
> >> > could
> >> > > >>> be a
> >> > > >>>>> SAN
> >> > > >>>>>> (not
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> volume on
> >> > > >>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The
> hypervisor
> >> > data
> >> > > >>>>> structure
> >> > > >>>>>>>>> must be
> >> > > >>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> >> > because
> >> > > >>> these
> >> > > >>>>> types of
> >> > > >>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>> have not
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor
> data
> >> > > >>>>> structure by an
> >> > > >>>>>>>>>>> admin.
> >> > > >>>>>>>>>>>>>>>>>> Once
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> >> > XenServer
> >> > > >>> for
> >> > > >>>>> this
> >> > > >>>>>>>>> volume,
> >> > > >>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>> attaches
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR
> to the
> >> > VM
> >> > > >>> in
> >> > > >>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 3:13
> >> > PM,
> >> > > >>>>> John Burwell
> >> > > >>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> >> > implementation of
> >> > > >>> the
> >> > > >>>>> Dynamic
> >> > > >>>>>> type
> >> > > >>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>> behavior
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI
> which
> >> > why I
> >> > > >>> ask
> >> > > >>>>> about the
> >> > > >>>>>>>>>>>>>>>>>> difference.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question --
> what is
> >> > the
> >> > > >>>>> definition
> >> > > >>>>>> of
> >> > > >>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>> Dynamic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10
> PM,
> >> > > >>> Mike
> >> > > >>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the
> iSCSI
> >> > type
> >> > > >>> is
> >> > > >>>>>> uniquely
> >> > > >>>>>>>>>>> used
> >> > > >>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up
> Primary
> >> > > >>> Storage
> >> > > >>>>> that is
> >> > > >>>>>>>>> directly
> >> > > >>>>>>>>>>>>>>>>>> based on
> >> > > >>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to
> skip the
> >> > > >>> step of
> >> > > >>>>> going
> >> > > >>>>>> to
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository
> based
> >> > on
> >> > > >>> that
> >> > > >>>>> iSCSI
> >> > > >>>>>>>>> target as
> >> > > >>>>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is
> only
> >> > > >>>>> supported for
> >> > > >>>>>>>>>>>>>>>>>> XenServer.
> >> > > >>>>>>>>>>>>>>>>>>>> For
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you
> must
> >> > first go
> >> > > >>> to
> >> > > >>>>> the
> >> > > >>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>> perform
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know
> what
> >> > RBD is.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> >> > 2:13
> >> > > >>> PM,
> >> > > >>>>> John
> >> > > >>>>>> Burwell
> >> > > >>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through
> the
> >> > code,
> >> > > >>> what
> >> > > >>>>> is the
> >> > > >>>>>>>>>>> difference
> >> > > >>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types?
> Why
> >> > isn't
> >> > > >>> RBD
> >> > > >>>>> considered
> >> > > >>>>>>>>>>>>> Dynamic?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> 3:46
> >> > PM,
> >> > > >>> Mike
> >> > > >>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> >> > storage is
> >> > > >>>>> defined in
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> >> > > >>>>> StoragePoolType {
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false),
> //
> >> > local
> >> > > >>>>> directory
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > NetworkFilesystem(true),
> >> > > >>> //
> >> > > >>>>> NFS or CIFS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> >> > shared
> >> > > >>> LUN,
> >> > > >>>>> with a
> >> > > >>>>>>>>>>> clusterfs
> >> > > >>>>>>>>>>>>>>>>>>>> overlay
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for
> e.g.,
> >> > ZFS
> >> > > >>>>> Comstar
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for
> iso
> >> > image
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> >> > XenServer
> >> > > >>> local
> >> > > >>>>> LVM SR
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > SharedMountPoint(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> >> > VMware
> >> > > >>> VMFS
> >> > > >>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), //
> for
> >> > > >>>>> XenServer, Storage
> >> > > >>>>>>>>> Pool
> >> > > >>>>>>>>>>>>>>>>>> is set
> >> > > >>>>>>>>>>>>>>>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> >> > XenServer
> >> > > >>> local
> >> > > >>>>> EXT SR
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> >> > dynamic,
> >> > > >>>>> zone-wide
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > StoragePoolType(boolean
> >> > > >>>>> shared) {
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared =
> shared;
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> >> > isShared() {
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013 at
> >> > 1:41
> >> > > >>> PM,
> >> > > >>>>> Mike
> >> > > >>>>>>>>> Tutkowski
> >> > > >>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example,
> let's say
> >> > > >>> another
> >> > > >>>>> storage
> >> > > >>>>>>>>>>> company
> >> > > >>>>>>>>>>>>>>>>>> wants
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to
> leverage its
> >> > > >>> Quality
> >> > > >>>>> of
> >> > > >>>>>> Service
> >> > > >>>>>>>>>>>>>>>>>> feature. It
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> storage, as
> >> > well.
> >> > > >>>>> They would
> >> > > >>>>>>>>> need
> >> > > >>>>>>>>>>>>> only
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've
> made
> >> > the
> >> > > >>>>> necessary
> >> > > >>>>>>>>> changes to
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their
> plug-
> >> > in.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> >> > at
> >> > > >>> 1:39
> >> > > >>>>> PM, Mike
> >> > > >>>>>>>>>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to
> imply
> >> > the
> >> > > >>>>> XenServer code
> >> > > >>>>>> is
> >> > > >>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> >> > attach
> >> > > >>> logic is
> >> > > >>>>> now aware
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>> dynamic,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is
> an
> >> > > >>>>> implementation of
> >> > > >>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>> kind of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new
> to 4.2
> >> > > >>> with
> >> > > >>>>> Edison's
> >> > > >>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> framework
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created
> a
> >> > new
> >> > > >>>>> framework that
> >> > > >>>>>>>>>>> supported
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> >> > dynamically.
> >> > > >>>>> However, when I
> >> > > >>>>>>>>>>>>>>>>>> visited with
> >> > > >>>>>>>>>>>>>>>>>>>>>> him
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we
> realized
> >> > that
> >> > > >>> it
> >> > > >>>>> was not
> >> > > >>>>>>>>>>>>>>>>>> complete. We
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing
> CloudStack
> >> > could
> >> > > >>> do
> >> > > >>>>> with these
> >> > > >>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>> unless
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> >> > recognize
> >> > > >>> this
> >> > > >>>>> new type of
> >> > > >>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>> create
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >> > structure.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> >> > at
> >> > > >>> 1:28
> >> > > >>>>> PM, John
> >> > > >>>>>>>>> Burwell
> >> > > >>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally
> odd
> >> > to me
> >> > > >>>>> that any
> >> > > >>>>>>>>>>> operation
> >> > > >>>>>>>>>>>>>>>>>> in the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or
> care
> >> > > >>> about
> >> > > >>>>> details. I
> >> > > >>>>>>>>> expect
> >> > > >>>>>>>>>>>>>>>>>> to see
> >> > > >>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set
> of
> >> > > >>> operations
> >> > > >>>>> that can be
> >> > > >>>>>>>>>>>>>>>>>>>> composed/driven
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> implementations to
> >> > > >>>>> allocate
> >> > > >>>>>> space/create
> >> > > >>>>>>>>>>>>>>>>>> structures
> >> > > >>>>>>>>>>>>>>>>>>>>>> per
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert
> this
> >> > > >>>>> dependency, we are
> >> > > >>>>>>>>> going
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> end
> >> > > >>>>>>>>>>>>>>>>>>>>>> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that
> will
> >> > make
> >> > > >>> the
> >> > > >>>>> system
> >> > > >>>>>>>>>>>>> increasingly
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am
> I
> >> > > >>>>> understanding that the
> >> > > >>>>>>>>> Xen
> >> > > >>>>>>>>>>>>>>>>>> specific
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >> > > >>>>> CitrixResourceBase
> >> > > >>>>>> class?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> > 2013 at
> >> > > >>> 3:12
> >> > > >>>>> PM, Mike
> >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into
> this
> >> > in a
> >> > > >>> bit
> >> > > >>>>> more
> >> > > >>>>>>>>> detail:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2
> and
> >> > aside
> >> > > >>> from
> >> > > >>>>> one setup
> >> > > >>>>>>>>>>>>> method
> >> > > >>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create
> a
> >> > > >>> volume on
> >> > > >>>>> the
> >> > > >>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> system,
> >> > > >>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>> go
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a
> data
> >> > > >>> structure
> >> > > >>>>> to make
> >> > > >>>>>> use
> >> > > >>>>>>>>> of
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> >> > > >>> XenServer
> >> > > >>>>> or a
> >> > > >>>>>> datastore
> >> > > >>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>> ESX(i)).
> >> > > >>>>>>>>>>>>>>>>>>>>>> VMs
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> >> > storage
> >> > > >>>>> system's volume.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's
> new
> >> > > >>> storage
> >> > > >>>>> framework,
> >> > > >>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>> need
> >> > > >>>>>>>>>>>>>>>>>>>> no
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can
> easily
> >> > > >>> create
> >> > > >>>>> a 1:1
> >> > > >>>>>>>>>>>>> relationship
> >> > > >>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and
> the
> >> > VM's
> >> > > >>> data
> >> > > >>>>> disk
> >> > > >>>>>>>>> (necessary
> >> > > >>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now
> write
> >> > a
> >> > > >>> plug-
> >> > > >>>>> in that is
> >> > > >>>>>>>>> called
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> >> > needed.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem
> that
> >> > the
> >> > > >>>>> storage
> >> > > >>>>>> framework
> >> > > >>>>>>>>> did
> >> > > >>>>>>>>>>>>>>>>>> not
> >> > > >>>>>>>>>>>>>>>>>>>>>> address
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >> > > >>> hypervisor-
> >> > > >>>>> specific data
> >> > > >>>>>>>>>>>>>>>>>> structure
> >> > > >>>>>>>>>>>>>>>>>>>> when
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> attach/detach.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being
> the
> >> > case,
> >> > > >>> I've
> >> > > >>>>> been
> >> > > >>>>>>>>> enhancing
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> to do
> >> > > >>>>>>>>>>>>>>>>>>>> so.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out
> and
> >> > > >>>>> submitted. I've got
> >> > > >>>>>>>>> ESX(i)
> >> > > >>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> my
> >> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend
> the
> >> > 4.2
> >> > > >>>>> freeze date.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> help a
> >> > bit? :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> 3,
> >> > 2013 at
> >> > > >>>>> 1:03 PM, Mike
> >> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> >> > plug-in -
> >> > > >>> by
> >> > > >>>>> itself - is
> >> > > >>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is
> with
> >> > the
> >> > > >>>>> volume-attach
> >> > > >>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>> (in the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework
> calls
> >> > into
> >> > > >>> the
> >> > > >>>>> plug-in to
> >> > > >>>>>>>>> have
> >> > > >>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>> create a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the
> time
> >> > > >>> comes to
> >> > > >>>>> attach the
> >> > > >>>>>>>>> volume
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be
> smart
> >> > > >>> enough
> >> > > >>>>> to recognize
> >> > > >>>>>>>>> it's
> >> > > >>>>>>>>>>>>>>>>>> being
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> >> > volume
> >> > > >>> has
> >> > > >>>>> just been
> >> > > >>>>>>>>> created)
> >> > > >>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>> create,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository
> (for
> >> > > >>>>> XenServer) or a
> >> > > >>>>>>>>> datastore
> >> > > >>>>>>>>>>>>>>>>>> (for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that
> was
> >> > just
> >> > > >>>>> created.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> >> > spending
> >> > > >>> most
> >> > > >>>>> of my time
> >> > > >>>>>>>>>>>>> recently
> >> > > >>>>>>>>>>>>>>>>>> making
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> >> > code.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that
> clear it
> >> > up?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >> > 2013
> >> > > >>> at
> >> > > >>>>> 12:48 PM,
> >> > > >>>>>> John
> >> > > >>>>>>>>>>>>>>>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> explain
> >> > > >>> why
> >> > > >>>>> the the storage
> >> > > >>>>>>>>>>>>> driver
> >> > > >>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> 2013,
> >> > at
> >> > > >>> 1:21
> >> > > >>>>> PM, Mike
> >> > > >>>>>>>>>>>>> Tutkowski
> >> > > >>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> >> > ultimately I
> >> > > >>>>> would like to
> >> > > >>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>> all
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports.
> I
> >> > think
> >> > > >>> I'm
> >> > > >>>>> just out of
> >> > > >>>>>>>>> time
> >> > > >>>>>>>>>>>>>>>>>> for 4.2
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right
> now this
> >> > > >>> plug-in
> >> > > >>>>> supports
> >> > > >>>>>>>>>>>>> XenServer.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards
> to 4.2
> >> > > >>> feature
> >> > > >>>>> freeze, I
> >> > > >>>>>>>>> have
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> working
> >> > > >>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just
> to
> >> > be
> >> > > >>> clear,
> >> > > >>>>> this is
> >> > > >>>>>> all
> >> > > >>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> regards
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > Compute
> >> > > >>>>> Offerings post
> >> > > >>>>>> 4.2.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun 3,
> >> > > >>> 2013
> >> > > >>>>> at 11:14 AM,
> >> > > >>>>>>>>> Kelcey
> >> > > >>>>>>>>>>>>>>>>>> Jamison
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> kelcey@bbits.ca
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there
> any
> >> > plan
> >> > > >>> on
> >> > > >>>>> supporting
> >> > > >>>>>>>>> KVM in
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>> patch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----
> Original
> >> > > >>>>> Message -----
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:
> "Mike
> >> > > >>>>> Tutkowski" <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >> > > >>>>> dev@cloudstack.apache.org
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> >> > Monday,
> >> > > >>> June
> >> > > >>>>> 3, 2013
> >> > > >>>>>>>>> 10:12:32 AM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:
> Re:
> >> > > >>> [MERGE]
> >> > > >>>>>>>>>>> disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>> MASTER
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree
> on
> >> > > >>> merging
> >> > > >>>>> Wei's feature
> >> > > >>>>>>>>>>> first,
> >> > > >>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>> mine.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his
> feature
> >> > is
> >> > > >>> for
> >> > > >>>>> KVM only,
> >> > > >>>>>>>>> then
> >> > > >>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> is a
> >> > > >>>>>>>>>>>>>>>>>>>> non
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in
> 4.2.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon,
> Jun
> >> > 3,
> >> > > >>> 2013
> >> > > >>>>> at 8:55 AM,
> >> > > >>>>>> Wei
> >> > > >>>>>>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> ustcweizhou@gmail.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> the
> >> > billing,
> >> > > >>> as
> >> > > >>>>> no one works
> >> > > >>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>> billing
> >> > > >>>>>>>>>>>>>>>>>>>> now,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> billing
> >> > by
> >> > > >>>>> themselves. They
> >> > > >>>>>>>>> can
> >> > > >>>>>>>>>>>>> get
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > disk_offering of
> >> > > >>> a
> >> > > >>>>> VMs and
> >> > > >>>>>> volumes
> >> > > >>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell
> user
> >> > the
> >> > > >>>>> exact
> >> > > >>>>>> limitation
> >> > > >>>>>>>>>>>>> value
> >> > > >>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >> > limitation
> >> > > >>> for
> >> > > >>>>> nics as
> >> > > >>>>>> well.
> >> > > >>>>>>>>> I
> >> > > >>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>> work
> >> > > >>>>>>>>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a
> part of
> >> > I/O
> >> > > >>>>> throttling?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry
> my
> >> > > >>>>> misunstand the second
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree
> with
> >> > > >>> what
> >> > > >>>>> you said about
> >> > > >>>>>> the
> >> > > >>>>>>>>>>> two
> >> > > >>>>>>>>>>>>>>>>>>>> features.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/6/3
> >> > John
> >> > > >>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Jun 3,
> >> > 2013,
> >> > > >>> at
> >> > > >>>>> 2:13 AM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> John,
> >> > Mike
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> hope
> >> > Mike's
> >> > > >>>>> aswer helps you.
> >> > > >>>>>>>>> I am
> >> > > >>>>>>>>>>>>>>>>>> trying
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> >> > think
> >> > > >>> billing
> >> > > >>>>> should
> >> > > >>>>>>>>> depend
> >> > > >>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>>> IO
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > limitation.
> >> > > >>> Please
> >> > > >>>>> review
> >> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
> >> > > >>>>>>>>>>>>>>>>>>>> you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk_io_stat
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
> the
> >> > IO
> >> > > >>>>> statistics
> >> > > >>>>>> including
> >> > > >>>>>>>>>>>>>>>>>> bytes/iops
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> individual
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> virtual
> >> > > >>> machine.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Going by
> >> > the
> >> > > >>> AWS
> >> > > >>>>> model,
> >> > > >>>>>> customers
> >> > > >>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>> billed
> >> > > >>>>>>>>>>>>>>>>>>>>>> more
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > provisioned
> >> > > >>> IOPS,
> >> > > >>>>> as well as,
> >> > > >>>>>> for
> >> > > >>>>>>>>>>>>> those
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://aws.amazon.com/ebs/).
> >> > > >>>>>> I
> >> > > >>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>> imagine
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> option to
> >> > > >>> employ
> >> > > >>>>> similar cost
> >> > > >>>>>>>>>>> models.
> >> > > >>>>>>>>>>>>>>>>>> Could
> >> > > >>>>>>>>>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> billing
> >> > model
> >> > > >>> in
> >> > > >>>>> the current
> >> > > >>>>>>>>> patch?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> Do
> >> > you
> >> > > >>> mean
> >> > > >>>>> IOPS runtime
> >> > > >>>>>>>>>>> change?
> >> > > >>>>>>>>>>>>>>>>>> KVM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > limitation for
> >> > > >>> a
> >> > > >>>>> running
> >> > > >>>>>> virtual
> >> > > >>>>>>>>>>>>>>>>>> machine
> >> > > >>>>>>>>>>>>>>>>>>>>>> through
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> However,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > CloudStack
> >> > > >>> does
> >> > > >>>>> not support
> >> > > >>>>>>>>>>> changing
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> offering
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > (computer
> >> > > >>>>> offering or disk
> >> > > >>>>>>>>>>>>> offering).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> meant at
> >> > the
> >> > > >>>>> Java interface
> >> > > >>>>>>>>> level.
> >> > > >>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>> apologize
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> more
> >> > > >>>>> generalize allocation
> >> > > >>>>>>>>>>>>>>>>>> algorithms
> >> > > >>>>>>>>>>>>>>>>>>>> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> describe
> >> > the
> >> > > >>>>> service guarantees
> >> > > >>>>>>>>>>>>>>>>>> provided by a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> It is a
> >> > > >>> good
> >> > > >>>>> question.
> >> > > >>>>>>>>> Maybe it
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>> better
> >> > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> disk_io_throttling as Mike
> >> > > >>>>>>>>> needs to
> >> > > >>>>>>>>>>>>>>>>>> consider
> >> > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> hypervisor
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> type, I
> >> > think.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> >> > expand
> >> > > >>> on
> >> > > >>>>> my thoughts
> >> > > >>>>>> in a
> >> > > >>>>>>>>>>>>> later
> >> > > >>>>>>>>>>>>>>>>>>>>>> response
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> touch
> >> > points
> >> > > >>>>> between these two
> >> > > >>>>>>>>>>>>>>>>>> features. I
> >> > > >>>>>>>>>>>>>>>>>>>>>> think
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> need
> >> > to be
> >> > > >>>>> merged before
> >> > > >>>>>>>>>>>>>>>>>> SolidFire, but
> >> > > >>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > coordination
> >> > > >>>>> between the
> >> > > >>>>>> branches
> >> > > >>>>>>>>>>>>>>>>>> (possibly
> >> > > >>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> disk_io_throttling)
> >> > > >>>>> to
> >> > > >>>>>>>>> coordinate on
> >> > > >>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>> issue.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/6/3
> >> > > >>> John
> >> > > >>>>> Burwell <
> >> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Mike,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >> > things I
> >> > > >>>>> want to
> >> > > >>>>>> understand
> >> > > >>>>>>>>>>> are
> >> > > >>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1)
> Is
> >> > there
> >> > > >>>>> value in
> >> > > >>>>>> capturing
> >> > > >>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>> policies
> >> > > >>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> common
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> data
> >> > model
> >> > > >>> (e.g.
> >> > > >>>>> for
> >> > > >>>>>>>>> billing/usage
> >> > > >>>>>>>>>>>>>>>>>>>> purposes,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> >> > Should
> >> > > >>> there
> >> > > >>>>> be a common
> >> > > >>>>>>>>>>>>>>>>>> interface model
> >> > > >>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > provisioning
> >> > > >>> at
> >> > > >>>>> runtime?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3)
> How
> >> > are
> >> > > >>>>> conflicting
> >> > > >>>>>>>>> provisioned
> >> > > >>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > hypervisor
> >> > > >>> and
> >> > > >>>>> storage device
> >> > > >>>>>>>>>>>>>>>>>> reconciled?
> >> > > >>>>>>>>>>>>>>>>>>>> In
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> where
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> lead
> >> > to
> >> > > >>>>> believe (and
> >> > > >>>>>> billed)
> >> > > >>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>> more
> >> > > >>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> storage
> >> > > >>> device
> >> > > >>>>> has been
> >> > > >>>>>>>>> configured
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > consistent
> >> > > >>>>> configuration
> >> > > >>>>>>>>> between a
> >> > > >>>>>>>>>>>>>>>>>> VM and a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> time,
> >> > but a
> >> > > >>>>> later
> >> > > >>>>>> modification
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> inconsistency.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Jun
> >> > 2,
> >> > > >>> 2013,
> >> > > >>>>> at 8:38 PM,
> >> > > >>>>>>>>> Mike
> >> > > >>>>>>>>>>>>>>>>>> Tutkowski
> >> > > >>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> mike.tutkowski@solidfire.com>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> John,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > believe
> >> > > >>> Wei's
> >> > > >>>>> feature deals
> >> > > >>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>> controlling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >> > > >>> hypervisor
> >> > > >>>>> side.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> My
> >> > feature
> >> > > >>> is
> >> > > >>>>> focused on
> >> > > >>>>>>>>>>>>> controlling
> >> > > >>>>>>>>>>>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>>>>>> from
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> hope
> >> > that
> >> > > >>>>> helps. :)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Sun,
> >> > Jun
> >> > > >>> 2,
> >> > > >>>>> 2013 at 6:35
> >> > > >>>>>> PM,
> >> > > >>>>>>>>>>>>> John
> >> > > >>>>>>>>>>>>>>>>>>>> Burwell
> >> > > >>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > jburwell@basho.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> My
> >> > > >>> opinion is
> >> > > >>>>> that no
> >> > > >>>>>> features
> >> > > >>>>>>>>>>>>>>>>>> should be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> functional
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> issues
> >> > > >>> have
> >> > > >>>>> been resolved
> >> > > >>>>>> and
> >> > > >>>>>>>>> it
> >> > > >>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>> ready
> >> > > >>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> total
> >> > Ops
> >> > > >>> vs
> >> > > >>>>> discrete
> >> > > >>>>>>>>> read/write
> >> > > >>>>>>>>>>>>>>>>>> ops issue
> >> > > >>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> >> > reviewed
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > I
> >> > > >>> don't
> >> > > >>>>> think this
> >> > > >>>>>>>>> criteria
> >> > > >>>>>>>>>>>>>>>>>> has been
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Also,
> >> > how
> >> > > >>>>> does this work
> >> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >> > > >>>>>>>>>>>>>>>>>> )?
> >> > > >>>>>>>>>>>>>>>>>>>> As I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> also
> >> > > >>> involves
> >> > > >>>>> provisioned
> >> > > >>>>>>>>> IOPS. I
> >> > > >>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>> like
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > scenario
> >> > > >>>>> where provisioned
> >> > > >>>>>>>>> IOPS
> >> > > >>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> KVM and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > unnecessarily
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> incompatible.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > Jun 1,
> >> > > >>> 2013,
> >> > > >>>>> at 6:47 AM,
> >> > > >>>>>>>>> Wei
> >> > > >>>>>>>>>>>>>>>>>> ZHOU <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> ustcweizhou@gmail.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Sure. I
> >> > will
> >> > > >>>>> change it next
> >> > > >>>>>>>>> week.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > 2013/6/1
> >> > > >>>>> Wido den Hollander
> >> > > >>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 06/01/2013 08:24 AM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Exactly. I
> >> > > >>> have
> >> > > >>>>> pushed the
> >> > > >>>>>>>>>>>>> features
> >> > > >>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > anyone
> >> > > >>>>> object thems for
> >> > > >>>>>>>>>>>>> technical
> >> > > >>>>>>>>>>>>>>>>>>>> reason
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> them.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> For
> >> > the
> >> > > >>> sake
> >> > > >>>>> of clarity I
> >> > > >>>>>> just
> >> > > >>>>>>>>>>>>> want
> >> > > >>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>> mention
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> change
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> the
> >> > total
> >> > > >>> IOps
> >> > > >>>>> to R/W IOps
> >> > > >>>>>>>>> asap
> >> > > >>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>> that we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> only
> >> > total
> >> > > >>> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> You
> >> > laid
> >> > > >>> the
> >> > > >>>>> groundwork for
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>> I/O
> >> > > >>>>>>>>>>>>>>>>>>>>>> throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> should
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > however
> >> > > >>>>> prevent that we
> >> > > >>>>>> create
> >> > > >>>>>>>>>>>>>>>>>> legacy from
> >> > > >>>>>>>>>>>>>>>>>>>>>> day
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > 2013/5/31
> >> > > >>>>> Wido den
> >> > > >>>>>> Hollander <
> >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 05/31/2013 03:59 PM, John
> >> > > >>>>>>>>>>>>> Burwell
> >> > > >>>>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> +1 --
> >> > this
> >> > > >>>>> enhancement must
> >> > > >>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> discretely
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> don't
> >> > see
> >> > > >>> how
> >> > > >>>>> it could be
> >> > > >>>>>>>>> fixed
> >> > > >>>>>>>>>>>>>>>>>> later
> >> > > >>>>>>>>>>>>>>>>>>>>>> because
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > correctly
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> split
> >> > total
> >> > > >>> IOPS
> >> > > >>>>> into read
> >> > > >>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> write.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> with a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> total
> >> > > >>>>> unless/until we
> >> > > >>>>>> decided
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> break
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> What
> >> > Wei
> >> > > >>>>> meant was merging
> >> > > >>>>>> it
> >> > > >>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>> master
> >> > > >>>>>>>>>>>>>>>>>>>>>> now
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 4.2
> >> > branch
> >> > > >>>>> and add Read /
> >> > > >>>>>>>>> Write
> >> > > >>>>>>>>>>>>> IOps
> >> > > >>>>>>>>>>>>>>>>>>>> before
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> will be
> >> > > >>>>> released with Read
> >> > > >>>>>> and
> >> > > >>>>>>>>>>>>> Write
> >> > > >>>>>>>>>>>>>>>>>>>> instead
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> This is
> >> > to
> >> > > >>>>> make the May 31st
> >> > > >>>>>>>>>>>>> feature
> >> > > >>>>>>>>>>>>>>>>>>>> freeze
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> moves
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (see
> >> > other
> >> > > >>>>> threads) then it
> >> > > >>>>>>>>> won't
> >> > > >>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>> necessary
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> also
> >> > > >>>>> completely agree that
> >> > > >>>>>>>>>>> there
> >> > > >>>>>>>>>>>>>>>>>> is no
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk
> >> > I/O.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Thanks,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> John
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > May 31,
> >> > > >>>>> 2013, at 9:51 AM,
> >> > > >>>>>>>>> Wido
> >> > > >>>>>>>>>>>>>>>>>> den
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wido@widodh.nl
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 05/31/2013 03:13 PM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi
> >> > Wido,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Thanks.
> >> > > >>> Good
> >> > > >>>>> question.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > thought
> >> > > >>>>> about at the
> >> > > >>>>>>>>>>> beginning.
> >> > > >>>>>>>>>>>>>>>>>>>> Finally I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > difference
> >> > > >>> of
> >> > > >>>>> read and write
> >> > > >>>>>>>>>>>>> mainly
> >> > > >>>>>>>>>>>>>>>>>>>> because
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> not
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> care
> >> > the
> >> > > >>>>> difference of sent
> >> > > >>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>> received
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> That
> >> > > >>>>> reasoning seems odd.
> >> > > >>>>>>>>>>>>>>>>>> Networking and
> >> > > >>>>>>>>>>>>>>>>>>>>>> disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> different.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Disk
> >> > I/O is
> >> > > >>>>> much more
> >> > > >>>>>>>>> expensive
> >> > > >>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>> most
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > bandwith.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> Implementing
> >> > > >>>>> it will be some
> >> > > >>>>>>>>>>>>>>>>>> copy-paste
> >> > > >>>>>>>>>>>>>>>>>>>>>> work.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> implemented
> >> > > >>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> few
> >> > days.
> >> > > >>> For
> >> > > >>>>> the deadline
> >> > > >>>>>> of
> >> > > >>>>>>>>>>>>>>>>>> feature
> >> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> after
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> that ,
> >> > if
> >> > > >>>>> needed.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> >> > think it's
> >> > > >>> a
> >> > > >>>>> feature we
> >> > > >>>>>>>>> can't
> >> > > >>>>>>>>>>>>>>>>>> miss. But
> >> > > >>>>>>>>>>>>>>>>>>>>>> if
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > window
> >> > > >>> we
> >> > > >>>>> have to make sure
> >> > > >>>>>> we
> >> > > >>>>>>>>>>>>> don't
> >> > > >>>>>>>>>>>>>>>>>>>> release
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> fix
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> in
> >> > 4.3,
> >> > > >>> that
> >> > > >>>>> will confuse
> >> > > >>>>>>>>>>>>> users.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > 2013/5/31
> >> > > >>>>> Wido den
> >> > > >>>>>> Hollander <
> >> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi Wei,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> On
> >> > > >>>>> 05/30/2013 06:03 PM, Wei
> >> > > >>>>>>>>> ZHOU
> >> > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Hi,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >> > would
> >> > > >>> like to
> >> > > >>>>> merge
> >> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >> > nobody
> >> > > >>>>> object, I will
> >> > > >>>>>> merge
> >> > > >>>>>>>>>>>>> into
> >> > > >>>>>>>>>>>>>>>>>> master
> >> > > >>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >> > > >>> purpose
> >> > > >>>>> is :
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Virtual
> >> > > >>>>> machines are running
> >> > > >>>>>>>>> on
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>> same
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > storage or
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> share
> >> > > >>> strage).
> >> > > >>>>> Because of
> >> > > >>>>>> the
> >> > > >>>>>>>>>>> rate
> >> > > >>>>>>>>>>>>>>>>>>>>>> limitation
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> iops),
> >> > if
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> one
> >> > VM
> >> > > >>> has
> >> > > >>>>> large disk
> >> > > >>>>>>>>> operation,
> >> > > >>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>> may
> >> > > >>>>>>>>>>>>>>>>>>>>>> affect
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> performance
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> other
> >> > VMs
> >> > > >>>>> running on the
> >> > > >>>>>> same
> >> > > >>>>>>>>>>>>>>>>>> storage
> >> > > >>>>>>>>>>>>>>>>>>>>>> device.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> is
> >> > > >>> neccesary
> >> > > >>>>> to set the
> >> > > >>>>>>>>>>> maximum
> >> > > >>>>>>>>>>>>>>>>>> rate
> >> > > >>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> VMs.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Looking at
> >> > > >>> the
> >> > > >>>>> code I see
> >> > > >>>>>> you
> >> > > >>>>>>>>>>> make
> >> > > >>>>>>>>>>>>>>>>>> no
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Qemu
> >> > and
> >> > > >>>>> libvirt support
> >> > > >>>>>>>>> setting
> >> > > >>>>>>>>>>>>>>>>>> both a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps
> >> > > >>> which
> >> > > >>>>> could benefit a
> >> > > >>>>>>>>> lot of
> >> > > >>>>>>>>>>>>>>>>>> users.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> It's
> >> > also
> >> > > >>>>> strange, in the
> >> > > >>>>>>>>> polling
> >> > > >>>>>>>>>>>>>>>>>> side you
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps,
> >> > but
> >> > > >>> on
> >> > > >>>>> the throttling
> >> > > >>>>>>>>> side
> >> > > >>>>>>>>>>>>>>>>>> you only
> >> > > >>>>>>>>>>>>>>>>>>>> go
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Write
> >> > IOps
> >> > > >>> are
> >> > > >>>>> usually much
> >> > > >>>>>>>>> more
> >> > > >>>>>>>>>>>>>>>>>> expensive
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> seems
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> like a
> >> > valid
> >> > > >>>>> use-case where
> >> > > >>>>>>>>> that
> >> > > >>>>>>>>>>>>> an
> >> > > >>>>>>>>>>>>>>>>>> admin
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> write
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> IOps
> >> > vs
> >> > > >>> Read
> >> > > >>>>> IOps.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Since
> >> > this
> >> > > >>> only
> >> > > >>>>> supports KVM
> >> > > >>>>>>>>> at
> >> > > >>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>> point I
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> great
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> value
> >> > to at
> >> > > >>>>> least have the
> >> > > >>>>>>>>>>>>>>>>>> mechanism in
> >> > > >>>>>>>>>>>>>>>>>>>>>> place
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> implementing
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> this
> >> > later
> >> > > >>>>> would be a lot of
> >> > > >>>>>>>>>>> work.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> a
> >> > > >>> hypervisor
> >> > > >>>>> doesn't
> >> > > >>>>>>>>> support
> >> > > >>>>>>>>>>>>>>>>>> setting
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> write
> >> > you
> >> > > >>> can
> >> > > >>>>> always sum
> >> > > >>>>>> both
> >> > > >>>>>>>>> up
> >> > > >>>>>>>>>>>>>>>>>> and set
> >> > > >>>>>>>>>>>>>>>>>>>>>> that
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Can
> >> > you
> >> > > >>>>> explain why you
> >> > > >>>>>>>>>>>>> implemented
> >> > > >>>>>>>>>>>>>>>>>> it
> >> > > >>>>>>>>>>>>>>>>>>>> this
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wido
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> The
> >> > > >>> feature
> >> > > >>>>> includes:
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (1) set
> >> > the
> >> > > >>>>> maximum rate of
> >> > > >>>>>>>>> VMs
> >> > > >>>>>>>>>>>>> (in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> configuration)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (2)
> >> > change
> >> > > >>> the
> >> > > >>>>> maximum rate
> >> > > >>>>>> of
> >> > > >>>>>>>>>>> VMs
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (3)
> >> > limit
> >> > > >>> the
> >> > > >>>>> disk rate
> >> > > >>>>>> (total
> >> > > >>>>>>>>>>> bps
> >> > > >>>>>>>>>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>> iops)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> JIRA
> >> > ticket:
> >> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> issues.apache.org/****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> jira/browse/CLOUDSTACK-1192<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1192>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> > > >>> 1192<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >> > 1192
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1192<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >> > 1192>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> FS (I
> >> > will
> >> > > >>>>> update later) :
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>> VM+Disk+IO+Throttling<https://
> >> <https://%0b/>> > >>>>>>>>>>>>> ****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cwiki.apache.org/confluence/****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >> > > >>>>> rottling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Merge
> >> > > >>> check
> >> > > >>>>> list :-
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> Did
> >> > you
> >> > > >>>>> check the branch's
> >> > > >>>>>>>>> RAT
> >> > > >>>>>>>>>>>>>>>>>> execution
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Yes
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> Are
> >> > > >>> there
> >> > > >>>>> new dependencies
> >> > > >>>>>>>>>>>>>>>>>> introduced?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> No
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >> > What
> >> > > >>>>> automated testing
> >> > > >>>>>> (unit
> >> > > >>>>>>>>>>> and
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> new
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > feature?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Unit
> >> > tests
> >> > > >>> are
> >> > > >>>>> added.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> >> > What
> >> > > >>>>> testing has been done
> >> > > >>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>>> check for
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (1) set
> >> > the
> >> > > >>>>> bytes rate and
> >> > > >>>>>>>>> IOPS
> >> > > >>>>>>>>>>>>>>>>>> rate on
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (2)
> >> > VM
> >> > > >>>>> operations, including
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > deploy,
> >> > > >>> stop,
> >> > > >>>>> start, reboot,
> >> > > >>>>>>>>>>>>>>>>>> destroy,
> >> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (3)
> >> > > >>> Volume
> >> > > >>>>> operations,
> >> > > >>>>>>>>> including
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Attach,
> >> > > >>>>> Detach
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> To
> >> > review
> >> > > >>> the
> >> > > >>>>> code, you can
> >> > > >>>>>>>>> try
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> git diff
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Best
> >> > > >>> regards,
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Wei
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [1]
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>
> >> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>> VM+Disk+IO+Throttling<https://
> >> <https://%0b/>> > >>>>>>>>>>>>> ****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cwiki.apache.org/confluence/****
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>
> >> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >> > > >>>>> rottling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [2]
> >> > > >>>>>>>>> refs/heads/disk_io_throttling
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [3]
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-
> 1301
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >> > > >>>>> 1301
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> issues.apache.org/****jira/browse/CLOUDSTACK-
> >> > > >>> 1301<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1301>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> > > >>> 1301<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >> > 1301
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 1301<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >> > 1301>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <ht**tps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > issues.apache.org/****jira/**browse/CLOUDSTACK-
> >> > > >>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> > > >>>>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> **<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> > > >>>>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> >> > 2071
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <**
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >> > > >>>>> 2071
> >> > > >>>>>>>>>>>>>>>>>>>> <
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >> > > >>>>> 2071>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> <h**ttps://
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-
> >> > 2071>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> (**CLOUDSTACK-1301
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> >> > VM
> >> > > >>> Disk
> >> > > >>>>> I/O
> >> > > >>>>>> Throttling)
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Mike
> >> > > >>>>> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *Senior
> >> > > >>>>> CloudStack Developer,
> >> > > >>>>>>>>>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>>>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> > > >>> 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > Advancing
> >> > > >>> the
> >> > > >>>>> way the world
> >> > > >>>>>>>>> uses
> >> > > >>>>>>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> > > >>> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > > >>> CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> > 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> >> > the
> >> > > >>> way
> >> > > >>>>> the world uses
> >> > > >>>>>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> > Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > > >>> CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> > 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing the
> >> > > >>> way
> >> > > >>>>> the world uses
> >> > > >>>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> > Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>>>>> SolidFire
> >> > > >>>>>>>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >> > way
> >> > > >>> the
> >> > > >>>>> world uses the
> >> > > >>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> > CloudStack
> >> > > >>>>> Developer,
> >> > > >>>>>> SolidFire
> >> > > >>>>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> >> > way
> >> > > >>> the
> >> > > >>>>> world uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >> > > >>>>> Developer, SolidFire
> >> > > >>>>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> >> > the
> >> > > >>>>> world uses the
> >> > > >>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >> > > >>> Developer,
> >> > > >>>>> SolidFire
> >> > > >>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> >> > the
> >> > > >>> world
> >> > > >>>>> uses the
> >> > > >>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> >> > > >>> Developer,
> >> > > >>>>> SolidFire
> >> > > >>>>>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>>>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way the
> >> > > >>> world
> >> > > >>>>> uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> > Developer,
> >> > > >>>>> SolidFire
> >> > > >>>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > > >>> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> >> > world
> >> > > >>> uses
> >> > > >>>>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> > Developer,
> >> > > >>>>> SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> > world
> >> > > >>> uses
> >> > > >>>>> the cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >> > > >>>>> SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >> > uses
> >> > > >>> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>
> >> > http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >> > > >>> SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> > mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> >> > uses
> >> > > >>> the
> >> > > >>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >> > > >>> SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> > the
> >> > > >>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> > > >>> SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> > the
> >> > > >>>>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> > SolidFire
> >> > > >>>>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> > the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> > SolidFire
> >> > > >>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >> > > >>> cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >> > > >>> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >> > Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > cloud<
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >> > Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >> > > >>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>>>>>>>
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>>>>
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> --
> >> > > >>>>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>>>> o: 303.746.7302
> >> > > >>>>>>>>>> Advancing the way the world uses the
> >> > > >>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>>>> *(tm)*
> >> > > >>>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> --
> >> > > >>>>>>>> *Mike Tutkowski*
> >> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>>> o: 303.746.7302
> >> > > >>>>>>>> Advancing the way the world uses the cloud<
> >> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>>> *(tm)*
> >> > > >>>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>>
> >> > > >>>>>>> --
> >> > > >>>>>>> *Mike Tutkowski*
> >> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>>>> o: 303.746.7302
> >> > > >>>>>>> Advancing the way the world uses the
> >> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>>>> *(tm)*
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> --
> >> > > >>>>> *Mike Tutkowski*
> >> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>>>> e: mike.tutkowski@solidfire.com
> >> > > >>>>> o: 303.746.7302
> >> > > >>>>> Advancing the way the world uses the
> >> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>>>> *(tm)*
> >> > > >>>>
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>> --
> >> > > >>> *Mike Tutkowski*
> >> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > >>> e: mike.tutkowski@solidfire.com
> >> > > >>> o: 303.746.7302
> >> > > >>> Advancing the way the world uses the
> >> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > >>> *(tm)*
> >> > > >>
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > *Mike Tutkowski*
> >> > > > *Senior CloudStack Developer, SolidFire Inc.*
> >> > > > e: mike.tutkowski@solidfire.com
> >> > > > o: 303.746.7302
> >> > > > Advancing the way the world uses the
> >> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > > *(tm)*
> >> > >
> >> > >
> >> >
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *(tm)*****
> >>
> >>
> >>
> >> ****
> >>
> >> ** **
> >>
> >> --
> >> *Mike Tutkowski*****
> >>
> >> *Senior CloudStack Developer, SolidFire Inc.*****
> >>
> >> e: mike.tutkowski@solidfire.com****
> >>
> >> o: 303.746.7302****
> >>
> >> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*****
> >>
> >>
> >>
> >> ****
> >>
> >> ** **
> >>
> >> --
> >> *Mike Tutkowski*****
> >>
> >> *Senior CloudStack Developer, SolidFire Inc.*****
> >>
> >> e: mike.tutkowski@solidfire.com****
> >>
> >> o: 303.746.7302****
> >>
> >> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*****
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
> >
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I believe we're on the same page, John.
I would add a new parameter to the createStoragePool API command (let's
call it "managed" and it can be true or false). If you pass in "true", but
the driver does not support being managed by CloudStack, an exception is
thrown by the driver. If you pass in "false", but the driver does not
support being managed outside of CloudStack, an exception is thrown by the
driver. It is possible one could write a driver that can be managed both
inside or outside of CloudStack (the SolidFire driver requires being
managed only by CloudStack).
This managed value would end up in the storage_pool table in a new column
("managed").
An example usage: After the storage framework calls on the driver's
createAsync method, it will check to see if the driver is managed. If the
driver is managed, the storage framework will invoke the proper (new)
command of the hypervisor in use to set up, say, the SR for the SAN volume.
There may be other cases when the isManaged() method is invoked on the
driver and - if true is returned - the storage framework sends a particular
command to the hypervisor in use.
********************
As far as reworking the default storage plug-in (which sends commands to
hypervisors) for 4.2, I would check with Edison on his schedule.
Thanks!
On Mon, Jun 10, 2013 at 11:53 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I want to make sure we are on the same page regarding isManaged. I see
> the addition of the following operations:
>
> * DataStoreDriver#supportsManagement() : boolean -- indicates whether or
> not the driver support/implements the management functions
> * DataStore#isManaged() : boolean -- indicates whether or not a device is
> managed by CloudStack
>
> Only DataStores associated with DataStoreDrivers that support management
> would be able to have the management flag enabled. This behavior allows
> operators to declare their intention to allow CloudStack to manage a
> device. This behavior would need to be exposed as flag on the HTTP API.
> Additionally, this flag would only be settable when a device is
> defined/created in the system (i.e. once a device is managed by CloudStack,
> you can't unmanaged it without dissociating it from CloudStack). In the
> future, we may consider relaxing that rule, but, in the short term, it
> greatly decreases the alternative flows.
>
> In terms of removing the dependencies from Storage to Hypervisor, I would
> really like to see this accomplished in 4.2. It will be much more
> difficult to address this issue post 4.2 because we will have more plugins
> to address. I also see this issue as a critical architectural issues that
> for the management server. I recommend we identify the dependencies that
> need to be inverted, and see if we can divide the work across a few more
> resources.
>
> Thanks,
> -John
>
> On Jun 10, 2013, at 1:36 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> Sounds good, Edison :)
>
> I can implement John's isManaged() logic and resubmit my code for review.
>
> Perhaps in 4.3 I could address the direct-attach scenario. If it requires
> some refactoring of the storage framework, then I could do that.
>
> Also, the default storage plug-in does have hypervisor logic in it in 4.2.
> I recommend we leave this as is for 4.2 and it can be refactored in 4.3
> with the above work I mentioned.
>
> Are you OK with that, guys?
>
>
> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com> wrote:
>
>> Ok, I am Ok with the way you are talking about in 4.2.****
>>
>> ** **
>>
>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> *Sent:* Sunday, June 09, 2013 8:59 PM
>> *To:* dev@cloudstack.apache.org
>> *Cc:* Edison Su; John Burwell
>> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>>
>> ** **
>>
>> Another thing we might want to consider for 4.2 is that the way the code
>> is currently implemented in master, I don't think it's even possible to
>> write a plug-in that connects storage directly to a VM because the
>> CloudStack storage framework will call the "attach" method of the
>> hypervisor in use and this method will fail because it assumes there is
>> (talking Xen here) an SR existent and in this situation there isn't one.*
>> ***
>>
>> ** **
>>
>> My thinking is that we should ignore the direct-attach-of-volume-to-VM
>> use case for 4.2. I think it will require too much of a rewrite at this
>> point in development (fewer than three weeks left until code freeze).****
>>
>> ** **
>>
>> If the plug-in implements an isManaged() method (as John suggested), the
>> storage framework can invoke the necessary commands on the hypervisor in
>> use.****
>>
>> ** **
>>
>> For example, the createAsync method is called. The storage plug-in
>> creates a SAN volume and updates the CS DB. The storage framework then asks
>> the driver if it's managed. If it is, the storage framework executes a
>> command against the hypervisor in use to have it create, say, an SR (with a
>> single VDI that takes up the entire SR). Now, when the storage framework
>> executes the "attach" command against the hypervisor, this command will
>> work, as well (because that logic assumes the prior existence of an SR and
>> it will, in fact, be present).****
>>
>> ** **
>>
>> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:****
>>
>> Hi Edison and John,****
>>
>> ** **
>>
>> I wanted to point out something I thought was relevant to this
>> conversation.****
>>
>> ** **
>>
>> My plug-in supports both XenServer and ESX(i).****
>>
>> ** **
>>
>> These two hypervisors handle their respective hypervisor data structures
>> (XenServer SR or ESX(i) datastore) quite differently.****
>>
>> ** **
>>
>> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
>> when you issue a remove-SR command to XenServer.****
>>
>> ** **
>>
>> When you later want to access the contents of that data, it is still
>> safely stored on the LUN and can be accessed from XenServer (from the same
>> or different cluster (what XenServer calls a resource pool)) by creating a
>> new SR that is based on the same IQN and LUN.****
>>
>> ** **
>>
>> XenServer SRs exist at the cluster level.****
>>
>> ** **
>>
>> In ESX(i), the datastore that resides on the SAN LUN is actually
>> destroyed when you issue a remove-datastore command to ESX(i). In other
>> words, the contents of the SAN LUN are destroyed.****
>>
>> ** **
>>
>> ESX(i) datastores exist at the datacenter level.****
>>
>> ** **
>>
>> The XenServer behavior (not destroying the contents of the LUN when the
>> SR is removed) is desirable; the ESX(i) behavior (destroying the contents
>> of the LUN when the datastore is removed) is not desirable.****
>>
>> ** **
>>
>> I can get around the ESX(i) behavior by removing the referenced IQN from
>> each host in the cluster when the CS storage framework notifies my plug-in
>> of a detach-volume event. When the CS volume is later attached again
>> (either to the same or a different ESX(i) cluster), I can have each host in
>> the relevant ESX(i) cluster reference the IQN and the datastore will be
>> back again.****
>>
>> ** **
>>
>> This sequence of events is critical for my plug-in to support both
>> environments:****
>>
>> *XenServer flow (events for Driver class):*****
>>
>> createAsync (called right before first attach) = creates SAN volume****
>>
>> preAttachVolume = creates SR and creates VDI or just introduces SR****
>>
>> (CS volume gets attached and later detached)****
>>
>> postDetachVolume = removes SR (this does not delete data on SAN volume)**
>> **
>>
>> deleteAsync (can be called when volume is not attached) = deletes the SAN
>> volume****
>>
>> *VMware flow (events for Driver class):*****
>>
>> createAsync (called right before first attach) = creates SAN volume,
>> creates datastore, and creates VMDK file****
>>
>> preAttachVolume = adds iSCSI target to each host in cluster****
>>
>> (CS volume gets attached and later detached)****
>>
>> postDetachVolume = removes iSCSI target from each host in cluster****
>>
>> deleteAsync (can be called when volume is not attached) = deletes
>> datastore and deletes the SAN volume****
>>
>> ************************
>>
>> So, the way I have it written up here, the driver is talking to the
>> hypervisor. As John has pointed out, this may not be ideal.****
>>
>> ** **
>>
>> Whatever we decide on, it needs to be able to support this kind of flow.
>> :)****
>>
>> ** **
>>
>> We could go John's route, where the driver says if it's managed or not.**
>> **
>>
>> ** **
>>
>> If it is, then the storage framework could send the necessary messages to
>> the hypervisor in question at the right times (just migrate the hypervisor
>> logic above from the driver to the right parts of the storage framework).
>> ****
>>
>> ** **
>>
>> This is fine, except that it does not resolve Edison's use case of
>> desiring to have the storage LUN potentially connected directly to a VM.*
>> ***
>>
>> ** **
>>
>> I'm not sure we should design around that use case, though. Is it common
>> enough? Has anyone asked for it? Is it even desirable? How are usage
>> statistics gathered for such volumes? It's nice to offer the flexibility,
>> but is it a good idea in this case?****
>>
>> ** **
>>
>> Thanks!****
>>
>> ** **
>>
>> ** **
>>
>> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:**
>> **
>>
>>
>>
>> > -----Original Message-----
>> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > Sent: Friday, June 07, 2013 4:08 PM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >
>> > Hi John,
>> >
>> > Yes, you are correct that Xen must layer an SR on top of the iSCSI
>> volume
>> > to use it. Same story for VMware: ESX(i) must layer a datastore on top
>> of
>> > the iSCSI volume to use it. I look at it like they are layering a
>> clustered
>> > file system on the SAN volume so the hypervisors can share access to the
>> > contents of the volume.
>> >
>> > In 4.1, by the time the hypervisor-attach-volume logic is called, the
>> SR or
>> > datastore has already been created (usually manually by an admin).
>> >
>> > This pre-setup of the, say, SR is, of course, not acceptable in an
>> > environment where each CloudStack volume that a user creates is mapped
>> > to a
>> > single SAN volume (via the SR).
>> >
>> > The question comes down to who should allocate the SR dynamically.
>> >
>> > We could have the storage framework ask the storage plug-in if it is
>> > managed. If it is, then the storage framework could send a message to
>> the
>> > hypervisor in question to create (let's talk Xen here) the SR ahead of
>> > time. Then, when the storage framework next sends the attach-volume
>> > command
>> > to the hypervisor, it should work without changes to that attach-volume
>> > logic (because - from the point of view of the attach logic - the SR is
>> > already existent, as expected).
>> >
>> > Now, as Edison has pointed out, this limits the power of the storage
>> > plug-in. A storage plug-in in this model cannot directly attach storage
>> to
>> > a VM (it must go through the hypervisor). Perhaps that is OK. We need to
>> > make a call on that.
>>
>>
>> Yes, I think we need to address this usage case, and won't limit the
>> power of the storage plug-in.
>> John, what's your idea to fit this usage case into your abstraction?
>>
>>
>> >
>> > If we want to give more power to the storage vendor, we would have to
>> > have
>> > the storage framework call into the storage plug-in to send the
>> appropriate
>> > attach commands to the hypervisor.
>> >
>> > Let's discuss and come to a consensus (or at least agree on a path) as
>> soon
>> > as we can. :)
>> >
>> > Thanks!
>> >
>> >
>> >
>> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> > > Mike,
>> > >
>> > > My understanding is the Xen expects an SR structure on all ISCSI
>> devices
>> > > -- at least that is how I read the code in your patch. Is my
>> understanding
>> > > correct? If so, the Xen plugin should be able to query the storage
>> device
>> > > to determine the presence of the SR structure and create it if it
>> does not
>> > > exist. Am I missing something in the implementation that makes that
>> type
>> > > of implementation impossible?
>> > >
>> > > Thanks,
>> > > -John
>> > >
>> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> > > wrote:
>> > >
>> > > > Yeah, if a storage vendor wanted to attach a volume directly to a
>> VM in
>> > > 4.2
>> > > > today, it would probably fail because the attach-volume logic
>> assumes
>> > the
>> > > > existence of the necessary hypervisor data structure (ex. SR on
>> Xen).
>> > > >
>> > > > If we wanted to enable such an attach, we could do it the way Edison
>> > > > suggests.
>> > > >
>> > > >
>> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
>> wrote:
>> > > >
>> > > >>
>> > > >>
>> > > >>> -----Original Message-----
>> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > > >>> Sent: Friday, June 07, 2013 2:37 PM
>> > > >>> To: dev@cloudstack.apache.org
>> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> > > >>>
>> > > >>> As we only have three weeks until feature freeze, we should come
>> to
>> > a
>> > > >>> consensus on this design point as soon as possible.
>> > > >>>
>> > > >>> Right now, if the storage framework asks my driver it is is
>> managed, it
>> > > >>> will say 'yes.' This means the framework will tell the driver to
>> > > perform
>> > > >>> its management activities. This then means the driver will call
>> into
>> > > the
>> > > >>> host (it doesn't know which hypervisor, by the way) to perform the
>> > > >> activity
>> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
>> > > >>>
>> > > >>> The driver doesn't know which hypervisor it's talking to, it just
>> > > sends a
>> > > >>> message to the host to perform the necessary pre-attach work.
>> > > >>
>> > > >> Could we just expose a method like "attachVolume/dettachVolume" on
>> > the
>> > > >> PrimaryDataStoreDriver driver?
>> > > >> For most of cases, the implementation of each driver would just
>> send
>> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
>> > the
>> > > >> implementation in a base class, so that can be shared by all of
>> this
>> > > >> drivers), and in each hypervisor resource code, which may just call
>> > > >> hypervisor's api to attach the volume to VM, while for certain
>> storage,
>> > > >> like SolidFire, may need to create a SR first, and create volume
>> on it,
>> > > >> then call hypervisor's API to attach volume to VM.
>> > > >> While for some other storage vendor, may want to bypass hypervisor
>> > > during
>> > > >> attaching volume, so inside driver's attachvolume implementation,
>> the
>> > > >> driver can do some magic, such as, directly talk to an agent
>> inside VM
>> > > >> instance, then create a disk inside VM.
>> > > >>
>> > > >> How do you guys think?
>> > > >>
>> > > >>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
>> > > wrote:
>> > > >>>
>> > > >>>>
>> > > >>>>
>> > > >>>>> -----Original Message-----
>> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
>> > > >>>>> To: dev@cloudstack.apache.org
>> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> > > >>>>>
>> > > >>>>> Hi John,
>> > > >>>>>
>> > > >>>>> How's about this:
>> > > >>>>>
>> > > >>>>> The driver can implement an isManaged() method. The
>> > > >>> VolumeManagerImpl
>> > > >>>>> can
>> > > >>>>> call into the driver to see if its managed. If it is, the
>> > > >>>> VolumeManagerImpl
>> > > >>>>> (which is responsible for calling into the hypervisor to attach
>> the
>> > > >> disk)
>> > > >>>>> can call into the hypervisor to create the necessary hypervisor
>> data
>> > > >>>>> structure (ex. for XenServer, a storage repository).
>> > > >>>>
>> > > >>>> The problem here is that storage vendor may work differently with
>> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while
>> maybe
>> > > >> other
>> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
>> > directly
>> > > >> to
>> > > >>>> VM instance, see the discuss(
>> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>> > > >>>
>> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>> > > >>> %3E
>> > > >>>> ).
>> > > >>>> So I would let storage provider to implement attach disk to VM,
>> > > >> instead of
>> > > >>>> implemented by cloudstack itself.
>> > > >>>>
>> > > >>>>
>> > > >>>>>
>> > > >>>>> If that's what you're going for, that works for me. By the way,
>> > > >> Edison's
>> > > >>>>> default storage plug-in (which handles the default storage
>> behavior
>> > > >> in
>> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that
>> talks to
>> > > >>>>> hypervisors. You might want to contact him and inform him of
>> your
>> > > >>>> concerns
>> > > >>>>> or that logic (as is) will make it to production.
>> > > >>>>>
>> > > >>>>> Please let me know if what I wrote in above (for my solution)
>> is OK
>> > > >> with
>> > > >>>>> you. :)
>> > > >>>>>
>> > > >>>>> Thanks!
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
>> jburwell@basho.com>
>> > > >>> wrote:
>> > > >>>>>
>> > > >>>>>> Mike,
>> > > >>>>>>
>> > > >>>>>> Please see my responses in-line below.
>> > > >>>>>>
>> > > >>>>>> Thanks,
>> > > >>>>>> -John
>> > > >>>>>>
>> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>> > > >>>> mike.tutkowski@solidfire.com>
>> > > >>>>>> wrote:
>> > > >>>>>>
>> > > >>>>>>> Hey John,
>> > > >>>>>>>
>> > > >>>>>>> I still have a bit more testing I'd like to do before I build
>> up
>> > > >> a
>> > > >>>> patch
>> > > >>>>>>> file, but this is the gist of what I've done:
>> > > >>>>>>>
>> > > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
>> > tells
>> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
>> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if
>> it
>> > > >>>>> managed.
>> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
>> > > >> whatever
>> > > >>>>>>> activity is required. In my case, this includes sending a
>> > > >> message to
>> > > >>>> the
>> > > >>>>>>> host where the VM is running to have, say XenServer, add a
>> > > >> storage
>> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
>> > > >> SAN
>> > > >>>>>> volume,
>> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
>> can
>> > > >> on
>> > > >>>> the
>> > > >>>>>>> storage repository). After this, the normal attach-volume
>> > > >> message is
>> > > >>>> sent
>> > > >>>>>>> to the host by VolumeManagerImpl.
>> > > >>>>>>
>> > > >>>>>> There should be **no** code from a storage driver to a
>> hypervisor.
>> > > >> I
>> > > >>>>>> apologize for the repetition, but we simply can not have
>> hypervisor
>> > > >>>>>> specific code in the storage layer. The circular dependencies
>> > > >> between
>> > > >>>> the
>> > > >>>>>> two layers are not sustainable in the long term. Either the
>> > > >>>> VirtualManager
>> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>> > > >>> coordinate
>> > > >>>>>> volume creation and then populating the SR. Ideally, we can
>> > > >>>> generalize the
>> > > >>>>>> process flow for attaching volumes such that the Xen hypervisor
>> > > >> plugin
>> > > >>>>>> would only implement callbacks to perform the attach action and
>> > > >> create
>> > > >>>>> the
>> > > >>>>>> structure and SR. To my mind, the SolidFire driver should
>> only be
>> > > >>>>>> allocating space and providing information about contents (e.g.
>> > > >> space
>> > > >>>>>> available, space consumed, streams to a URI, file handle for a
>> URI,
>> > > >>>> etc)
>> > > >>>>>> and capabilities.
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>>>
>> > > >>>>>>> * The reverse is performed for a detach-volume command.
>> > > >>>>>>>
>> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
>> driver.
>> > > >>>>>> Edison's
>> > > >>>>>>> default driver simply does a "return false;". We could add a
>> new
>> > > >>>>>> parameter
>> > > >>>>>>> to the createStoragePool API command, if we want, to remove
>> > the
>> > > >>>>>> hard-coded
>> > > >>>>>>> return values in the drivers (although my driver will probably
>> > > >> just
>> > > >>>>>> ignore
>> > > >>>>>>> this parameter and always return true since it wouldn't make
>> > > >> sense
>> > > >>>> for it
>> > > >>>>>>> to ever return false). We'd need another column in the
>> > > >> storage_pool
>> > > >>>>> table
>> > > >>>>>>> to store this value.
>> > > >>>>>>
>> > > >>>>>> Yes, I think we should have a parameter added to the
>> > > >>> createStoragePool
>> > > >>>>>> surfaced to the HTTP API that allows DataStores to be
>> configured
>> > > >> for
>> > > >>>>>> management when their underlying drivers support it. To
>> simplify
>> > > >>>> things,
>> > > >>>>>> this flag should only be mutable when the DataStore is
>> created. It
>> > > >>>> would be
>> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
>> > > >>>> creation.
>> > > >>>>>>
>> > > >>>>>>>
>> > > >>>>>>> Sound like I'm in sync with what you were thinking?
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>> > > >>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>> wrote:
>> > > >>>>>>>
>> > > >>>>>>>> I agree, John. Just wanted to point out that I have a working
>> > > >> GUI
>> > > >>>> for
>> > > >>>>>> you
>> > > >>>>>>>> to review (in that document), if you'd like to check it out.
>> > > >>>>>>>>
>> > > >>>>>>>> Thanks!
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>> > > >> jburwell@basho.com>
>> > > >>>>>> wrote:
>> > > >>>>>>>>
>> > > >>>>>>>>> Mike,
>> > > >>>>>>>>>
>> > > >>>>>>>>> I would like the UIs of two features reviewed together to
>> > > >> ensure
>> > > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
>> > > >> and
>> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
>> > > >>>> confusion,
>> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
>> > > >> help
>> > > >>>>>>>>> minimize any potential confusion.
>> > > >>>>>>>>>
>> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
>> itself
>> > > >> if
>> > > >>>> it
>> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
>> > > >>>> throttled
>> > > >>>>>>>>> IOPs or storage provisioned IOPs.
>> > > >>>>>>>>>
>> > > >>>>>>>>> Thanks,
>> > > >>>>>>>>> -John
>> > > >>>>>>>>>
>> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>> > > >>>>>>>>> <mi...@solidfire.com> wrote:
>> > > >>>>>>>>>
>> > > >>>>>>>>>> Hi John,
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
>> > to
>> > > >>> you
>> > > >>>>> the
>> > > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
>> > > >> IOPS.
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
>> > > >> and we
>> > > >>>>>>>>> decided
>> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
>> > Offering
>> > > >>>> dialog.
>> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
>> > should
>> > > >>> be
>> > > >>>>> able
>> > > >>>>>> to
>> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only
>> use
>> > > >> Max
>> > > >>>>>> IOPS).
>> > > >>>>>>>>>> These fields are optional and can be ignored for storage
>> that
>> > > >>>> does not
>> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
>> the
>> > > >>>> admin can
>> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
>> > > >> IOPS.
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
>> > > >> mine,
>> > > >>>> as
>> > > >>>>>> is
>> > > >>>>>>>>>> decided by the admin).
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> I'm not sure what we can do about these two features
>> > > >> expressing
>> > > >>>> the
>> > > >>>>>>>>> speed
>> > > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS
>> for
>> > > >>>> QoS in
>> > > >>>>>>>>> any
>> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
>> read/write
>> > > >>>> IOPS).
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Thanks
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>> > > >>> <jb...@basho.com>
>> > > >>>>>>>>> wrote:
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
>> > > >>> Storage/Hypervisor
>> > > >>>>>> layer
>> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
>> > > >>>> hypervisor
>> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
>> > > >> following
>> > > >>>>>> issues
>> > > >>>>>>>>>>> outstanding:
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>> > > >>> read/write
>> > > >>>>>>>>> values
>> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
>> > > >> IOPS.
>> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
>> > > >>> storage
>> > > >>>>>> volumes
>> > > >>>>>>>>>>> with provisioned IOPS?
>> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
>> > > >>>> provisioned
>> > > >>>>>> IOPS
>> > > >>>>>>>>>>> that will permit providers to discriminate these
>> guaranteed
>> > > >>>>>> operations
>> > > >>>>>>>>> in
>> > > >>>>>>>>>>> the event they want to bill for it differently?
>> > > >>>>>>>>>>> 4. What is the user experience for throttled and
>> provisioned
>> > > >>> IOPS
>> > > >>>>>> that
>> > > >>>>>>>>>>> minimizes confusion of these concepts?
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> My thinking is that a VM can have either utilize
>> hypervisor
>> > > >>>> throttled
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
>> solve
>> > > >>>> items 1
>> > > >>>>>>>>> and 2.
>> > > >>>>>>>>>>> Since the two facilities would not be permitted to operate
>> > > >>>> together,
>> > > >>>>>>>>> they
>> > > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
>> > > >> can be
>> > > >>>>>>>>> resolved
>> > > >>>>>>>>>>> with an additional flag or two on the usage records. As
>> for
>> > > >>>> Item 4,
>> > > >>>>>> I
>> > > >>>>>>>>> am
>> > > >>>>>>>>>>> not familiar with how these two enhancements are
>> > depicted in
>> > > >>> the
>> > > >>>>> user
>> > > >>>>>>>>>>> interface. I think we need to review the UI enhancements
>> > for
>> > > >>>> both
>> > > >>>>>>>>>>> enhancements and ensure they are consistent.
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> Do these solutions make sense?
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>> -John
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>> > <us...@gmail.com>
>> > > >>>>> wrote:
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>> John and Mike,
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
>> > 2780/2729,
>> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>> > > >>> CLOUDSTACK-
>> > > >>>>> 2875 ) this
>> > > >>>>>>>>> week.
>> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
>> > > >> ask
>> > > >>>> for
>> > > >>>>>>>>> second
>> > > >>>>>>>>>>>> merge request after testing.
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
>> John,
>> > > >> and
>> > > >>>>> have
>> > > >>>>>>>>> been
>> > > >>>>>>>>>>>>> re-working this section of code today.
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> Thanks for the comments,
>> > > >>>>>>>>>>>>> Mike
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>> > > >>>>> <jb...@basho.com>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> See my responses in-line below.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks for the response.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
>> > whether or
>> > > >>>>> not a
>> > > >>>>>>>>> Volume
>> > > >>>>>>>>>>> is
>> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>> > > >>> actual
>> > > >>>>>>>>> isManaged
>> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
>> > > >> resources
>> > > >>>> for
>> > > >>>>>> the
>> > > >>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>> if it is managed."
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
>> > method
>> > > >>>>> added to
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
>> > > >> then the
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
>> > > >>> method
>> > > >>>>> (which
>> > > >>>>>>>>>>> would
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
>> > and
>> > > >>> this
>> > > >>>>>> would
>> > > >>>>>>>>>>> call
>> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
>> > on
>> > > >>>>>>>>> XenServer,
>> > > >>>>>>>>>>> an
>> > > >>>>>>>>>>>>> SR?
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>> > > >>>>> DataStoreDriver.
>> > > >>>>>>>>> Since
>> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
>> > I
>> > > >>>> would
>> > > >>>>>> add
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
>> > straw
>> > > >>> man
>> > > >>>>> term
>> > > >>>>>>>>> --
>> > > >>>>>>>>>>>>>> something other than volume) methods to
>> > > >>>>>>>>>>> allocate/create/deallocate/delete
>> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
>> > > >>> property
>> > > >>>>> of
>> > > >>>>>>>>>>>>> DataStore
>> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
>> > > >>> DataStoreDriver
>> > > >>>>> can be
>> > > >>>>>>>>>>>>> managed.
>> > > >>>>>>>>>>>>>> This approach allows operators to override
>> > manageability
>> > > >> of
>> > > >>>>>> devices.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
>> > > >> plugin
>> > > >>>>>> would
>> > > >>>>>>>>> be
>> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
>> > to
>> > > >>>>> create any
>> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There
>> should
>> > > >> be no
>> > > >>>>>>>>>>>>> dependencies
>> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
>> > > >> earlier,
>> > > >>>> such
>> > > >>>>>>>>>>>>> circular
>> > > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
>> > mess.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks again, John
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>> > > >>>>> <jb...@basho.com>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
>> > that
>> > > >>>>>> supports n
>> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
>> > > >> hypervisors.
>> > > >>>>>> Such
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
>> > > >>> beast.
>> > > >>>>>>>>>>>>> Therefore, my
>> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
>> > the
>> > > >>>>> Storage
>> > > >>>>>> layer
>> > > >>>>>>>>>>>>> towards
>> > > >>>>>>>>>>>>>>> this separation.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
>> > whether
>> > > >>> or
>> > > >>>>> not a
>> > > >>>>>>>>> Volume
>> > > >>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
>> > an
>> > > >>> actual
>> > > >>>>>>>>>>> isManaged
>> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
>> > > >> resources
>> > > >>>> for
>> > > >>>>>> the
>> > > >>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
>> needs
>> > > >> to
>> > > >>>>>> indicate
>> > > >>>>>>>>>>>>> whether
>> > > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
>> > > >> think
>> > > >>>> we
>> > > >>>>>>>>> need to
>> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to
>> have
>> > > >>>>>> something
>> > > >>>>>>>>>>>>> that is
>> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
>> > > >>> managing
>> > > >>>>> the
>> > > >>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
>> > unmanaged).
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
>> > in
>> > > >>> the
>> > > >>>>> Xen
>> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
>> > Storage
>> > > >>>>> layer to
>> > > >>>>>>>>>>> perform
>> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
>> > > >> file,
>> > > >>>> etc).
>> > > >>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>> > > >>>> hypervisor, as
>> > > >>>>>>>>> such,
>> > > >>>>>>>>>>>>>>> should be confined to its implementation. From my
>> > > >>>>> perspective,
>> > > >>>>>>>>>>> nothing
>> > > >>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
>> > From
>> > > >>> its
>> > > >>>>>>>>>>>>> perspective,
>> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
>> > to
>> > > >>>> query
>> > > >>>>>> the
>> > > >>>>>>>>>>> data
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
>> > level
>> > > >>>>> layers
>> > > >>>>>>>>> (e.g.
>> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
>> > composition
>> > > >>> of
>> > > >>>>>>>>>>> operations
>> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
>> > Volume
>> > > >>>>> storage
>> > > >>>>>>>>>>> protocol
>> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
>> > implemented
>> > > >> in
>> > > >>>> my
>> > > >>>>>>>>> patch,
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>> could do the following:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>> > > >>>>>>>>> VolumeManagerImpl
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
>> > > >>>>> hypervisor.
>> > > >>>>>> Before
>> > > >>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
>> > dealing
>> > > >>> with
>> > > >>>>>>>>> Dynamic
>> > > >>>>>>>>>>> (or
>> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
>> send a
>> > > >>>> "Create
>> > > >>>>>>>>> SR"
>> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
>> > would
>> > > >>> then
>> > > >>>>>>>>> proceed
>> > > >>>>>>>>>>> to
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
>> > > >> add
>> > > >>>>>> another
>> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
>> > > >> particular
>> > > >>>>> type
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>> storage.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> What do you think?
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> Hey John,
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
>> > outlines
>> > > >>> how
>> > > >>>>> the
>> > > >>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be
>> of
>> > > >> use
>> > > >>>> to
>> > > >>>>>>>>> you, as
>> > > >>>>>>>>>>>>> well,
>> > > >>>>>>>>>>>>>>>> as you perform the code review.
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> I have shared this document with you (you should
>> > have
>> > > >>>>> received
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> Talk to you later!
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>> > > >>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
>> > thoughts,
>> > > >>> but
>> > > >>>>>> want
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
>> solidfire,
>> > > >>>>>>>>>>>>> disk_io_throttle,
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
>> > > >>>>> coherent
>> > > >>>>>>>>> form
>> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
>> > the
>> > > >>>> notion
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
>> > > >> the
>> > > >>>>> notion
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
>> > was
>> > > >>>>>>>>> incomplete,
>> > > >>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>> I had
>> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
>> > > >> device.
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
>> > > >> this.
>> > > >>>> :)
>> > > >>>>> Do
>> > > >>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>> have a
>> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
>> > > >>> what
>> > > >>>>> you're
>> > > >>>>>>>>> looking
>> > > >>>>>>>>>>>>>>>>>> for?
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>> > > >>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
>> > the
>> > > >>>> notion
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
>> > > >>>>> capabilities
>> > > >>>>>>>>> (i.e.
>> > > >>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
>> > > >>> managed
>> > > >>>>> by
>> > > >>>>>>>>> CloudStack)
>> > > >>>>>>>>>>>>>>>>>> from the
>> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
>> > a
>> > > >>>>> semantic
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
>> > I
>> > > >>> also
>> > > >>>>> want
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
>> > the
>> > > >>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> definitions.
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>> > > >>>>>>>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
>> > questions
>> > > >>> for
>> > > >>>>> me
>> > > >>>>>> with
>> > > >>>>>>>>>>>>>>>>>> regards to
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
>> > and
>> > > >>> SAN
>> > > >>>>>> volumes?
>> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
>> > (LUN
>> > > >>> 0).
>> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
>> > and
>> > > >>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> Volumes?
>> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
>> > > >>> server or
>> > > >>>>>>>>> externally
>> > > >>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> operator?
>> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
>> > > >>> volume
>> > > >>>>> (same
>> > > >>>>>> as a
>> > > >>>>>>>>>>> SAN
>> > > >>>>>>>>>>>>>>>>>> LUN) is
>> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
>> > plug-in)
>> > > >>>>> the
>> > > >>>>>> first
>> > > >>>>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
>> > but
>> > > >>>> still
>> > > >>>>>>>>> want
>> > > >>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> use a
>> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
>> > > >>> today
>> > > >>>>>> (prior
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> 4.2). The
>> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
>> > in
>> > > >>> this
>> > > >>>>> case -
>> > > >>>>>>>>>>>>>>>>>> multiple VMs
>> > > >>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
>> > you
>> > > >>> can do
>> > > >>>>> this
>> > > >>>>>>>>>>>>> today,
>> > > >>>>>>>>>>>>>>>>>> it is
>> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
>> > server
>> > > >>> or
>> > > >>>>> externally
>> > > >>>>>>>>> by
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> operator?
>> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
>> > > >>> volumes
>> > > >>>>> are
>> > > >>>>>>>>>>>>> completely
>> > > >>>>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
>> > There
>> > > >>> is
>> > > >>>>> no
>> > > >>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>> interaction.
>> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
>> > > >>> volume
>> > > >>>>> and a
>> > > >>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
>> > > >>>>> supports
>> > > >>>>>> true
>> > > >>>>>>>>>>> QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
>> > > >> related
>> > > >>>>> and
>> > > >>>>>>>>>>>>> expected
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> operate.
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
>> > multiple
>> > > >>> VMs
>> > > >>>>> with
>> > > >>>>>>>>>>>>>>>>>> provisioned
>> > > >>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
>> > > >>> single
>> > > >>>>> SAN
>> > > >>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>> (LUN) to a
>> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
>> > > >>> multiple
>> > > >>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
>> > guarantee
>> > > >>> QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
>> > volume
>> > > >>> house
>> > > >>>>> more
>> > > >>>>>>>>> than
>> > > >>>>>>>>>>>>> one
>> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
>> > > >>> without
>> > > >>>>> any of
>> > > >>>>>> my
>> > > >>>>>>>>>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>>>> code.
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
>> > Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
>> > volume
>> > > >>> for
>> > > >>>>>>>>>>> CloudStack's
>> > > >>>>>>>>>>>>>>>>>> use
>> > > >>>>>>>>>>>>>>>>>>>> onto
>> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
>> > Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
>> > concerns me
>> > > >>> --
>> > > >>>>>> concepts
>> > > >>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
>> > Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
>> > today
>> > > >>>> only
>> > > >>>>>> used
>> > > >>>>>>>>>>> (as
>> > > >>>>>>>>>>>>>>>>>> far as
>> > > >>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
>> > not
>> > > >>>>> PreSetup an
>> > > >>>>>>>>> SR).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
>> > from
>> > > >>>>> VMware, it
>> > > >>>>>>>>> uses
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> vmfs
>> > > >>>>>>>>>>>>>>>>>>>>>> type.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
>> > from
>> > > >>> KVM,
>> > > >>>>> it uses
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
>> > > >>> here
>> > > >>>>> was to
>> > > >>>>>>>>> make a
>> > > >>>>>>>>>>>>>>>>>> new type
>> > > >>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
>> > > >>> Edison
>> > > >>>>> added
>> > > >>>>>>>>> into
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
>> > > >>> specificy,
>> > > >>>>> though:
>> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
>> > > >>> Tutkowski
>> > > >>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
>> > know
>> > > >>> or
>> > > >>>>> care
>> > > >>>>>> that
>> > > >>>>>>>>> it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>> used
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
>> > > >>>>> questions
>> > > >>>>>>>>> about
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
>> > > >>>>> SolidFire
>> > > >>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>> needs to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
>> > > >>> hypervisor
>> > > >>>>> logic
>> > > >>>>>>>>>>> knows
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>> treat
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
>> > constraint
>> > > >>>>> and just
>> > > >>>>>>>>>>>>>>>>>> calling it
>> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
>> > for
>> > > >>> the
>> > > >>>>> hypervisor
>> > > >>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
>> > perform
>> > > >>> the
>> > > >>>>> necessary
>> > > >>>>>>>>>>>>>>>>>> activities.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
>> > > >>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>> > > >>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
>> > able to
>> > > >>>>> manage a
>> > > >>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>> > > >>> CloudStack
>> > > >>>>> (e.g. the
>> > > >>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>> shared
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
>> > > >>> chosen
>> > > >>>>> to
>> > > >>>>>>>>> dedicate
>> > > >>>>>>>>>>>>>>>>>> only a
>> > > >>>>>>>>>>>>>>>>>>>>>> portion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
>> > > >>> make
>> > > >>>>> sense?"
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
>> > SAN
>> > > >>>>> volume via
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
>> > would
>> > > >>>>> work unless
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
>> > an
>> > > >>> SR.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
>> > > >>>>> CloudStack, an
>> > > >>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>> allocates
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
>> > > >>> exposes it
>> > > >>>>>> through
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> LUN. They
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
>> > to
>> > > >>>>> CloudStack,
>> > > >>>>>>>>> but
>> > > >>>>>>>>>>>>>>>>>> not allow
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
>> > > >>>>> changes, we
>> > > >>>>>>>>> would
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>> passing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
>> > > >>>>> necessary
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>> (like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
>> > have
>> > > >>> the
>> > > >>>>> storage
>> > > >>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>> itself
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
>> > support
>> > > >>> from
>> > > >>>>> the
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> side and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
>> > to
>> > > >>> create
>> > > >>>>> the
>> > > >>>>>>>>> SR
>> > > >>>>>>>>>>>>>>>>>> before the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
>> > plugin
>> > > >>>>> should
>> > > >>>>>> have a
>> > > >>>>>>>>>>>>> rich
>> > > >>>>>>>>>>>>>>>>>> enough
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
>> > > >>> available
>> > > >>>>> for
>> > > >>>>>> volume
>> > > >>>>>>>>>>>>>>>>>> storage.
>> > > >>>>>>>>>>>>>>>>>>>> For
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>> > > >>> interrogation of
>> > > >>>>> the
>> > > >>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
>> > itself
>> > > >>>>> shouldn't
>> > > >>>>>>>>> know
>> > > >>>>>>>>>>>>>>>>>> or care
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
>> > able
>> > > >>> to
>> > > >>>>>> answer
>> > > >>>>>>>>>>>>>>>>>> questions
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
>> > > >>> driver
>> > > >>>>> (which
>> > > >>>>>>>>> is a
>> > > >>>>>>>>>>>>>>>>>> type of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
>> > done, a
>> > > >>> new
>> > > >>>>> Primary
>> > > >>>>>>>>>>>>>>>>>> Storage shows
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
>> > > >>> that
>> > > >>>>>> references
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>> tag
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
>> > CloudStack
>> > > >>>>> volume. This
>> > > >>>>>>>>> leads
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> a new
>> > > >>>>>>>>>>>>>>>>>>>>>> row
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
>> > CloudStack
>> > > >>>>> volume to a
>> > > >>>>>>>>> VM
>> > > >>>>>>>>>>>>>>>>>> (attach
>> > > >>>>>>>>>>>>>>>>>>>>>> disk).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
>> > calling
>> > > >>> the
>> > > >>>>> plug-in
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> create a
>> > > >>>>>>>>>>>>>>>>>>>> new
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
>> > > >>> SAN).
>> > > >>>>> The
>> > > >>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>> also
>> > > >>>>>>>>>>>>>>>>>>>>>> updates
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
>> > data
>> > > >>>>> (like the
>> > > >>>>>>>>> IQN of
>> > > >>>>>>>>>>>>>>>>>> the SAN
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
>> > the
>> > > >>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> volume is in
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
>> > the
>> > > >>>>> volume
>> > > >>>>>>>>> will be
>> > > >>>>>>>>>>>>>>>>>> in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
>> > the
>> > > >>>>> plug-in
>> > > >>>>>>>>> code
>> > > >>>>>>>>>>>>>>>>>> will not
>> > > >>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
>> > and
>> > > >>>>> detects
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
>> > form
>> > > >>> of a
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
>> > > >>>>> terminology,
>> > > >>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>> in these
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
>> > > >>>>> allocation on
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
>> > > >>> mean
>> > > >>>>> 'SAN
>> > > >>>>>>>>> volume'.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
>> > user
>> > > >>> can
>> > > >>>>> make in
>> > > >>>>>>>>>>>>>>>>>> CloudStack, I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
>> > well
>> > > >>> make
>> > > >>>>> sense, but
>> > > >>>>>>>>> I'm
>> > > >>>>>>>>>>>>>>>>>> having a
>> > > >>>>>>>>>>>>>>>>>>>>>> hard
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
>> > in on
>> > > >>>>> this
>> > > >>>>>>>>> conversation
>> > > >>>>>>>>>>>>>>>>>> as he
>> > > >>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>> > > >>> approach I
>> > > >>>>> took.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
>> > > >>> John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
>> > two
>> > > >>>>> distinct
>> > > >>>>>>>>> concepts
>> > > >>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
>> > protocols.
>> > > >>> In
>> > > >>>>> both
>> > > >>>>>>>>> cases, we
>> > > >>>>>>>>>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
>> > system to
>> > > >>>>>> create/delete
>> > > >>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
>> > the
>> > > >>>>> volume to
>> > > >>>>>> be
>> > > >>>>>>>>>>>>>>>>>> volume to be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
>> > context.
>> > > >>> To
>> > > >>>>> ensure
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>> are in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
>> > > >>>>> definitions,
>> > > >>>>>>>>> refers to
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> physical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
>> > correct?
>> > > >>>>> Minimally,
>> > > >>>>>> we
>> > > >>>>>>>>>>>>> must
>> > > >>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>> able
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
>> > device
>> > > >>> to
>> > > >>>>> move bits
>> > > >>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>> one place
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>> > > >>> Optionally, a
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
>> > Therefore,
>> > > >>>>> we can have
>> > > >>>>>> a
>> > > >>>>>>>>>>>>>>>>>> unmanaged
>> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
>> > and
>> > > >>> we
>> > > >>>>> can have a
>> > > >>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
>> > > >>> capable
>> > > >>>>> of
>> > > >>>>>>>>> allocating
>> > > >>>>>>>>>>>>>>>>>> LUNs and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
>> > CloudStack
>> > > >>> may
>> > > >>>>> be able
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> manage a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
>> > > >>>>> unmanaged by
>> > > >>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> (e.g.
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
>> > the
>> > > >>>>> operator has
>> > > >>>>>>>>> chosen
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
>> > Does
>> > > >>> my
>> > > >>>>>> reasoning
>> > > >>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>> sense?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
>> > are
>> > > >>>>> reasonable, it
>> > > >>>>>>>>> seems
>> > > >>>>>>>>>>>>>>>>>>>> appropriate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
>> > concerns
>> > > >>> from
>> > > >>>>>>>>> StoragePoolType,
>> > > >>>>>>>>>>>>>>>>>> add the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
>> > attached
>> > > >>> driver
>> > > >>>>> that
>> > > >>>>>>>>>>> indicates
>> > > >>>>>>>>>>>>>>>>>> whether
>> > > >>>>>>>>>>>>>>>>>>>>>> or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
>> > > >>> establish
>> > > >>>>> a
>> > > >>>>>>>>> abstraction
>> > > >>>>>>>>>>>>>>>>>>>>>> representing a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
>> > that
>> > > >>> is
>> > > >>>>>>>>> associated
>> > > >>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>> volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
>> > hypervisor
>> > > >>>>> drivers can
>> > > >>>>>>>>>>>>> declare
>> > > >>>>>>>>>>>>>>>>>> which
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
>> > encounter
>> > > >>> a
>> > > >>>>> device
>> > > >>>>>> managed
>> > > >>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
>> > exposed
>> > > >>> by
>> > > >>>>> the driver to
>> > > >>>>>>>>>>>>> automate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
>> > > >>> sense,
>> > > >>>>> then we can
>> > > >>>>>>>>> sit
>> > > >>>>>>>>>>>>>>>>>> down and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
>> > between
>> > > >>> the
>> > > >>>>> current
>> > > >>>>>> iSCSI
>> > > >>>>>>>>>>>>> type
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
>> > go in
>> > > >>> and
>> > > >>>>> create
>> > > >>>>>> a
>> > > >>>>>>>>>>>>>>>>>> Primary
>> > > >>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
>> > time,
>> > > >>>>> the
>> > > >>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>> volume must
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
>> > allocated).
>> > > >>>>> Future
>> > > >>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
>> > created
>> > > >>>>> behind the
>> > > >>>>>>>>>>>>> scenes.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
>> > to
>> > > >>> go in
>> > > >>>>> and
>> > > >>>>>> create
>> > > >>>>>>>>>>>>>>>>>> Primary
>> > > >>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
>> > delete
>> > > >>>>> volumes on
>> > > >>>>>>>>> its
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
>> > the
>> > > >>>>> storage
>> > > >>>>>>>>>>>>>>>>>> framework). When
>> > > >>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
>> > that
>> > > >>> was
>> > > >>>>> created,
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>>>> tells
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
>> > volume.
>> > > >>> After
>> > > >>>>> this is
>> > > >>>>>>>>> done,
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
>> > called.
>> > > >>>>> No
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
>> > volume
>> > > >>> was
>> > > >>>>> just
>> > > >>>>>>>>> created.
>> > > >>>>>>>>>>>>> The
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
>> > PM,
>> > > >>> Mike
>> > > >>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
>> > should
>> > > >>>>> probably
>> > > >>>>>> have
>> > > >>>>>>>>>>>>>>>>>> defined them
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
>> > > >>> pre-
>> > > >>>>> allocated
>> > > >>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>> an admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
>> > > >>> hypervisor
>> > > >>>>> data
>> > > >>>>>>>>> structure
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
>> > SR),
>> > > >>> then
>> > > >>>>> that
>> > > >>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
>> > Disks
>> > > >>> (VDI)
>> > > >>>>> are later
>> > > >>>>>>>>>>>>> placed
>> > > >>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
>> > > >>> these
>> > > >>>>> cases, the
>> > > >>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
>> > > >>> already
>> > > >>>>> in place
>> > > >>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> simply
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
>> > structure
>> > > >>> to
>> > > >>>>> the VM in
>> > > >>>>>>>>>>>>>>>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
>> > that is
>> > > >>> not
>> > > >>>>>>>>>>> pre-allocated.
>> > > >>>>>>>>>>>>>>>>>> Instead
>> > > >>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
>> > could
>> > > >>> be a
>> > > >>>>> SAN
>> > > >>>>>> (not
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> volume on
>> > > >>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
>> > data
>> > > >>>>> structure
>> > > >>>>>>>>> must be
>> > > >>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
>> > because
>> > > >>> these
>> > > >>>>> types of
>> > > >>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>> have not
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
>> > > >>>>> structure by an
>> > > >>>>>>>>>>> admin.
>> > > >>>>>>>>>>>>>>>>>> Once
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
>> > XenServer
>> > > >>> for
>> > > >>>>> this
>> > > >>>>>>>>> volume,
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> attaches
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
>> > VM
>> > > >>> in
>> > > >>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
>> > PM,
>> > > >>>>> John Burwell
>> > > >>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>> > implementation of
>> > > >>> the
>> > > >>>>> Dynamic
>> > > >>>>>> type
>> > > >>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>> behavior
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
>> > why I
>> > > >>> ask
>> > > >>>>> about the
>> > > >>>>>>>>>>>>>>>>>> difference.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
>> > the
>> > > >>>>> definition
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> Dynamic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
>> > > >>> Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
>> > type
>> > > >>> is
>> > > >>>>>> uniquely
>> > > >>>>>>>>>>> used
>> > > >>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
>> > > >>> Storage
>> > > >>>>> that is
>> > > >>>>>>>>> directly
>> > > >>>>>>>>>>>>>>>>>> based on
>> > > >>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
>> > > >>> step of
>> > > >>>>> going
>> > > >>>>>> to
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
>> > on
>> > > >>> that
>> > > >>>>> iSCSI
>> > > >>>>>>>>> target as
>> > > >>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
>> > > >>>>> supported for
>> > > >>>>>>>>>>>>>>>>>> XenServer.
>> > > >>>>>>>>>>>>>>>>>>>> For
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
>> > first go
>> > > >>> to
>> > > >>>>> the
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>> perform
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
>> > RBD is.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> > 2:13
>> > > >>> PM,
>> > > >>>>> John
>> > > >>>>>> Burwell
>> > > >>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
>> > code,
>> > > >>> what
>> > > >>>>> is the
>> > > >>>>>>>>>>> difference
>> > > >>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
>> > isn't
>> > > >>> RBD
>> > > >>>>> considered
>> > > >>>>>>>>>>>>> Dynamic?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
>> > PM,
>> > > >>> Mike
>> > > >>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
>> > storage is
>> > > >>>>> defined in
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>> > > >>>>> StoragePoolType {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
>> > local
>> > > >>>>> directory
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > NetworkFilesystem(true),
>> > > >>> //
>> > > >>>>> NFS or CIFS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
>> > shared
>> > > >>> LUN,
>> > > >>>>> with a
>> > > >>>>>>>>>>> clusterfs
>> > > >>>>>>>>>>>>>>>>>>>> overlay
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
>> > ZFS
>> > > >>>>> Comstar
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
>> > image
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
>> > XenServer
>> > > >>> local
>> > > >>>>> LVM SR
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > SharedMountPoint(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
>> > VMware
>> > > >>> VMFS
>> > > >>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
>> > > >>>>> XenServer, Storage
>> > > >>>>>>>>> Pool
>> > > >>>>>>>>>>>>>>>>>> is set
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
>> > XenServer
>> > > >>> local
>> > > >>>>> EXT SR
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
>> > dynamic,
>> > > >>>>> zone-wide
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > StoragePoolType(boolean
>> > > >>>>> shared) {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
>> > isShared() {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> > 1:41
>> > > >>> PM,
>> > > >>>>> Mike
>> > > >>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
>> > > >>> another
>> > > >>>>> storage
>> > > >>>>>>>>>>> company
>> > > >>>>>>>>>>>>>>>>>> wants
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
>> > > >>> Quality
>> > > >>>>> of
>> > > >>>>>> Service
>> > > >>>>>>>>>>>>>>>>>> feature. It
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
>> > well.
>> > > >>>>> They would
>> > > >>>>>>>>> need
>> > > >>>>>>>>>>>>> only
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
>> > the
>> > > >>>>> necessary
>> > > >>>>>>>>> changes to
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
>> > in.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> > at
>> > > >>> 1:39
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
>> > the
>> > > >>>>> XenServer code
>> > > >>>>>> is
>> > > >>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
>> > attach
>> > > >>> logic is
>> > > >>>>> now aware
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>> dynamic,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
>> > > >>>>> implementation of
>> > > >>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>> kind of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
>> > > >>> with
>> > > >>>>> Edison's
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
>> > new
>> > > >>>>> framework that
>> > > >>>>>>>>>>> supported
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
>> > dynamically.
>> > > >>>>> However, when I
>> > > >>>>>>>>>>>>>>>>>> visited with
>> > > >>>>>>>>>>>>>>>>>>>>>> him
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
>> > that
>> > > >>> it
>> > > >>>>> was not
>> > > >>>>>>>>>>>>>>>>>> complete. We
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
>> > could
>> > > >>> do
>> > > >>>>> with these
>> > > >>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>> unless
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
>> > recognize
>> > > >>> this
>> > > >>>>> new type of
>> > > >>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>> create
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>> > structure.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> > at
>> > > >>> 1:28
>> > > >>>>> PM, John
>> > > >>>>>>>>> Burwell
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
>> > to me
>> > > >>>>> that any
>> > > >>>>>>>>>>> operation
>> > > >>>>>>>>>>>>>>>>>> in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
>> > > >>> about
>> > > >>>>> details. I
>> > > >>>>>>>>> expect
>> > > >>>>>>>>>>>>>>>>>> to see
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
>> > > >>> operations
>> > > >>>>> that can be
>> > > >>>>>>>>>>>>>>>>>>>> composed/driven
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
>> > > >>>>> allocate
>> > > >>>>>> space/create
>> > > >>>>>>>>>>>>>>>>>> structures
>> > > >>>>>>>>>>>>>>>>>>>>>> per
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
>> > > >>>>> dependency, we are
>> > > >>>>>>>>> going
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> end
>> > > >>>>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
>> > make
>> > > >>> the
>> > > >>>>> system
>> > > >>>>>>>>>>>>> increasingly
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
>> > > >>>>> understanding that the
>> > > >>>>>>>>> Xen
>> > > >>>>>>>>>>>>>>>>>> specific
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>> > > >>>>> CitrixResourceBase
>> > > >>>>>> class?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013 at
>> > > >>> 3:12
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
>> > in a
>> > > >>> bit
>> > > >>>>> more
>> > > >>>>>>>>> detail:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
>> > aside
>> > > >>> from
>> > > >>>>> one setup
>> > > >>>>>>>>>>>>> method
>> > > >>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
>> > > >>> volume on
>> > > >>>>> the
>> > > >>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> system,
>> > > >>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>> go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
>> > > >>> structure
>> > > >>>>> to make
>> > > >>>>>> use
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>> > > >>> XenServer
>> > > >>>>> or a
>> > > >>>>>> datastore
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> ESX(i)).
>> > > >>>>>>>>>>>>>>>>>>>>>> VMs
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
>> > storage
>> > > >>>>> system's volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
>> > > >>> storage
>> > > >>>>> framework,
>> > > >>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> need
>> > > >>>>>>>>>>>>>>>>>>>> no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
>> > > >>> create
>> > > >>>>> a 1:1
>> > > >>>>>>>>>>>>> relationship
>> > > >>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
>> > VM's
>> > > >>> data
>> > > >>>>> disk
>> > > >>>>>>>>> (necessary
>> > > >>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
>> > a
>> > > >>> plug-
>> > > >>>>> in that is
>> > > >>>>>>>>> called
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
>> > needed.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
>> > the
>> > > >>>>> storage
>> > > >>>>>> framework
>> > > >>>>>>>>> did
>> > > >>>>>>>>>>>>>>>>>> not
>> > > >>>>>>>>>>>>>>>>>>>>>> address
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>> > > >>> hypervisor-
>> > > >>>>> specific data
>> > > >>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>> when
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
>> > case,
>> > > >>> I've
>> > > >>>>> been
>> > > >>>>>>>>> enhancing
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> to do
>> > > >>>>>>>>>>>>>>>>>>>> so.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
>> > > >>>>> submitted. I've got
>> > > >>>>>>>>> ESX(i)
>> > > >>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> my
>> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
>> > 4.2
>> > > >>>>> freeze date.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
>> > bit? :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013 at
>> > > >>>>> 1:03 PM, Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
>> > plug-in -
>> > > >>> by
>> > > >>>>> itself - is
>> > > >>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
>> > the
>> > > >>>>> volume-attach
>> > > >>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>> (in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
>> > into
>> > > >>> the
>> > > >>>>> plug-in to
>> > > >>>>>>>>> have
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> create a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
>> > > >>> comes to
>> > > >>>>> attach the
>> > > >>>>>>>>> volume
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
>> > > >>> enough
>> > > >>>>> to recognize
>> > > >>>>>>>>> it's
>> > > >>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
>> > volume
>> > > >>> has
>> > > >>>>> just been
>> > > >>>>>>>>> created)
>> > > >>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> create,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
>> > > >>>>> XenServer) or a
>> > > >>>>>>>>> datastore
>> > > >>>>>>>>>>>>>>>>>> (for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
>> > just
>> > > >>>>> created.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
>> > spending
>> > > >>> most
>> > > >>>>> of my time
>> > > >>>>>>>>>>>>> recently
>> > > >>>>>>>>>>>>>>>>>> making
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
>> > code.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
>> > up?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013
>> > > >>> at
>> > > >>>>> 12:48 PM,
>> > > >>>>>> John
>> > > >>>>>>>>>>>>>>>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
>> > > >>> why
>> > > >>>>> the the storage
>> > > >>>>>>>>>>>>> driver
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
>> > at
>> > > >>> 1:21
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
>> > ultimately I
>> > > >>>>> would like to
>> > > >>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>> all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
>> > think
>> > > >>> I'm
>> > > >>>>> just out of
>> > > >>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>> for 4.2
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
>> > > >>> plug-in
>> > > >>>>> supports
>> > > >>>>>>>>>>>>> XenServer.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
>> > > >>> feature
>> > > >>>>> freeze, I
>> > > >>>>>>>>> have
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> working
>> > > >>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
>> > be
>> > > >>> clear,
>> > > >>>>> this is
>> > > >>>>>> all
>> > > >>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> regards
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > Compute
>> > > >>>>> Offerings post
>> > > >>>>>> 4.2.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > > >>> 2013
>> > > >>>>> at 11:14 AM,
>> > > >>>>>>>>> Kelcey
>> > > >>>>>>>>>>>>>>>>>> Jamison
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
>> > plan
>> > > >>> on
>> > > >>>>> supporting
>> > > >>>>>>>>> KVM in
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
>> > > >>>>> Message -----
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
>> > > >>>>> Tutkowski" <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>> > > >>>>> dev@cloudstack.apache.org
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
>> > Monday,
>> > > >>> June
>> > > >>>>> 3, 2013
>> > > >>>>>>>>> 10:12:32 AM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
>> > > >>> [MERGE]
>> > > >>>>>>>>>>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>> MASTER
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
>> > > >>> merging
>> > > >>>>> Wei's feature
>> > > >>>>>>>>>>> first,
>> > > >>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>> mine.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
>> > is
>> > > >>> for
>> > > >>>>> KVM only,
>> > > >>>>>>>>> then
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> is a
>> > > >>>>>>>>>>>>>>>>>>>> non
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
>> > 3,
>> > > >>> 2013
>> > > >>>>> at 8:55 AM,
>> > > >>>>>> Wei
>> > > >>>>>>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
>> > billing,
>> > > >>> as
>> > > >>>>> no one works
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> billing
>> > > >>>>>>>>>>>>>>>>>>>> now,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
>> > by
>> > > >>>>> themselves. They
>> > > >>>>>>>>> can
>> > > >>>>>>>>>>>>> get
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > disk_offering of
>> > > >>> a
>> > > >>>>> VMs and
>> > > >>>>>> volumes
>> > > >>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
>> > the
>> > > >>>>> exact
>> > > >>>>>> limitation
>> > > >>>>>>>>>>>>> value
>> > > >>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>> > limitation
>> > > >>> for
>> > > >>>>> nics as
>> > > >>>>>> well.
>> > > >>>>>>>>> I
>> > > >>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
>> > I/O
>> > > >>>>> throttling?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
>> > > >>>>> misunstand the second
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
>> > > >>> what
>> > > >>>>> you said about
>> > > >>>>>> the
>> > > >>>>>>>>>>> two
>> > > >>>>>>>>>>>>>>>>>>>> features.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>> > John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>> > 2013,
>> > > >>> at
>> > > >>>>> 2:13 AM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > Mike
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
>> > Mike's
>> > > >>>>> aswer helps you.
>> > > >>>>>>>>> I am
>> > > >>>>>>>>>>>>>>>>>> trying
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
>> > think
>> > > >>> billing
>> > > >>>>> should
>> > > >>>>>>>>> depend
>> > > >>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> IO
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > limitation.
>> > > >>> Please
>> > > >>>>> review
>> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
>> > > >>>>>>>>>>>>>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
>> > IO
>> > > >>>>> statistics
>> > > >>>>>> including
>> > > >>>>>>>>>>>>>>>>>> bytes/iops
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
>> > > >>> machine.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
>> > the
>> > > >>> AWS
>> > > >>>>> model,
>> > > >>>>>> customers
>> > > >>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>> billed
>> > > >>>>>>>>>>>>>>>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > provisioned
>> > > >>> IOPS,
>> > > >>>>> as well as,
>> > > >>>>>> for
>> > > >>>>>>>>>>>>> those
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://aws.amazon.com/ebs/).
>> > > >>>>>> I
>> > > >>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>> imagine
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
>> > > >>> employ
>> > > >>>>> similar cost
>> > > >>>>>>>>>>> models.
>> > > >>>>>>>>>>>>>>>>>> Could
>> > > >>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
>> > model
>> > > >>> in
>> > > >>>>> the current
>> > > >>>>>>>>> patch?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
>> > you
>> > > >>> mean
>> > > >>>>> IOPS runtime
>> > > >>>>>>>>>>> change?
>> > > >>>>>>>>>>>>>>>>>> KVM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > limitation for
>> > > >>> a
>> > > >>>>> running
>> > > >>>>>> virtual
>> > > >>>>>>>>>>>>>>>>>> machine
>> > > >>>>>>>>>>>>>>>>>>>>>> through
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > CloudStack
>> > > >>> does
>> > > >>>>> not support
>> > > >>>>>>>>>>> changing
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > (computer
>> > > >>>>> offering or disk
>> > > >>>>>>>>>>>>> offering).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
>> > the
>> > > >>>>> Java interface
>> > > >>>>>>>>> level.
>> > > >>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>> apologize
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
>> > > >>>>> generalize allocation
>> > > >>>>>>>>>>>>>>>>>> algorithms
>> > > >>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
>> > the
>> > > >>>>> service guarantees
>> > > >>>>>>>>>>>>>>>>>> provided by a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
>> > > >>> good
>> > > >>>>> question.
>> > > >>>>>>>>> Maybe it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>> better
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> disk_io_throttling as Mike
>> > > >>>>>>>>> needs to
>> > > >>>>>>>>>>>>>>>>>> consider
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
>> > think.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
>> > expand
>> > > >>> on
>> > > >>>>> my thoughts
>> > > >>>>>> in a
>> > > >>>>>>>>>>>>> later
>> > > >>>>>>>>>>>>>>>>>>>>>> response
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
>> > points
>> > > >>>>> between these two
>> > > >>>>>>>>>>>>>>>>>> features. I
>> > > >>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
>> > to be
>> > > >>>>> merged before
>> > > >>>>>>>>>>>>>>>>>> SolidFire, but
>> > > >>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > coordination
>> > > >>>>> between the
>> > > >>>>>> branches
>> > > >>>>>>>>>>>>>>>>>> (possibly
>> > > >>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> disk_io_throttling)
>> > > >>>>> to
>> > > >>>>>>>>> coordinate on
>> > > >>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>> issue.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>> > > >>> John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > things I
>> > > >>>>> want to
>> > > >>>>>> understand
>> > > >>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
>> > there
>> > > >>>>> value in
>> > > >>>>>> capturing
>> > > >>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>> policies
>> > > >>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > model
>> > > >>> (e.g.
>> > > >>>>> for
>> > > >>>>>>>>> billing/usage
>> > > >>>>>>>>>>>>>>>>>>>> purposes,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
>> > Should
>> > > >>> there
>> > > >>>>> be a common
>> > > >>>>>>>>>>>>>>>>>> interface model
>> > > >>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > provisioning
>> > > >>> at
>> > > >>>>> runtime?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
>> > are
>> > > >>>>> conflicting
>> > > >>>>>>>>> provisioned
>> > > >>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > hypervisor
>> > > >>> and
>> > > >>>>> storage device
>> > > >>>>>>>>>>>>>>>>>> reconciled?
>> > > >>>>>>>>>>>>>>>>>>>> In
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
>> > to
>> > > >>>>> believe (and
>> > > >>>>>> billed)
>> > > >>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>> device
>> > > >>>>> has been
>> > > >>>>>>>>> configured
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > consistent
>> > > >>>>> configuration
>> > > >>>>>>>>> between a
>> > > >>>>>>>>>>>>>>>>>> VM and a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
>> > but a
>> > > >>>>> later
>> > > >>>>>> modification
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> inconsistency.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
>> > 2,
>> > > >>> 2013,
>> > > >>>>> at 8:38 PM,
>> > > >>>>>>>>> Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > believe
>> > > >>> Wei's
>> > > >>>>> feature deals
>> > > >>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>> controlling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>> hypervisor
>> > > >>>>> side.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> > feature
>> > > >>> is
>> > > >>>>> focused on
>> > > >>>>>>>>>>>>> controlling
>> > > >>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
>> > that
>> > > >>>>> helps. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
>> > Jun
>> > > >>> 2,
>> > > >>>>> 2013 at 6:35
>> > > >>>>>> PM,
>> > > >>>>>>>>>>>>> John
>> > > >>>>>>>>>>>>>>>>>>>> Burwell
>> > > >>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > jburwell@basho.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> > > >>> opinion is
>> > > >>>>> that no
>> > > >>>>>> features
>> > > >>>>>>>>>>>>>>>>>> should be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
>> > > >>> have
>> > > >>>>> been resolved
>> > > >>>>>> and
>> > > >>>>>>>>> it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>> ready
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > Ops
>> > > >>> vs
>> > > >>>>> discrete
>> > > >>>>>>>>> read/write
>> > > >>>>>>>>>>>>>>>>>> ops issue
>> > > >>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
>> > reviewed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > I
>> > > >>> don't
>> > > >>>>> think this
>> > > >>>>>>>>> criteria
>> > > >>>>>>>>>>>>>>>>>> has been
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
>> > how
>> > > >>>>> does this work
>> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>> > > >>>>>>>>>>>>>>>>>> )?
>> > > >>>>>>>>>>>>>>>>>>>> As I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>> > > >>> involves
>> > > >>>>> provisioned
>> > > >>>>>>>>> IOPS. I
>> > > >>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>> like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > scenario
>> > > >>>>> where provisioned
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> KVM and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > unnecessarily
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> incompatible.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > Jun 1,
>> > > >>> 2013,
>> > > >>>>> at 6:47 AM,
>> > > >>>>>>>>> Wei
>> > > >>>>>>>>>>>>>>>>>> ZHOU <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
>> > will
>> > > >>>>> change it next
>> > > >>>>>>>>> week.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/6/1
>> > > >>>>> Wido den Hollander
>> > > >>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 06/01/2013 08:24 AM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Exactly. I
>> > > >>> have
>> > > >>>>> pushed the
>> > > >>>>>>>>>>>>> features
>> > > >>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > anyone
>> > > >>>>> object thems for
>> > > >>>>>>>>>>>>> technical
>> > > >>>>>>>>>>>>>>>>>>>> reason
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>> > the
>> > > >>> sake
>> > > >>>>> of clarity I
>> > > >>>>>> just
>> > > >>>>>>>>>>>>> want
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> mention
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > total
>> > > >>> IOps
>> > > >>>>> to R/W IOps
>> > > >>>>>>>>> asap
>> > > >>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>> that we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>> > total
>> > > >>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
>> > laid
>> > > >>> the
>> > > >>>>> groundwork for
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>> throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > however
>> > > >>>>> prevent that we
>> > > >>>>>> create
>> > > >>>>>>>>>>>>>>>>>> legacy from
>> > > >>>>>>>>>>>>>>>>>>>>>> day
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/5/31
>> > > >>>>> Wido den
>> > > >>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/31/2013 03:59 PM, John
>> > > >>>>>>>>>>>>> Burwell
>> > > >>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
>> > this
>> > > >>>>> enhancement must
>> > > >>>>>> to
>> > > >>>>>>>>>>>>>>>>>> discretely
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > see
>> > > >>> how
>> > > >>>>> it could be
>> > > >>>>>>>>> fixed
>> > > >>>>>>>>>>>>>>>>>> later
>> > > >>>>>>>>>>>>>>>>>>>>>> because
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > correctly
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
>> > total
>> > > >>> IOPS
>> > > >>>>> into read
>> > > >>>>>> and
>> > > >>>>>>>>>>>>>>>>>> write.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>> unless/until we
>> > > >>>>>> decided
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> break
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
>> > Wei
>> > > >>>>> meant was merging
>> > > >>>>>> it
>> > > >>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>> master
>> > > >>>>>>>>>>>>>>>>>>>>>> now
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > branch
>> > > >>>>> and add Read /
>> > > >>>>>>>>> Write
>> > > >>>>>>>>>>>>> IOps
>> > > >>>>>>>>>>>>>>>>>>>> before
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
>> > > >>>>> released with Read
>> > > >>>>>> and
>> > > >>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>> instead
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
>> > to
>> > > >>>>> make the May 31st
>> > > >>>>>>>>>>>>> feature
>> > > >>>>>>>>>>>>>>>>>>>> freeze
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
>> > other
>> > > >>>>> threads) then it
>> > > >>>>>>>>> won't
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>> necessary
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
>> > > >>>>> completely agree that
>> > > >>>>>>>>>>> there
>> > > >>>>>>>>>>>>>>>>>> is no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > I/O.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > May 31,
>> > > >>>>> 2013, at 9:51 AM,
>> > > >>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>> den
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/31/2013 03:13 PM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>> > Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks.
>> > > >>> Good
>> > > >>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > thought
>> > > >>>>> about at the
>> > > >>>>>>>>>>> beginning.
>> > > >>>>>>>>>>>>>>>>>>>> Finally I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > difference
>> > > >>> of
>> > > >>>>> read and write
>> > > >>>>>>>>>>>>> mainly
>> > > >>>>>>>>>>>>>>>>>>>> because
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
>> > the
>> > > >>>>> difference of sent
>> > > >>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> received
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>> > > >>>>> reasoning seems odd.
>> > > >>>>>>>>>>>>>>>>>> Networking and
>> > > >>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>> > I/O is
>> > > >>>>> much more
>> > > >>>>>>>>> expensive
>> > > >>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> most
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > bandwith.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> Implementing
>> > > >>>>> it will be some
>> > > >>>>>>>>>>>>>>>>>> copy-paste
>> > > >>>>>>>>>>>>>>>>>>>>>> work.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> implemented
>> > > >>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
>> > days.
>> > > >>> For
>> > > >>>>> the deadline
>> > > >>>>>> of
>> > > >>>>>>>>>>>>>>>>>> feature
>> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
>> > if
>> > > >>>>> needed.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>> > think it's
>> > > >>> a
>> > > >>>>> feature we
>> > > >>>>>>>>> can't
>> > > >>>>>>>>>>>>>>>>>> miss. But
>> > > >>>>>>>>>>>>>>>>>>>>>> if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > window
>> > > >>> we
>> > > >>>>> have to make sure
>> > > >>>>>> we
>> > > >>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
>> > 4.3,
>> > > >>> that
>> > > >>>>> will confuse
>> > > >>>>>>>>>>>>> users.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/5/31
>> > > >>>>> Wido den
>> > > >>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/30/2013 06:03 PM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > would
>> > > >>> like to
>> > > >>>>> merge
>> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > nobody
>> > > >>>>> object, I will
>> > > >>>>>> merge
>> > > >>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>> master
>> > > >>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>> purpose
>> > > >>>>> is :
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
>> > > >>>>> machines are running
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> same
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > storage or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
>> > > >>> strage).
>> > > >>>>> Because of
>> > > >>>>>> the
>> > > >>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>>>> limitation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
>> > if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
>> > VM
>> > > >>> has
>> > > >>>>> large disk
>> > > >>>>>>>>> operation,
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>> affect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>> > VMs
>> > > >>>>> running on the
>> > > >>>>>> same
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>> device.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> > > >>> neccesary
>> > > >>>>> to set the
>> > > >>>>>>>>>>> maximum
>> > > >>>>>>>>>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Looking at
>> > > >>> the
>> > > >>>>> code I see
>> > > >>>>>> you
>> > > >>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>> no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
>> > and
>> > > >>>>> libvirt support
>> > > >>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>> both a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > > >>> which
>> > > >>>>> could benefit a
>> > > >>>>>>>>> lot of
>> > > >>>>>>>>>>>>>>>>>> users.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
>> > also
>> > > >>>>> strange, in the
>> > > >>>>>>>>> polling
>> > > >>>>>>>>>>>>>>>>>> side you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>> > but
>> > > >>> on
>> > > >>>>> the throttling
>> > > >>>>>>>>> side
>> > > >>>>>>>>>>>>>>>>>> you only
>> > > >>>>>>>>>>>>>>>>>>>> go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > IOps
>> > > >>> are
>> > > >>>>> usually much
>> > > >>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>> expensive
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
>> > valid
>> > > >>>>> use-case where
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > vs
>> > > >>> Read
>> > > >>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
>> > this
>> > > >>> only
>> > > >>>>> supports KVM
>> > > >>>>>>>>> at
>> > > >>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>> point I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>> > to at
>> > > >>>>> least have the
>> > > >>>>>>>>>>>>>>>>>> mechanism in
>> > > >>>>>>>>>>>>>>>>>>>>>> place
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> implementing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > later
>> > > >>>>> would be a lot of
>> > > >>>>>>>>>>> work.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
>> > > >>> hypervisor
>> > > >>>>> doesn't
>> > > >>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > you
>> > > >>> can
>> > > >>>>> always sum
>> > > >>>>>> both
>> > > >>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>> and set
>> > > >>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> > you
>> > > >>>>> explain why you
>> > > >>>>>>>>>>>>> implemented
>> > > >>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>> feature
>> > > >>>>> includes:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> > the
>> > > >>>>> maximum rate of
>> > > >>>>>>>>> VMs
>> > > >>>>>>>>>>>>> (in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> configuration)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> > change
>> > > >>> the
>> > > >>>>> maximum rate
>> > > >>>>>> of
>> > > >>>>>>>>>>> VMs
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> > limit
>> > > >>> the
>> > > >>>>> disk rate
>> > > >>>>>> (total
>> > > >>>>>>>>>>> bps
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> iops)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
>> > ticket:
>> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> issues.apache.org/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> jira/browse/CLOUDSTACK-1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1192>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>> 1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 1192
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 1192>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
>> > will
>> > > >>>>> update later) :
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>> VM+Disk+IO+Throttling<https://
>> <https://%0b/>> > >>>>>>>>>>>>> ****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cwiki.apache.org/confluence/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>
>> > > >>>
>> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>> > > >>>>> rottling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Merge
>> > > >>> check
>> > > >>>>> list :-
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
>> > you
>> > > >>>>> check the branch's
>> > > >>>>>>>>> RAT
>> > > >>>>>>>>>>>>>>>>>> execution
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
>> > > >>> there
>> > > >>>>> new dependencies
>> > > >>>>>>>>>>>>>>>>>> introduced?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> > What
>> > > >>>>> automated testing
>> > > >>>>>> (unit
>> > > >>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > feature?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
>> > tests
>> > > >>> are
>> > > >>>>> added.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> > What
>> > > >>>>> testing has been done
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> check for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> > the
>> > > >>>>> bytes rate and
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>> rate on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> > VM
>> > > >>>>> operations, including
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > deploy,
>> > > >>> stop,
>> > > >>>>> start, reboot,
>> > > >>>>>>>>>>>>>>>>>> destroy,
>> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> > > >>> Volume
>> > > >>>>> operations,
>> > > >>>>>>>>> including
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
>> > > >>>>> Detach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
>> > review
>> > > >>> the
>> > > >>>>> code, you can
>> > > >>>>>>>>> try
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
>> > > >>> regards,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>> VM+Disk+IO+Throttling<https://
>> <https://%0b/>> > >>>>>>>>>>>>> ****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cwiki.apache.org/confluence/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>
>> > > >>>
>> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>> > > >>>>> rottling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>> > > >>>>>>>>> refs/heads/disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>>>> 1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1301>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 1301
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 1301>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > issues.apache.org/****jira/**browse/CLOUDSTACK-
>> > > >>> 2071
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 2071
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 2071>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <h**ttps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 2071>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> (**CLOUDSTACK-1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> > VM
>> > > >>> Disk
>> > > >>>>> I/O
>> > > >>>>>> Throttling)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > > >>>>> Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>>>> CloudStack Developer,
>> > > >>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > > >>> 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Advancing
>> > > >>> the
>> > > >>>>> way the world
>> > > >>>>>>>>> uses
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > > >>> Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>> CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> > the
>> > > >>> way
>> > > >>>>> the world uses
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>> CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > > >>> way
>> > > >>>>> the world uses
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > way
>> > > >>> the
>> > > >>>>> world uses the
>> > > >>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > CloudStack
>> > > >>>>> Developer,
>> > > >>>>>> SolidFire
>> > > >>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > way
>> > > >>> the
>> > > >>>>> world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>>>> Developer, SolidFire
>> > > >>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> > the
>> > > >>>>> world uses the
>> > > >>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>> Developer,
>> > > >>>>> SolidFire
>> > > >>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> > the
>> > > >>> world
>> > > >>>>> uses the
>> > > >>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>> Developer,
>> > > >>>>> SolidFire
>> > > >>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > > >>> world
>> > > >>>>> uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > Developer,
>> > > >>>>> SolidFire
>> > > >>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > world
>> > > >>> uses
>> > > >>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > Developer,
>> > > >>>>> SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > world
>> > > >>> uses
>> > > >>>>> the cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>>>> SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> > uses
>> > > >>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> > uses
>> > > >>> the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > SolidFire
>> > > >>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > > >>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> --
>> > > >>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>> *(tm)*
>> > > >>>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> --
>> > > >>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>> o: 303.746.7302
>> > > >>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>> *(tm)*
>> > > >>>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>> --
>> > > >>>>>>> *Mike Tutkowski*
>> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>> o: 303.746.7302
>> > > >>>>>>> Advancing the way the world uses the
>> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>> *(tm)*
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> --
>> > > >>>>> *Mike Tutkowski*
>> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>> o: 303.746.7302
>> > > >>>>> Advancing the way the world uses the
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>> *(tm)*
>> > > >>>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> --
>> > > >>> *Mike Tutkowski*
>> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>> e: mike.tutkowski@solidfire.com
>> > > >>> o: 303.746.7302
>> > > >>> Advancing the way the world uses the
>> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>> *(tm)*
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Mike Tutkowski*
>> > > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > > e: mike.tutkowski@solidfire.com
>> > > > o: 303.746.7302
>> > > > Advancing the way the world uses the
>> > > > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > > *(tm)*
>> > >
>> > >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *(tm)*****
>>
>>
>>
>> ****
>>
>> ** **
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*****
>>
>>
>>
>> ****
>>
>> ** **
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*****
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
I want to make sure we are on the same page regarding isManaged. I see the addition of the following operations:
* DataStoreDriver#supportsManagement() : boolean -- indicates whether or not the driver support/implements the management functions
* DataStore#isManaged() : boolean -- indicates whether or not a device is managed by CloudStack
Only DataStores associated with DataStoreDrivers that support management would be able to have the management flag enabled. This behavior allows operators to declare their intention to allow CloudStack to manage a device. This behavior would need to be exposed as flag on the HTTP API. Additionally, this flag would only be settable when a device is defined/created in the system (i.e. once a device is managed by CloudStack, you can't unmanaged it without dissociating it from CloudStack). In the future, we may consider relaxing that rule, but, in the short term, it greatly decreases the alternative flows.
In terms of removing the dependencies from Storage to Hypervisor, I would really like to see this accomplished in 4.2. It will be much more difficult to address this issue post 4.2 because we will have more plugins to address. I also see this issue as a critical architectural issues that for the management server. I recommend we identify the dependencies that need to be inverted, and see if we can divide the work across a few more resources.
Thanks,
-John
On Jun 10, 2013, at 1:36 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Sounds good, Edison :)
>
> I can implement John's isManaged() logic and resubmit my code for review.
>
> Perhaps in 4.3 I could address the direct-attach scenario. If it requires some refactoring of the storage framework, then I could do that.
>
> Also, the default storage plug-in does have hypervisor logic in it in 4.2. I recommend we leave this as is for 4.2 and it can be refactored in 4.3 with the above work I mentioned.
>
> Are you OK with that, guys?
>
>
> On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com> wrote:
> Ok, I am Ok with the way you are talking about in 4.2.
>
>
>
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Sunday, June 09, 2013 8:59 PM
> To: dev@cloudstack.apache.org
> Cc: Edison Su; John Burwell
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
>
>
> Another thing we might want to consider for 4.2 is that the way the code is currently implemented in master, I don't think it's even possible to write a plug-in that connects storage directly to a VM because the CloudStack storage framework will call the "attach" method of the hypervisor in use and this method will fail because it assumes there is (talking Xen here) an SR existent and in this situation there isn't one.
>
>
>
> My thinking is that we should ignore the direct-attach-of-volume-to-VM use case for 4.2. I think it will require too much of a rewrite at this point in development (fewer than three weeks left until code freeze).
>
>
>
> If the plug-in implements an isManaged() method (as John suggested), the storage framework can invoke the necessary commands on the hypervisor in use.
>
>
>
> For example, the createAsync method is called. The storage plug-in creates a SAN volume and updates the CS DB. The storage framework then asks the driver if it's managed. If it is, the storage framework executes a command against the hypervisor in use to have it create, say, an SR (with a single VDI that takes up the entire SR). Now, when the storage framework executes the "attach" command against the hypervisor, this command will work, as well (because that logic assumes the prior existence of an SR and it will, in fact, be present).
>
>
>
> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
>
> Hi Edison and John,
>
>
>
> I wanted to point out something I thought was relevant to this conversation.
>
>
>
> My plug-in supports both XenServer and ESX(i).
>
>
>
> These two hypervisors handle their respective hypervisor data structures (XenServer SR or ESX(i) datastore) quite differently.
>
>
>
> In XenServer, the SR that resides on the SAN LUN is simply "forgotten" when you issue a remove-SR command to XenServer.
>
>
>
> When you later want to access the contents of that data, it is still safely stored on the LUN and can be accessed from XenServer (from the same or different cluster (what XenServer calls a resource pool)) by creating a new SR that is based on the same IQN and LUN.
>
>
>
> XenServer SRs exist at the cluster level.
>
>
>
> In ESX(i), the datastore that resides on the SAN LUN is actually destroyed when you issue a remove-datastore command to ESX(i). In other words, the contents of the SAN LUN are destroyed.
>
>
>
> ESX(i) datastores exist at the datacenter level.
>
>
>
> The XenServer behavior (not destroying the contents of the LUN when the SR is removed) is desirable; the ESX(i) behavior (destroying the contents of the LUN when the datastore is removed) is not desirable.
>
>
>
> I can get around the ESX(i) behavior by removing the referenced IQN from each host in the cluster when the CS storage framework notifies my plug-in of a detach-volume event. When the CS volume is later attached again (either to the same or a different ESX(i) cluster), I can have each host in the relevant ESX(i) cluster reference the IQN and the datastore will be back again.
>
>
>
> This sequence of events is critical for my plug-in to support both environments:
>
> XenServer flow (events for Driver class):
>
> createAsync (called right before first attach) = creates SAN volume
>
> preAttachVolume = creates SR and creates VDI or just introduces SR
>
> (CS volume gets attached and later detached)
>
> postDetachVolume = removes SR (this does not delete data on SAN volume)
>
> deleteAsync (can be called when volume is not attached) = deletes the SAN volume
>
> VMware flow (events for Driver class):
>
> createAsync (called right before first attach) = creates SAN volume, creates datastore, and creates VMDK file
>
> preAttachVolume = adds iSCSI target to each host in cluster
>
> (CS volume gets attached and later detached)
>
> postDetachVolume = removes iSCSI target from each host in cluster
>
> deleteAsync (can be called when volume is not attached) = deletes datastore and deletes the SAN volume
>
> ********************
>
> So, the way I have it written up here, the driver is talking to the hypervisor. As John has pointed out, this may not be ideal.
>
>
>
> Whatever we decide on, it needs to be able to support this kind of flow. :)
>
>
>
> We could go John's route, where the driver says if it's managed or not.
>
>
>
> If it is, then the storage framework could send the necessary messages to the hypervisor in question at the right times (just migrate the hypervisor logic above from the driver to the right parts of the storage framework).
>
>
>
> This is fine, except that it does not resolve Edison's use case of desiring to have the storage LUN potentially connected directly to a VM.
>
>
>
> I'm not sure we should design around that use case, though. Is it common enough? Has anyone asked for it? Is it even desirable? How are usage statistics gathered for such volumes? It's nice to offer the flexibility, but is it a good idea in this case?
>
>
>
> Thanks!
>
>
>
>
>
> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > Sent: Friday, June 07, 2013 4:08 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > Hi John,
> >
> > Yes, you are correct that Xen must layer an SR on top of the iSCSI volume
> > to use it. Same story for VMware: ESX(i) must layer a datastore on top of
> > the iSCSI volume to use it. I look at it like they are layering a clustered
> > file system on the SAN volume so the hypervisors can share access to the
> > contents of the volume.
> >
> > In 4.1, by the time the hypervisor-attach-volume logic is called, the SR or
> > datastore has already been created (usually manually by an admin).
> >
> > This pre-setup of the, say, SR is, of course, not acceptable in an
> > environment where each CloudStack volume that a user creates is mapped
> > to a
> > single SAN volume (via the SR).
> >
> > The question comes down to who should allocate the SR dynamically.
> >
> > We could have the storage framework ask the storage plug-in if it is
> > managed. If it is, then the storage framework could send a message to the
> > hypervisor in question to create (let's talk Xen here) the SR ahead of
> > time. Then, when the storage framework next sends the attach-volume
> > command
> > to the hypervisor, it should work without changes to that attach-volume
> > logic (because - from the point of view of the attach logic - the SR is
> > already existent, as expected).
> >
> > Now, as Edison has pointed out, this limits the power of the storage
> > plug-in. A storage plug-in in this model cannot directly attach storage to
> > a VM (it must go through the hypervisor). Perhaps that is OK. We need to
> > make a call on that.
>
>
> Yes, I think we need to address this usage case, and won't limit the power of the storage plug-in.
> John, what's your idea to fit this usage case into your abstraction?
>
>
> >
> > If we want to give more power to the storage vendor, we would have to
> > have
> > the storage framework call into the storage plug-in to send the appropriate
> > attach commands to the hypervisor.
> >
> > Let's discuss and come to a consensus (or at least agree on a path) as soon
> > as we can. :)
> >
> > Thanks!
> >
> >
> >
> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com> wrote:
> >
> > > Mike,
> > >
> > > My understanding is the Xen expects an SR structure on all ISCSI devices
> > > -- at least that is how I read the code in your patch. Is my understanding
> > > correct? If so, the Xen plugin should be able to query the storage device
> > > to determine the presence of the SR structure and create it if it does not
> > > exist. Am I missing something in the implementation that makes that type
> > > of implementation impossible?
> > >
> > > Thanks,
> > > -John
> > >
> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <mi...@solidfire.com>
> > > wrote:
> > >
> > > > Yeah, if a storage vendor wanted to attach a volume directly to a VM in
> > > 4.2
> > > > today, it would probably fail because the attach-volume logic assumes
> > the
> > > > existence of the necessary hypervisor data structure (ex. SR on Xen).
> > > >
> > > > If we wanted to enable such an attach, we could do it the way Edison
> > > > suggests.
> > > >
> > > >
> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com> wrote:
> > > >
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > >>> Sent: Friday, June 07, 2013 2:37 PM
> > > >>> To: dev@cloudstack.apache.org
> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > > >>>
> > > >>> As we only have three weeks until feature freeze, we should come to
> > a
> > > >>> consensus on this design point as soon as possible.
> > > >>>
> > > >>> Right now, if the storage framework asks my driver it is is managed, it
> > > >>> will say 'yes.' This means the framework will tell the driver to
> > > perform
> > > >>> its management activities. This then means the driver will call into
> > > the
> > > >>> host (it doesn't know which hypervisor, by the way) to perform the
> > > >> activity
> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> > > >>>
> > > >>> The driver doesn't know which hypervisor it's talking to, it just
> > > sends a
> > > >>> message to the host to perform the necessary pre-attach work.
> > > >>
> > > >> Could we just expose a method like "attachVolume/dettachVolume" on
> > the
> > > >> PrimaryDataStoreDriver driver?
> > > >> For most of cases, the implementation of each driver would just send
> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
> > the
> > > >> implementation in a base class, so that can be shared by all of this
> > > >> drivers), and in each hypervisor resource code, which may just call
> > > >> hypervisor's api to attach the volume to VM, while for certain storage,
> > > >> like SolidFire, may need to create a SR first, and create volume on it,
> > > >> then call hypervisor's API to attach volume to VM.
> > > >> While for some other storage vendor, may want to bypass hypervisor
> > > during
> > > >> attaching volume, so inside driver's attachvolume implementation, the
> > > >> driver can do some magic, such as, directly talk to an agent inside VM
> > > >> instance, then create a disk inside VM.
> > > >>
> > > >> How do you guys think?
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> > > wrote:
> > > >>>
> > > >>>>
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> > > >>>>> To: dev@cloudstack.apache.org
> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > > >>>>>
> > > >>>>> Hi John,
> > > >>>>>
> > > >>>>> How's about this:
> > > >>>>>
> > > >>>>> The driver can implement an isManaged() method. The
> > > >>> VolumeManagerImpl
> > > >>>>> can
> > > >>>>> call into the driver to see if its managed. If it is, the
> > > >>>> VolumeManagerImpl
> > > >>>>> (which is responsible for calling into the hypervisor to attach the
> > > >> disk)
> > > >>>>> can call into the hypervisor to create the necessary hypervisor data
> > > >>>>> structure (ex. for XenServer, a storage repository).
> > > >>>>
> > > >>>> The problem here is that storage vendor may work differently with
> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> > > >> other
> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> > directly
> > > >> to
> > > >>>> VM instance, see the discuss(
> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> > > >>>
> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> > > >>> %3E
> > > >>>> ).
> > > >>>> So I would let storage provider to implement attach disk to VM,
> > > >> instead of
> > > >>>> implemented by cloudstack itself.
> > > >>>>
> > > >>>>
> > > >>>>>
> > > >>>>> If that's what you're going for, that works for me. By the way,
> > > >> Edison's
> > > >>>>> default storage plug-in (which handles the default storage behavior
> > > >> in
> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
> > > >>>>> hypervisors. You might want to contact him and inform him of your
> > > >>>> concerns
> > > >>>>> or that logic (as is) will make it to production.
> > > >>>>>
> > > >>>>> Please let me know if what I wrote in above (for my solution) is OK
> > > >> with
> > > >>>>> you. :)
> > > >>>>>
> > > >>>>> Thanks!
> > > >>>>>
> > > >>>>>
> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
> > > >>> wrote:
> > > >>>>>
> > > >>>>>> Mike,
> > > >>>>>>
> > > >>>>>> Please see my responses in-line below.
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> -John
> > > >>>>>>
> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > > >>>> mike.tutkowski@solidfire.com>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hey John,
> > > >>>>>>>
> > > >>>>>>> I still have a bit more testing I'd like to do before I build up
> > > >> a
> > > >>>> patch
> > > >>>>>>> file, but this is the gist of what I've done:
> > > >>>>>>>
> > > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
> > tells
> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> > > >>>>> managed.
> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> > > >> whatever
> > > >>>>>>> activity is required. In my case, this includes sending a
> > > >> message to
> > > >>>> the
> > > >>>>>>> host where the VM is running to have, say XenServer, add a
> > > >> storage
> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
> > > >> SAN
> > > >>>>>> volume,
> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can
> > > >> on
> > > >>>> the
> > > >>>>>>> storage repository). After this, the normal attach-volume
> > > >> message is
> > > >>>> sent
> > > >>>>>>> to the host by VolumeManagerImpl.
> > > >>>>>>
> > > >>>>>> There should be **no** code from a storage driver to a hypervisor.
> > > >> I
> > > >>>>>> apologize for the repetition, but we simply can not have hypervisor
> > > >>>>>> specific code in the storage layer. The circular dependencies
> > > >> between
> > > >>>> the
> > > >>>>>> two layers are not sustainable in the long term. Either the
> > > >>>> VirtualManager
> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> > > >>> coordinate
> > > >>>>>> volume creation and then populating the SR. Ideally, we can
> > > >>>> generalize the
> > > >>>>>> process flow for attaching volumes such that the Xen hypervisor
> > > >> plugin
> > > >>>>>> would only implement callbacks to perform the attach action and
> > > >> create
> > > >>>>> the
> > > >>>>>> structure and SR. To my mind, the SolidFire driver should only be
> > > >>>>>> allocating space and providing information about contents (e.g.
> > > >> space
> > > >>>>>> available, space consumed, streams to a URI, file handle for a URI,
> > > >>>> etc)
> > > >>>>>> and capabilities.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> * The reverse is performed for a detach-volume command.
> > > >>>>>>>
> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
> > > >>>>>> Edison's
> > > >>>>>>> default driver simply does a "return false;". We could add a new
> > > >>>>>> parameter
> > > >>>>>>> to the createStoragePool API command, if we want, to remove
> > the
> > > >>>>>> hard-coded
> > > >>>>>>> return values in the drivers (although my driver will probably
> > > >> just
> > > >>>>>> ignore
> > > >>>>>>> this parameter and always return true since it wouldn't make
> > > >> sense
> > > >>>> for it
> > > >>>>>>> to ever return false). We'd need another column in the
> > > >> storage_pool
> > > >>>>> table
> > > >>>>>>> to store this value.
> > > >>>>>>
> > > >>>>>> Yes, I think we should have a parameter added to the
> > > >>> createStoragePool
> > > >>>>>> surfaced to the HTTP API that allows DataStores to be configured
> > > >> for
> > > >>>>>> management when their underlying drivers support it. To simplify
> > > >>>> things,
> > > >>>>>> this flag should only be mutable when the DataStore is created. It
> > > >>>> would be
> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
> > > >>>> creation.
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> Sound like I'm in sync with what you were thinking?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > > >>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> I agree, John. Just wanted to point out that I have a working
> > > >> GUI
> > > >>>> for
> > > >>>>>> you
> > > >>>>>>>> to review (in that document), if you'd like to check it out.
> > > >>>>>>>>
> > > >>>>>>>> Thanks!
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> > > >> jburwell@basho.com>
> > > >>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Mike,
> > > >>>>>>>>>
> > > >>>>>>>>> I would like the UIs of two features reviewed together to
> > > >> ensure
> > > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
> > > >> and
> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
> > > >>>> confusion,
> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
> > > >> help
> > > >>>>>>>>> minimize any potential confusion.
> > > >>>>>>>>>
> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work itself
> > > >> if
> > > >>>> it
> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> > > >>>> throttled
> > > >>>>>>>>> IOPs or storage provisioned IOPs.
> > > >>>>>>>>>
> > > >>>>>>>>> Thanks,
> > > >>>>>>>>> -John
> > > >>>>>>>>>
> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > > >>>>>>>>> <mi...@solidfire.com> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hi John,
> > > >>>>>>>>>>
> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
> > to
> > > >>> you
> > > >>>>> the
> > > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
> > > >> IOPS.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> > > >> and we
> > > >>>>>>>>> decided
> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> > Offering
> > > >>>> dialog.
> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> > should
> > > >>> be
> > > >>>>> able
> > > >>>>>> to
> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only use
> > > >> Max
> > > >>>>>> IOPS).
> > > >>>>>>>>>> These fields are optional and can be ignored for storage that
> > > >>>> does not
> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> > > >>>> admin can
> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
> > > >> IOPS.
> > > >>>>>>>>>>
> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
> > > >> mine,
> > > >>>> as
> > > >>>>>> is
> > > >>>>>>>>>> decided by the admin).
> > > >>>>>>>>>>
> > > >>>>>>>>>> I'm not sure what we can do about these two features
> > > >> expressing
> > > >>>> the
> > > >>>>>>>>> speed
> > > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
> > > >>>> QoS in
> > > >>>>>>>>> any
> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
> > > >>>> IOPS).
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thanks
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > > >>> <jb...@basho.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Wei,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> > > >>> Storage/Hypervisor
> > > >>>>>> layer
> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
> > > >>>> hypervisor
> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> > > >> following
> > > >>>>>> issues
> > > >>>>>>>>>>> outstanding:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > > >>> read/write
> > > >>>>>>>>> values
> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
> > > >> IOPS.
> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> > > >>> storage
> > > >>>>>> volumes
> > > >>>>>>>>>>> with provisioned IOPS?
> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> > > >>>> provisioned
> > > >>>>>> IOPS
> > > >>>>>>>>>>> that will permit providers to discriminate these guaranteed
> > > >>>>>> operations
> > > >>>>>>>>> in
> > > >>>>>>>>>>> the event they want to bill for it differently?
> > > >>>>>>>>>>> 4. What is the user experience for throttled and provisioned
> > > >>> IOPS
> > > >>>>>> that
> > > >>>>>>>>>>> minimizes confusion of these concepts?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> > > >>>> throttled
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
> > > >>>> items 1
> > > >>>>>>>>> and 2.
> > > >>>>>>>>>>> Since the two facilities would not be permitted to operate
> > > >>>> together,
> > > >>>>>>>>> they
> > > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
> > > >> can be
> > > >>>>>>>>> resolved
> > > >>>>>>>>>>> with an additional flag or two on the usage records. As for
> > > >>>> Item 4,
> > > >>>>>> I
> > > >>>>>>>>> am
> > > >>>>>>>>>>> not familiar with how these two enhancements are
> > depicted in
> > > >>> the
> > > >>>>> user
> > > >>>>>>>>>>> interface. I think we need to review the UI enhancements
> > for
> > > >>>> both
> > > >>>>>>>>>>> enhancements and ensure they are consistent.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Do these solutions make sense?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks,
> > > >>>>>>>>>>> -John
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> > <us...@gmail.com>
> > > >>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> John and Mike,
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> > 2780/2729,
> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > > >>> CLOUDSTACK-
> > > >>>>> 2875 ) this
> > > >>>>>>>>> week.
> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
> > > >> ask
> > > >>>> for
> > > >>>>>>>>> second
> > > >>>>>>>>>>>> merge request after testing.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> I believe I understand where you're going with this, John,
> > > >> and
> > > >>>>> have
> > > >>>>>>>>> been
> > > >>>>>>>>>>>>> re-working this section of code today.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks for the comments,
> > > >>>>>>>>>>>>> Mike
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > > >>>>> <jb...@basho.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> See my responses in-line below.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks for the response.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> > whether or
> > > >>>>> not a
> > > >>>>>>>>> Volume
> > > >>>>>>>>>>> is
> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> > > >>> actual
> > > >>>>>>>>> isManaged
> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> > > >> resources
> > > >>>> for
> > > >>>>>> the
> > > >>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>> if it is managed."
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> > method
> > > >>>>> added to
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> > > >> then the
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
> > > >>> method
> > > >>>>> (which
> > > >>>>>>>>>>> would
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
> > and
> > > >>> this
> > > >>>>>> would
> > > >>>>>>>>>>> call
> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
> > on
> > > >>>>>>>>> XenServer,
> > > >>>>>>>>>>> an
> > > >>>>>>>>>>>>> SR?
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> > > >>>>> DataStoreDriver.
> > > >>>>>>>>> Since
> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
> > I
> > > >>>> would
> > > >>>>>> add
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> > straw
> > > >>> man
> > > >>>>> term
> > > >>>>>>>>> --
> > > >>>>>>>>>>>>>> something other than volume) methods to
> > > >>>>>>>>>>> allocate/create/deallocate/delete
> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> > > >>> property
> > > >>>>> of
> > > >>>>>>>>>>>>> DataStore
> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> > > >>> DataStoreDriver
> > > >>>>> can be
> > > >>>>>>>>>>>>> managed.
> > > >>>>>>>>>>>>>> This approach allows operators to override
> > manageability
> > > >> of
> > > >>>>>> devices.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> > > >> plugin
> > > >>>>>> would
> > > >>>>>>>>> be
> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
> > to
> > > >>>>> create any
> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There should
> > > >> be no
> > > >>>>>>>>>>>>> dependencies
> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> > > >> earlier,
> > > >>>> such
> > > >>>>>>>>>>>>> circular
> > > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
> > mess.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks again, John
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > > >>>>> <jb...@basho.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
> > that
> > > >>>>>> supports n
> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> > > >> hypervisors.
> > > >>>>>> Such
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> > > >>> beast.
> > > >>>>>>>>>>>>> Therefore, my
> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
> > the
> > > >>>>> Storage
> > > >>>>>> layer
> > > >>>>>>>>>>>>> towards
> > > >>>>>>>>>>>>>>> this separation.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> > whether
> > > >>> or
> > > >>>>> not a
> > > >>>>>>>>> Volume
> > > >>>>>>>>>>> is
> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
> > an
> > > >>> actual
> > > >>>>>>>>>>> isManaged
> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> > > >> resources
> > > >>>> for
> > > >>>>>> the
> > > >>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
> > > >> to
> > > >>>>>> indicate
> > > >>>>>>>>>>>>> whether
> > > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
> > > >> think
> > > >>>> we
> > > >>>>>>>>> need to
> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
> > > >>>>>> something
> > > >>>>>>>>>>>>> that is
> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> > > >>> managing
> > > >>>>> the
> > > >>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> > unmanaged).
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
> > in
> > > >>> the
> > > >>>>> Xen
> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> > Storage
> > > >>>>> layer to
> > > >>>>>>>>>>> perform
> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
> > > >> file,
> > > >>>> etc).
> > > >>>>>>>>>>> This
> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> > > >>>> hypervisor, as
> > > >>>>>>>>> such,
> > > >>>>>>>>>>>>>>> should be confined to its implementation. From my
> > > >>>>> perspective,
> > > >>>>>>>>>>> nothing
> > > >>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
> > From
> > > >>> its
> > > >>>>>>>>>>>>> perspective,
> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
> > to
> > > >>>> query
> > > >>>>>> the
> > > >>>>>>>>>>> data
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
> > level
> > > >>>>> layers
> > > >>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> > composition
> > > >>> of
> > > >>>>>>>>>>> operations
> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> > Volume
> > > >>>>> storage
> > > >>>>>>>>>>> protocol
> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> > implemented
> > > >> in
> > > >>>> my
> > > >>>>>>>>> patch,
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>> could do the following:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> > > >>>>>>>>> VolumeManagerImpl
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> > > >>>>> hypervisor.
> > > >>>>>> Before
> > > >>>>>>>>>>> this
> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> > dealing
> > > >>> with
> > > >>>>>>>>> Dynamic
> > > >>>>>>>>>>> (or
> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
> > > >>>> "Create
> > > >>>>>>>>> SR"
> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> > would
> > > >>> then
> > > >>>>>>>>> proceed
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
> > > >> add
> > > >>>>>> another
> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
> > > >> particular
> > > >>>>> type
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>> storage.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> What do you think?
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Hey John,
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
> > outlines
> > > >>> how
> > > >>>>> the
> > > >>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> > > >> use
> > > >>>> to
> > > >>>>>>>>> you, as
> > > >>>>>>>>>>>>> well,
> > > >>>>>>>>>>>>>>>> as you perform the code review.
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> I have shared this document with you (you should
> > have
> > > >>>>> received
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Talk to you later!
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > > >>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> > thoughts,
> > > >>> but
> > > >>>>>> want
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > > >>>>>>>>>>>>> disk_io_throttle,
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> > > >>>>> coherent
> > > >>>>>>>>> form
> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
> > the
> > > >>>> notion
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
> > > >> the
> > > >>>>> notion
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
> > was
> > > >>>>>>>>> incomplete,
> > > >>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>> I had
> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> > > >> device.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> > > >> this.
> > > >>>> :)
> > > >>>>> Do
> > > >>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>> have a
> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> > > >>> what
> > > >>>>> you're
> > > >>>>>>>>> looking
> > > >>>>>>>>>>>>>>>>>> for?
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > > >>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
> > the
> > > >>>> notion
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> > > >>>>> capabilities
> > > >>>>>>>>> (i.e.
> > > >>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> > > >>> managed
> > > >>>>> by
> > > >>>>>>>>> CloudStack)
> > > >>>>>>>>>>>>>>>>>> from the
> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
> > a
> > > >>>>> semantic
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
> > I
> > > >>> also
> > > >>>>> want
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
> > the
> > > >>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> definitions.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > > >>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> > questions
> > > >>> for
> > > >>>>> me
> > > >>>>>> with
> > > >>>>>>>>>>>>>>>>>> regards to
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
> > and
> > > >>> SAN
> > > >>>>>> volumes?
> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
> > (LUN
> > > >>> 0).
> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
> > and
> > > >>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> Volumes?
> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> > > >>> server or
> > > >>>>>>>>> externally
> > > >>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> > > >>> volume
> > > >>>>> (same
> > > >>>>>> as a
> > > >>>>>>>>>>> SAN
> > > >>>>>>>>>>>>>>>>>> LUN) is
> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
> > plug-in)
> > > >>>>> the
> > > >>>>>> first
> > > >>>>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
> > but
> > > >>>> still
> > > >>>>>>>>> want
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> use a
> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> > > >>> today
> > > >>>>>> (prior
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> 4.2). The
> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
> > in
> > > >>> this
> > > >>>>> case -
> > > >>>>>>>>>>>>>>>>>> multiple VMs
> > > >>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
> > you
> > > >>> can do
> > > >>>>> this
> > > >>>>>>>>>>>>> today,
> > > >>>>>>>>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
> > server
> > > >>> or
> > > >>>>> externally
> > > >>>>>>>>> by
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> > > >>> volumes
> > > >>>>> are
> > > >>>>>>>>>>>>> completely
> > > >>>>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
> > There
> > > >>> is
> > > >>>>> no
> > > >>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>> interaction.
> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> > > >>> volume
> > > >>>>> and a
> > > >>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > > >>>>> supports
> > > >>>>>> true
> > > >>>>>>>>>>> QoS.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> > > >> related
> > > >>>>> and
> > > >>>>>>>>>>>>> expected
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> operate.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> > multiple
> > > >>> VMs
> > > >>>>> with
> > > >>>>>>>>>>>>>>>>>> provisioned
> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> > > >>> single
> > > >>>>> SAN
> > > >>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>> (LUN) to a
> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> > > >>> multiple
> > > >>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> > guarantee
> > > >>> QoS.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> > volume
> > > >>> house
> > > >>>>> more
> > > >>>>>>>>> than
> > > >>>>>>>>>>>>> one
> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> > > >>> without
> > > >>>>> any of
> > > >>>>>> my
> > > >>>>>>>>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>>>> code.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> > volume
> > > >>> for
> > > >>>>>>>>>>> CloudStack's
> > > >>>>>>>>>>>>>>>>>> use
> > > >>>>>>>>>>>>>>>>>>>> onto
> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> > Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> > concerns me
> > > >>> --
> > > >>>>>> concepts
> > > >>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
> > today
> > > >>>> only
> > > >>>>>> used
> > > >>>>>>>>>>> (as
> > > >>>>>>>>>>>>>>>>>> far as
> > > >>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
> > not
> > > >>>>> PreSetup an
> > > >>>>>>>>> SR).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> > from
> > > >>>>> VMware, it
> > > >>>>>>>>> uses
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> vmfs
> > > >>>>>>>>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> > from
> > > >>> KVM,
> > > >>>>> it uses
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> > > >>> here
> > > >>>>> was to
> > > >>>>>>>>> make a
> > > >>>>>>>>>>>>>>>>>> new type
> > > >>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> > > >>> Edison
> > > >>>>> added
> > > >>>>>>>>> into
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> > > >>> specificy,
> > > >>>>> though:
> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> > > >>> Tutkowski
> > > >>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
> > know
> > > >>> or
> > > >>>>> care
> > > >>>>>> that
> > > >>>>>>>>> it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>> used
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > > >>>>> questions
> > > >>>>>>>>> about
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > > >>>>> SolidFire
> > > >>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>> needs to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> > > >>> hypervisor
> > > >>>>> logic
> > > >>>>>>>>>>> knows
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> treat
> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> > constraint
> > > >>>>> and just
> > > >>>>>>>>>>>>>>>>>> calling it
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
> > for
> > > >>> the
> > > >>>>> hypervisor
> > > >>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> > perform
> > > >>> the
> > > >>>>> necessary
> > > >>>>>>>>>>>>>>>>>> activities.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> > > >>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > > >>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
> > able to
> > > >>>>> manage a
> > > >>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > > >>> CloudStack
> > > >>>>> (e.g. the
> > > >>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>> shared
> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> > > >>> chosen
> > > >>>>> to
> > > >>>>>>>>> dedicate
> > > >>>>>>>>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>>>>>>>> portion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> > > >>> make
> > > >>>>> sense?"
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
> > SAN
> > > >>>>> volume via
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
> > would
> > > >>>>> work unless
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
> > an
> > > >>> SR.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > > >>>>> CloudStack, an
> > > >>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>> allocates
> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> > > >>> exposes it
> > > >>>>>> through
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> LUN. They
> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
> > to
> > > >>>>> CloudStack,
> > > >>>>>>>>> but
> > > >>>>>>>>>>>>>>>>>> not allow
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > > >>>>> changes, we
> > > >>>>>>>>> would
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>> passing
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > > >>>>> necessary
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>> (like
> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
> > have
> > > >>> the
> > > >>>>> storage
> > > >>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>> itself
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> > support
> > > >>> from
> > > >>>>> the
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> side and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
> > to
> > > >>> create
> > > >>>>> the
> > > >>>>>>>>> SR
> > > >>>>>>>>>>>>>>>>>> before the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
> > plugin
> > > >>>>> should
> > > >>>>>> have a
> > > >>>>>>>>>>>>> rich
> > > >>>>>>>>>>>>>>>>>> enough
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> > > >>> available
> > > >>>>> for
> > > >>>>>> volume
> > > >>>>>>>>>>>>>>>>>> storage.
> > > >>>>>>>>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > > >>> interrogation of
> > > >>>>> the
> > > >>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
> > itself
> > > >>>>> shouldn't
> > > >>>>>>>>> know
> > > >>>>>>>>>>>>>>>>>> or care
> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
> > able
> > > >>> to
> > > >>>>>> answer
> > > >>>>>>>>>>>>>>>>>> questions
> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> > > >>> driver
> > > >>>>> (which
> > > >>>>>>>>> is a
> > > >>>>>>>>>>>>>>>>>> type of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> > done, a
> > > >>> new
> > > >>>>> Primary
> > > >>>>>>>>>>>>>>>>>> Storage shows
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> > > >>> that
> > > >>>>>> references
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> tag
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> > CloudStack
> > > >>>>> volume. This
> > > >>>>>>>>> leads
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> a new
> > > >>>>>>>>>>>>>>>>>>>>>> row
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> > CloudStack
> > > >>>>> volume to a
> > > >>>>>>>>> VM
> > > >>>>>>>>>>>>>>>>>> (attach
> > > >>>>>>>>>>>>>>>>>>>>>> disk).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
> > calling
> > > >>> the
> > > >>>>> plug-in
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> > > >>> SAN).
> > > >>>>> The
> > > >>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>> also
> > > >>>>>>>>>>>>>>>>>>>>>> updates
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
> > data
> > > >>>>> (like the
> > > >>>>>>>>> IQN of
> > > >>>>>>>>>>>>>>>>>> the SAN
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
> > the
> > > >>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> volume is in
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
> > the
> > > >>>>> volume
> > > >>>>>>>>> will be
> > > >>>>>>>>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
> > the
> > > >>>>> plug-in
> > > >>>>>>>>> code
> > > >>>>>>>>>>>>>>>>>> will not
> > > >>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
> > and
> > > >>>>> detects
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
> > form
> > > >>> of a
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > > >>>>> terminology,
> > > >>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>> in these
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > > >>>>> allocation on
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> > > >>> mean
> > > >>>>> 'SAN
> > > >>>>>>>>> volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
> > user
> > > >>> can
> > > >>>>> make in
> > > >>>>>>>>>>>>>>>>>> CloudStack, I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
> > well
> > > >>> make
> > > >>>>> sense, but
> > > >>>>>>>>> I'm
> > > >>>>>>>>>>>>>>>>>> having a
> > > >>>>>>>>>>>>>>>>>>>>>> hard
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
> > in on
> > > >>>>> this
> > > >>>>>>>>> conversation
> > > >>>>>>>>>>>>>>>>>> as he
> > > >>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > > >>> approach I
> > > >>>>> took.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> > > >>> John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
> > two
> > > >>>>> distinct
> > > >>>>>>>>> concepts
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> > protocols.
> > > >>> In
> > > >>>>> both
> > > >>>>>>>>> cases, we
> > > >>>>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> > system to
> > > >>>>>> create/delete
> > > >>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
> > the
> > > >>>>> volume to
> > > >>>>>> be
> > > >>>>>>>>>>>>>>>>>> volume to be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> > context.
> > > >>> To
> > > >>>>> ensure
> > > >>>>>>>>> that
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>> are in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > > >>>>> definitions,
> > > >>>>>>>>> refers to
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> > correct?
> > > >>>>> Minimally,
> > > >>>>>> we
> > > >>>>>>>>>>>>> must
> > > >>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>> able
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
> > device
> > > >>> to
> > > >>>>> move bits
> > > >>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>> one place
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > > >>> Optionally, a
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> > Therefore,
> > > >>>>> we can have
> > > >>>>>> a
> > > >>>>>>>>>>>>>>>>>> unmanaged
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
> > and
> > > >>> we
> > > >>>>> can have a
> > > >>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> > > >>> capable
> > > >>>>> of
> > > >>>>>>>>> allocating
> > > >>>>>>>>>>>>>>>>>> LUNs and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> > CloudStack
> > > >>> may
> > > >>>>> be able
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> manage a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > > >>>>> unmanaged by
> > > >>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
> > the
> > > >>>>> operator has
> > > >>>>>>>>> chosen
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
> > Does
> > > >>> my
> > > >>>>>> reasoning
> > > >>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>> sense?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
> > are
> > > >>>>> reasonable, it
> > > >>>>>>>>> seems
> > > >>>>>>>>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> > concerns
> > > >>> from
> > > >>>>>>>>> StoragePoolType,
> > > >>>>>>>>>>>>>>>>>> add the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> > attached
> > > >>> driver
> > > >>>>> that
> > > >>>>>>>>>>> indicates
> > > >>>>>>>>>>>>>>>>>> whether
> > > >>>>>>>>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> > > >>> establish
> > > >>>>> a
> > > >>>>>>>>> abstraction
> > > >>>>>>>>>>>>>>>>>>>>>> representing a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
> > that
> > > >>> is
> > > >>>>>>>>> associated
> > > >>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>> volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> > hypervisor
> > > >>>>> drivers can
> > > >>>>>>>>>>>>> declare
> > > >>>>>>>>>>>>>>>>>> which
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> > encounter
> > > >>> a
> > > >>>>> device
> > > >>>>>> managed
> > > >>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> > exposed
> > > >>> by
> > > >>>>> the driver to
> > > >>>>>>>>>>>>> automate
> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> > > >>> sense,
> > > >>>>> then we can
> > > >>>>>>>>> sit
> > > >>>>>>>>>>>>>>>>>> down and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> > between
> > > >>> the
> > > >>>>> current
> > > >>>>>> iSCSI
> > > >>>>>>>>>>>>> type
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
> > go in
> > > >>> and
> > > >>>>> create
> > > >>>>>> a
> > > >>>>>>>>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
> > time,
> > > >>>>> the
> > > >>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>> volume must
> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> > allocated).
> > > >>>>> Future
> > > >>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
> > created
> > > >>>>> behind the
> > > >>>>>>>>>>>>> scenes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
> > to
> > > >>> go in
> > > >>>>> and
> > > >>>>>> create
> > > >>>>>>>>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
> > delete
> > > >>>>> volumes on
> > > >>>>>>>>> its
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
> > the
> > > >>>>> storage
> > > >>>>>>>>>>>>>>>>>> framework). When
> > > >>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
> > that
> > > >>> was
> > > >>>>> created,
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>>>> tells
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> > volume.
> > > >>> After
> > > >>>>> this is
> > > >>>>>>>>> done,
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
> > called.
> > > >>>>> No
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> > volume
> > > >>> was
> > > >>>>> just
> > > >>>>>>>>> created.
> > > >>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
> > PM,
> > > >>> Mike
> > > >>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
> > should
> > > >>>>> probably
> > > >>>>>> have
> > > >>>>>>>>>>>>>>>>>> defined them
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> > > >>> pre-
> > > >>>>> allocated
> > > >>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>> an admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> > > >>> hypervisor
> > > >>>>> data
> > > >>>>>>>>> structure
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
> > SR),
> > > >>> then
> > > >>>>> that
> > > >>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
> > Disks
> > > >>> (VDI)
> > > >>>>> are later
> > > >>>>>>>>>>>>> placed
> > > >>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> > > >>> these
> > > >>>>> cases, the
> > > >>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> > > >>> already
> > > >>>>> in place
> > > >>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> > structure
> > > >>> to
> > > >>>>> the VM in
> > > >>>>>>>>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
> > that is
> > > >>> not
> > > >>>>>>>>>>> pre-allocated.
> > > >>>>>>>>>>>>>>>>>> Instead
> > > >>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
> > could
> > > >>> be a
> > > >>>>> SAN
> > > >>>>>> (not
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> volume on
> > > >>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
> > data
> > > >>>>> structure
> > > >>>>>>>>> must be
> > > >>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> > because
> > > >>> these
> > > >>>>> types of
> > > >>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>> have not
> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > > >>>>> structure by an
> > > >>>>>>>>>>> admin.
> > > >>>>>>>>>>>>>>>>>> Once
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> > XenServer
> > > >>> for
> > > >>>>> this
> > > >>>>>>>>> volume,
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
> > VM
> > > >>> in
> > > >>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
> > PM,
> > > >>>>> John Burwell
> > > >>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> > implementation of
> > > >>> the
> > > >>>>> Dynamic
> > > >>>>>> type
> > > >>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>> behavior
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
> > why I
> > > >>> ask
> > > >>>>> about the
> > > >>>>>>>>>>>>>>>>>> difference.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
> > the
> > > >>>>> definition
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> > > >>> Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
> > type
> > > >>> is
> > > >>>>>> uniquely
> > > >>>>>>>>>>> used
> > > >>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> > > >>> Storage
> > > >>>>> that is
> > > >>>>>>>>> directly
> > > >>>>>>>>>>>>>>>>>> based on
> > > >>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> > > >>> step of
> > > >>>>> going
> > > >>>>>> to
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
> > on
> > > >>> that
> > > >>>>> iSCSI
> > > >>>>>>>>> target as
> > > >>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > > >>>>> supported for
> > > >>>>>>>>>>>>>>>>>> XenServer.
> > > >>>>>>>>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
> > first go
> > > >>> to
> > > >>>>> the
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>> perform
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
> > RBD is.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 2:13
> > > >>> PM,
> > > >>>>> John
> > > >>>>>> Burwell
> > > >>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
> > code,
> > > >>> what
> > > >>>>> is the
> > > >>>>>>>>>>> difference
> > > >>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
> > isn't
> > > >>> RBD
> > > >>>>> considered
> > > >>>>>>>>>>>>> Dynamic?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
> > PM,
> > > >>> Mike
> > > >>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> > storage is
> > > >>>>> defined in
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > > >>>>> StoragePoolType {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
> > local
> > > >>>>> directory
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > NetworkFilesystem(true),
> > > >>> //
> > > >>>>> NFS or CIFS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> > shared
> > > >>> LUN,
> > > >>>>> with a
> > > >>>>>>>>>>> clusterfs
> > > >>>>>>>>>>>>>>>>>>>> overlay
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
> > ZFS
> > > >>>>> Comstar
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
> > image
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> > XenServer
> > > >>> local
> > > >>>>> LVM SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > SharedMountPoint(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> > VMware
> > > >>> VMFS
> > > >>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > > >>>>> XenServer, Storage
> > > >>>>>>>>> Pool
> > > >>>>>>>>>>>>>>>>>> is set
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> > XenServer
> > > >>> local
> > > >>>>> EXT SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> > dynamic,
> > > >>>>> zone-wide
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > StoragePoolType(boolean
> > > >>>>> shared) {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> > isShared() {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 1:41
> > > >>> PM,
> > > >>>>> Mike
> > > >>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> > > >>> another
> > > >>>>> storage
> > > >>>>>>>>>>> company
> > > >>>>>>>>>>>>>>>>>> wants
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> > > >>> Quality
> > > >>>>> of
> > > >>>>>> Service
> > > >>>>>>>>>>>>>>>>>> feature. It
> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
> > well.
> > > >>>>> They would
> > > >>>>>>>>> need
> > > >>>>>>>>>>>>> only
> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
> > the
> > > >>>>> necessary
> > > >>>>>>>>> changes to
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
> > in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > > >>> 1:39
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
> > the
> > > >>>>> XenServer code
> > > >>>>>> is
> > > >>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> > attach
> > > >>> logic is
> > > >>>>> now aware
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > > >>>>> implementation of
> > > >>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> > > >>> with
> > > >>>>> Edison's
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
> > new
> > > >>>>> framework that
> > > >>>>>>>>>>> supported
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> > dynamically.
> > > >>>>> However, when I
> > > >>>>>>>>>>>>>>>>>> visited with
> > > >>>>>>>>>>>>>>>>>>>>>> him
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
> > that
> > > >>> it
> > > >>>>> was not
> > > >>>>>>>>>>>>>>>>>> complete. We
> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
> > could
> > > >>> do
> > > >>>>> with these
> > > >>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>> unless
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> > recognize
> > > >>> this
> > > >>>>> new type of
> > > >>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> > structure.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > > >>> 1:28
> > > >>>>> PM, John
> > > >>>>>>>>> Burwell
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
> > to me
> > > >>>>> that any
> > > >>>>>>>>>>> operation
> > > >>>>>>>>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> > > >>> about
> > > >>>>> details. I
> > > >>>>>>>>> expect
> > > >>>>>>>>>>>>>>>>>> to see
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> > > >>> operations
> > > >>>>> that can be
> > > >>>>>>>>>>>>>>>>>>>> composed/driven
> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > > >>>>> allocate
> > > >>>>>> space/create
> > > >>>>>>>>>>>>>>>>>> structures
> > > >>>>>>>>>>>>>>>>>>>>>> per
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > > >>>>> dependency, we are
> > > >>>>>>>>> going
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> end
> > > >>>>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
> > make
> > > >>> the
> > > >>>>> system
> > > >>>>>>>>>>>>> increasingly
> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > > >>>>> understanding that the
> > > >>>>>>>>> Xen
> > > >>>>>>>>>>>>>>>>>> specific
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > > >>>>> CitrixResourceBase
> > > >>>>>> class?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > > >>> 3:12
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
> > in a
> > > >>> bit
> > > >>>>> more
> > > >>>>>>>>> detail:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
> > aside
> > > >>> from
> > > >>>>> one setup
> > > >>>>>>>>>>>>> method
> > > >>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> > > >>> volume on
> > > >>>>> the
> > > >>>>>> storage
> > > >>>>>>>>>>>>>>>>>> system,
> > > >>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> > > >>> structure
> > > >>>>> to make
> > > >>>>>> use
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > > >>> XenServer
> > > >>>>> or a
> > > >>>>>> datastore
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> ESX(i)).
> > > >>>>>>>>>>>>>>>>>>>>>> VMs
> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> > storage
> > > >>>>> system's volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> > > >>> storage
> > > >>>>> framework,
> > > >>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> > > >>> create
> > > >>>>> a 1:1
> > > >>>>>>>>>>>>> relationship
> > > >>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
> > VM's
> > > >>> data
> > > >>>>> disk
> > > >>>>>>>>> (necessary
> > > >>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
> > a
> > > >>> plug-
> > > >>>>> in that is
> > > >>>>>>>>> called
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> > needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
> > the
> > > >>>>> storage
> > > >>>>>> framework
> > > >>>>>>>>> did
> > > >>>>>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>>>> address
> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > > >>> hypervisor-
> > > >>>>> specific data
> > > >>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>> when
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
> > case,
> > > >>> I've
> > > >>>>> been
> > > >>>>>>>>> enhancing
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>>>>>>>> so.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > > >>>>> submitted. I've got
> > > >>>>>>>>> ESX(i)
> > > >>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
> > 4.2
> > > >>>>> freeze date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
> > bit? :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > > >>>>> 1:03 PM, Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> > plug-in -
> > > >>> by
> > > >>>>> itself - is
> > > >>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
> > the
> > > >>>>> volume-attach
> > > >>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>> (in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
> > into
> > > >>> the
> > > >>>>> plug-in to
> > > >>>>>>>>> have
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> > > >>> comes to
> > > >>>>> attach the
> > > >>>>>>>>> volume
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> > > >>> enough
> > > >>>>> to recognize
> > > >>>>>>>>> it's
> > > >>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> > volume
> > > >>> has
> > > >>>>> just been
> > > >>>>>>>>> created)
> > > >>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> create,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > > >>>>> XenServer) or a
> > > >>>>>>>>> datastore
> > > >>>>>>>>>>>>>>>>>> (for
> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
> > just
> > > >>>>> created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> > spending
> > > >>> most
> > > >>>>> of my time
> > > >>>>>>>>>>>>> recently
> > > >>>>>>>>>>>>>>>>>> making
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> > code.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
> > up?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > > >>> at
> > > >>>>> 12:48 PM,
> > > >>>>>> John
> > > >>>>>>>>>>>>>>>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> > > >>> why
> > > >>>>> the the storage
> > > >>>>>>>>>>>>> driver
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> > at
> > > >>> 1:21
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> > ultimately I
> > > >>>>> would like to
> > > >>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
> > think
> > > >>> I'm
> > > >>>>> just out of
> > > >>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>> for 4.2
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> > > >>> plug-in
> > > >>>>> supports
> > > >>>>>>>>>>>>> XenServer.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> > > >>> feature
> > > >>>>> freeze, I
> > > >>>>>>>>> have
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> working
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
> > be
> > > >>> clear,
> > > >>>>> this is
> > > >>>>>> all
> > > >>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> regards
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > Compute
> > > >>>>> Offerings post
> > > >>>>>> 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > > >>> 2013
> > > >>>>> at 11:14 AM,
> > > >>>>>>>>> Kelcey
> > > >>>>>>>>>>>>>>>>>> Jamison
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
> > plan
> > > >>> on
> > > >>>>> supporting
> > > >>>>>>>>> KVM in
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > > >>>>> Message -----
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > > >>>>> Tutkowski" <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > > >>>>> dev@cloudstack.apache.org
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> > Monday,
> > > >>> June
> > > >>>>> 3, 2013
> > > >>>>>>>>> 10:12:32 AM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> > > >>> [MERGE]
> > > >>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> MASTER
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> > > >>> merging
> > > >>>>> Wei's feature
> > > >>>>>>>>>>> first,
> > > >>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>> mine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
> > is
> > > >>> for
> > > >>>>> KVM only,
> > > >>>>>>>>> then
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> is a
> > > >>>>>>>>>>>>>>>>>>>> non
> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> > 3,
> > > >>> 2013
> > > >>>>> at 8:55 AM,
> > > >>>>>> Wei
> > > >>>>>>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> > billing,
> > > >>> as
> > > >>>>> no one works
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> billing
> > > >>>>>>>>>>>>>>>>>>>> now,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
> > by
> > > >>>>> themselves. They
> > > >>>>>>>>> can
> > > >>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_offering of
> > > >>> a
> > > >>>>> VMs and
> > > >>>>>> volumes
> > > >>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
> > the
> > > >>>>> exact
> > > >>>>>> limitation
> > > >>>>>>>>>>>>> value
> > > >>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > limitation
> > > >>> for
> > > >>>>> nics as
> > > >>>>>> well.
> > > >>>>>>>>> I
> > > >>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
> > I/O
> > > >>>>> throttling?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > > >>>>> misunstand the second
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> > > >>> what
> > > >>>>> you said about
> > > >>>>>> the
> > > >>>>>>>>>>> two
> > > >>>>>>>>>>>>>>>>>>>> features.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> > 2013,
> > > >>> at
> > > >>>>> 2:13 AM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> > Mike's
> > > >>>>> aswer helps you.
> > > >>>>>>>>> I am
> > > >>>>>>>>>>>>>>>>>> trying
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> > think
> > > >>> billing
> > > >>>>> should
> > > >>>>>>>>> depend
> > > >>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> IO
> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > limitation.
> > > >>> Please
> > > >>>>> review
> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
> > > >>>>>>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
> > IO
> > > >>>>> statistics
> > > >>>>>> including
> > > >>>>>>>>>>>>>>>>>> bytes/iops
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> > > >>> machine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
> > the
> > > >>> AWS
> > > >>>>> model,
> > > >>>>>> customers
> > > >>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>> billed
> > > >>>>>>>>>>>>>>>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > provisioned
> > > >>> IOPS,
> > > >>>>> as well as,
> > > >>>>>> for
> > > >>>>>>>>>>>>> those
> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://aws.amazon.com/ebs/).
> > > >>>>>> I
> > > >>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>> imagine
> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> > > >>> employ
> > > >>>>> similar cost
> > > >>>>>>>>>>> models.
> > > >>>>>>>>>>>>>>>>>> Could
> > > >>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
> > model
> > > >>> in
> > > >>>>> the current
> > > >>>>>>>>> patch?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
> > you
> > > >>> mean
> > > >>>>> IOPS runtime
> > > >>>>>>>>>>> change?
> > > >>>>>>>>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > limitation for
> > > >>> a
> > > >>>>> running
> > > >>>>>> virtual
> > > >>>>>>>>>>>>>>>>>> machine
> > > >>>>>>>>>>>>>>>>>>>>>> through
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > CloudStack
> > > >>> does
> > > >>>>> not support
> > > >>>>>>>>>>> changing
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (computer
> > > >>>>> offering or disk
> > > >>>>>>>>>>>>> offering).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
> > the
> > > >>>>> Java interface
> > > >>>>>>>>> level.
> > > >>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>> apologize
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > > >>>>> generalize allocation
> > > >>>>>>>>>>>>>>>>>> algorithms
> > > >>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
> > the
> > > >>>>> service guarantees
> > > >>>>>>>>>>>>>>>>>> provided by a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> > > >>> good
> > > >>>>> question.
> > > >>>>>>>>> Maybe it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> disk_io_throttling as Mike
> > > >>>>>>>>> needs to
> > > >>>>>>>>>>>>>>>>>> consider
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
> > think.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> > expand
> > > >>> on
> > > >>>>> my thoughts
> > > >>>>>> in a
> > > >>>>>>>>>>>>> later
> > > >>>>>>>>>>>>>>>>>>>>>> response
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
> > points
> > > >>>>> between these two
> > > >>>>>>>>>>>>>>>>>> features. I
> > > >>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
> > to be
> > > >>>>> merged before
> > > >>>>>>>>>>>>>>>>>> SolidFire, but
> > > >>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > coordination
> > > >>>>> between the
> > > >>>>>> branches
> > > >>>>>>>>>>>>>>>>>> (possibly
> > > >>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> disk_io_throttling)
> > > >>>>> to
> > > >>>>>>>>> coordinate on
> > > >>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>> issue.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > > >>> John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > things I
> > > >>>>> want to
> > > >>>>>> understand
> > > >>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
> > there
> > > >>>>> value in
> > > >>>>>> capturing
> > > >>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>> policies
> > > >>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > model
> > > >>> (e.g.
> > > >>>>> for
> > > >>>>>>>>> billing/usage
> > > >>>>>>>>>>>>>>>>>>>> purposes,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> > Should
> > > >>> there
> > > >>>>> be a common
> > > >>>>>>>>>>>>>>>>>> interface model
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > provisioning
> > > >>> at
> > > >>>>> runtime?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
> > are
> > > >>>>> conflicting
> > > >>>>>>>>> provisioned
> > > >>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > hypervisor
> > > >>> and
> > > >>>>> storage device
> > > >>>>>>>>>>>>>>>>>> reconciled?
> > > >>>>>>>>>>>>>>>>>>>> In
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
> > to
> > > >>>>> believe (and
> > > >>>>>> billed)
> > > >>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>> device
> > > >>>>> has been
> > > >>>>>>>>> configured
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > consistent
> > > >>>>> configuration
> > > >>>>>>>>> between a
> > > >>>>>>>>>>>>>>>>>> VM and a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
> > but a
> > > >>>>> later
> > > >>>>>> modification
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> inconsistency.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
> > 2,
> > > >>> 2013,
> > > >>>>> at 8:38 PM,
> > > >>>>>>>>> Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > believe
> > > >>> Wei's
> > > >>>>> feature deals
> > > >>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>> controlling
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>> hypervisor
> > > >>>>> side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > feature
> > > >>> is
> > > >>>>> focused on
> > > >>>>>>>>>>>>> controlling
> > > >>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> > that
> > > >>>>> helps. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
> > Jun
> > > >>> 2,
> > > >>>>> 2013 at 6:35
> > > >>>>>> PM,
> > > >>>>>>>>>>>>> John
> > > >>>>>>>>>>>>>>>>>>>> Burwell
> > > >>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > > >>> opinion is
> > > >>>>> that no
> > > >>>>>> features
> > > >>>>>>>>>>>>>>>>>> should be
> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> > > >>> have
> > > >>>>> been resolved
> > > >>>>>> and
> > > >>>>>>>>> it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>> ready
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > Ops
> > > >>> vs
> > > >>>>> discrete
> > > >>>>>>>>> read/write
> > > >>>>>>>>>>>>>>>>>> ops issue
> > > >>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> > reviewed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > I
> > > >>> don't
> > > >>>>> think this
> > > >>>>>>>>> criteria
> > > >>>>>>>>>>>>>>>>>> has been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> > how
> > > >>>>> does this work
> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> > > >>>>>>>>>>>>>>>>>> )?
> > > >>>>>>>>>>>>>>>>>>>> As I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> > > >>> involves
> > > >>>>> provisioned
> > > >>>>>>>>> IOPS. I
> > > >>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > scenario
> > > >>>>> where provisioned
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> KVM and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > unnecessarily
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> incompatible.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Jun 1,
> > > >>> 2013,
> > > >>>>> at 6:47 AM,
> > > >>>>>>>>> Wei
> > > >>>>>>>>>>>>>>>>>> ZHOU <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
> > will
> > > >>>>> change it next
> > > >>>>>>>>> week.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/6/1
> > > >>>>> Wido den Hollander
> > > >>>>>> <
> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 06/01/2013 08:24 AM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Exactly. I
> > > >>> have
> > > >>>>> pushed the
> > > >>>>>>>>>>>>> features
> > > >>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > anyone
> > > >>>>> object thems for
> > > >>>>>>>>>>>>> technical
> > > >>>>>>>>>>>>>>>>>>>> reason
> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> > the
> > > >>> sake
> > > >>>>> of clarity I
> > > >>>>>> just
> > > >>>>>>>>>>>>> want
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> mention
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > total
> > > >>> IOps
> > > >>>>> to R/W IOps
> > > >>>>>>>>> asap
> > > >>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > total
> > > >>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
> > laid
> > > >>> the
> > > >>>>> groundwork for
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > however
> > > >>>>> prevent that we
> > > >>>>>> create
> > > >>>>>>>>>>>>>>>>>> legacy from
> > > >>>>>>>>>>>>>>>>>>>>>> day
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/5/31
> > > >>>>> Wido den
> > > >>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/31/2013 03:59 PM, John
> > > >>>>>>>>>>>>> Burwell
> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
> > this
> > > >>>>> enhancement must
> > > >>>>>> to
> > > >>>>>>>>>>>>>>>>>> discretely
> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > see
> > > >>> how
> > > >>>>> it could be
> > > >>>>>>>>> fixed
> > > >>>>>>>>>>>>>>>>>> later
> > > >>>>>>>>>>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > correctly
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
> > total
> > > >>> IOPS
> > > >>>>> into read
> > > >>>>>> and
> > > >>>>>>>>>>>>>>>>>> write.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>> unless/until we
> > > >>>>>> decided
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> break
> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
> > Wei
> > > >>>>> meant was merging
> > > >>>>>> it
> > > >>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>>>>>>>> now
> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > branch
> > > >>>>> and add Read /
> > > >>>>>>>>> Write
> > > >>>>>>>>>>>>> IOps
> > > >>>>>>>>>>>>>>>>>>>> before
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > > >>>>> released with Read
> > > >>>>>> and
> > > >>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>> instead
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
> > to
> > > >>>>> make the May 31st
> > > >>>>>>>>>>>>> feature
> > > >>>>>>>>>>>>>>>>>>>> freeze
> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
> > other
> > > >>>>> threads) then it
> > > >>>>>>>>> won't
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>> necessary
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > > >>>>> completely agree that
> > > >>>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>>> is no
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > I/O.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > May 31,
> > > >>>>> 2013, at 9:51 AM,
> > > >>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>> den
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/31/2013 03:13 PM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> > Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks.
> > > >>> Good
> > > >>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > thought
> > > >>>>> about at the
> > > >>>>>>>>>>> beginning.
> > > >>>>>>>>>>>>>>>>>>>> Finally I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > difference
> > > >>> of
> > > >>>>> read and write
> > > >>>>>>>>>>>>> mainly
> > > >>>>>>>>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
> > the
> > > >>>>> difference of sent
> > > >>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> received
> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > > >>>>> reasoning seems odd.
> > > >>>>>>>>>>>>>>>>>> Networking and
> > > >>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > I/O is
> > > >>>>> much more
> > > >>>>>>>>> expensive
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> most
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > bandwith.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> Implementing
> > > >>>>> it will be some
> > > >>>>>>>>>>>>>>>>>> copy-paste
> > > >>>>>>>>>>>>>>>>>>>>>> work.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> implemented
> > > >>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
> > days.
> > > >>> For
> > > >>>>> the deadline
> > > >>>>>> of
> > > >>>>>>>>>>>>>>>>>> feature
> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
> > if
> > > >>>>> needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> > think it's
> > > >>> a
> > > >>>>> feature we
> > > >>>>>>>>> can't
> > > >>>>>>>>>>>>>>>>>> miss. But
> > > >>>>>>>>>>>>>>>>>>>>>> if
> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > window
> > > >>> we
> > > >>>>> have to make sure
> > > >>>>>> we
> > > >>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
> > 4.3,
> > > >>> that
> > > >>>>> will confuse
> > > >>>>>>>>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/5/31
> > > >>>>> Wido den
> > > >>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/30/2013 06:03 PM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > would
> > > >>> like to
> > > >>>>> merge
> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > nobody
> > > >>>>> object, I will
> > > >>>>>> merge
> > > >>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>> purpose
> > > >>>>> is :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > > >>>>> machines are running
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> same
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > storage or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> > > >>> strage).
> > > >>>>> Because of
> > > >>>>>> the
> > > >>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>>>> limitation
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
> > if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
> > VM
> > > >>> has
> > > >>>>> large disk
> > > >>>>>>>>> operation,
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>> affect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > VMs
> > > >>>>> running on the
> > > >>>>>> same
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> device.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > >>> neccesary
> > > >>>>> to set the
> > > >>>>>>>>>>> maximum
> > > >>>>>>>>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Looking at
> > > >>> the
> > > >>>>> code I see
> > > >>>>>> you
> > > >>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
> > and
> > > >>>>> libvirt support
> > > >>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>> both a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > >>> which
> > > >>>>> could benefit a
> > > >>>>>>>>> lot of
> > > >>>>>>>>>>>>>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
> > also
> > > >>>>> strange, in the
> > > >>>>>>>>> polling
> > > >>>>>>>>>>>>>>>>>> side you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > but
> > > >>> on
> > > >>>>> the throttling
> > > >>>>>>>>> side
> > > >>>>>>>>>>>>>>>>>> you only
> > > >>>>>>>>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > IOps
> > > >>> are
> > > >>>>> usually much
> > > >>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>> expensive
> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
> > valid
> > > >>>>> use-case where
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > vs
> > > >>> Read
> > > >>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
> > this
> > > >>> only
> > > >>>>> supports KVM
> > > >>>>>>>>> at
> > > >>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>> point I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > to at
> > > >>>>> least have the
> > > >>>>>>>>>>>>>>>>>> mechanism in
> > > >>>>>>>>>>>>>>>>>>>>>> place
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> implementing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > later
> > > >>>>> would be a lot of
> > > >>>>>>>>>>> work.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> > > >>> hypervisor
> > > >>>>> doesn't
> > > >>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > you
> > > >>> can
> > > >>>>> always sum
> > > >>>>>> both
> > > >>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>> and set
> > > >>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > you
> > > >>>>> explain why you
> > > >>>>>>>>>>>>> implemented
> > > >>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>> feature
> > > >>>>> includes:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> > the
> > > >>>>> maximum rate of
> > > >>>>>>>>> VMs
> > > >>>>>>>>>>>>> (in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> configuration)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > change
> > > >>> the
> > > >>>>> maximum rate
> > > >>>>>> of
> > > >>>>>>>>>>> VMs
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > limit
> > > >>> the
> > > >>>>> disk rate
> > > >>>>>> (total
> > > >>>>>>>>>>> bps
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> iops)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
> > ticket:
> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> jira/browse/CLOUDSTACK-1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>> 1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 1192
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
> > will
> > > >>>>> update later) :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>> VM+Disk+IO+Throttling<https://
> > > >>>>>>>>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>
> > > >>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > >>>>> rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Merge
> > > >>> check
> > > >>>>> list :-
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
> > you
> > > >>>>> check the branch's
> > > >>>>>>>>> RAT
> > > >>>>>>>>>>>>>>>>>> execution
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> > > >>> there
> > > >>>>> new dependencies
> > > >>>>>>>>>>>>>>>>>> introduced?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > What
> > > >>>>> automated testing
> > > >>>>>> (unit
> > > >>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > feature?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
> > tests
> > > >>> are
> > > >>>>> added.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > What
> > > >>>>> testing has been done
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> check for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> > the
> > > >>>>> bytes rate and
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>> rate on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > VM
> > > >>>>> operations, including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > deploy,
> > > >>> stop,
> > > >>>>> start, reboot,
> > > >>>>>>>>>>>>>>>>>> destroy,
> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > > >>> Volume
> > > >>>>> operations,
> > > >>>>>>>>> including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > > >>>>> Detach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> > review
> > > >>> the
> > > >>>>> code, you can
> > > >>>>>>>>> try
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> > > >>> regards,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>> VM+Disk+IO+Throttling<https://
> > > >>>>>>>>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>
> > > >>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > >>>>> rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > > >>>>>>>>> refs/heads/disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>>>> 1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 1301
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/****jira/**browse/CLOUDSTACK-
> > > >>> 2071
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 2071
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <h**ttps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> (**CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > VM
> > > >>> Disk
> > > >>>>> I/O
> > > >>>>>> Throttling)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > >>>>> Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>>>> CloudStack Developer,
> > > >>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > > >>> 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Advancing
> > > >>> the
> > > >>>>> way the world
> > > >>>>>>>>> uses
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > >>> Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>> CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > > >>> way
> > > >>>>> the world uses
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>> CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > > >>> way
> > > >>>>> the world uses
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > >>> the
> > > >>>>> world uses the
> > > >>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > >>>>> Developer,
> > > >>>>>> SolidFire
> > > >>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > >>> the
> > > >>>>> world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>>>> Developer, SolidFire
> > > >>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > >>>>> world uses the
> > > >>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>> Developer,
> > > >>>>> SolidFire
> > > >>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > >>> world
> > > >>>>> uses the
> > > >>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>> Developer,
> > > >>>>> SolidFire
> > > >>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > > >>> world
> > > >>>>> uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>>>> SolidFire
> > > >>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > >>> uses
> > > >>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>>>> SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > >>> uses
> > > >>>>> the cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>>>> SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > >>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > >>> the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>> *(tm)*
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> *Mike Tutkowski*
> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>> o: 303.746.7302
> > > >>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>> *(tm)*
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> *Mike Tutkowski*
> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>> o: 303.746.7302
> > > >>>>>>> Advancing the way the world uses the
> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>> *(tm)*
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> *Mike Tutkowski*
> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>> e: mike.tutkowski@solidfire.com
> > > >>>>> o: 303.746.7302
> > > >>>>> Advancing the way the world uses the
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>> *(tm)*
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> *Mike Tutkowski*
> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> e: mike.tutkowski@solidfire.com
> > > >>> o: 303.746.7302
> > > >>> Advancing the way the world uses the
> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>> *(tm)*
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkowski@solidfire.com
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > *(tm)*
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
>
>
>
>
>
>
> --
> Mike Tutkowski
>
> Senior CloudStack Developer, SolidFire Inc.
>
> e: mike.tutkowski@solidfire.com
>
> o: 303.746.7302
>
> Advancing the way the world uses the cloudâ„¢
>
>
>
>
>
>
> --
> Mike Tutkowski
>
> Senior CloudStack Developer, SolidFire Inc.
>
> e: mike.tutkowski@solidfire.com
>
> o: 303.746.7302
>
> Advancing the way the world uses the cloudâ„¢
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Sounds good, Edison :)
I can implement John's isManaged() logic and resubmit my code for review.
Perhaps in 4.3 I could address the direct-attach scenario. If it requires
some refactoring of the storage framework, then I could do that.
Also, the default storage plug-in does have hypervisor logic in it in 4.2.
I recommend we leave this as is for 4.2 and it can be refactored in 4.3
with the above work I mentioned.
Are you OK with that, guys?
On Mon, Jun 10, 2013 at 11:30 AM, Edison Su <Ed...@citrix.com> wrote:
> Ok, I am Ok with the way you are talking about in 4.2.****
>
> ** **
>
> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> *Sent:* Sunday, June 09, 2013 8:59 PM
> *To:* dev@cloudstack.apache.org
> *Cc:* Edison Su; John Burwell
> *Subject:* Re: [MERGE] disk_io_throttling to MASTER****
>
> ** **
>
> Another thing we might want to consider for 4.2 is that the way the code
> is currently implemented in master, I don't think it's even possible to
> write a plug-in that connects storage directly to a VM because the
> CloudStack storage framework will call the "attach" method of the
> hypervisor in use and this method will fail because it assumes there is
> (talking Xen here) an SR existent and in this situation there isn't one.**
> **
>
> ** **
>
> My thinking is that we should ignore the direct-attach-of-volume-to-VM use
> case for 4.2. I think it will require too much of a rewrite at this point
> in development (fewer than three weeks left until code freeze).****
>
> ** **
>
> If the plug-in implements an isManaged() method (as John suggested), the
> storage framework can invoke the necessary commands on the hypervisor in
> use.****
>
> ** **
>
> For example, the createAsync method is called. The storage plug-in creates
> a SAN volume and updates the CS DB. The storage framework then asks the
> driver if it's managed. If it is, the storage framework executes a command
> against the hypervisor in use to have it create, say, an SR (with a single
> VDI that takes up the entire SR). Now, when the storage framework executes
> the "attach" command against the hypervisor, this command will work, as
> well (because that logic assumes the prior existence of an SR and it will,
> in fact, be present).****
>
> ** **
>
> On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:****
>
> Hi Edison and John,****
>
> ** **
>
> I wanted to point out something I thought was relevant to this
> conversation.****
>
> ** **
>
> My plug-in supports both XenServer and ESX(i).****
>
> ** **
>
> These two hypervisors handle their respective hypervisor data structures
> (XenServer SR or ESX(i) datastore) quite differently.****
>
> ** **
>
> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> when you issue a remove-SR command to XenServer.****
>
> ** **
>
> When you later want to access the contents of that data, it is still
> safely stored on the LUN and can be accessed from XenServer (from the same
> or different cluster (what XenServer calls a resource pool)) by creating a
> new SR that is based on the same IQN and LUN.****
>
> ** **
>
> XenServer SRs exist at the cluster level.****
>
> ** **
>
> In ESX(i), the datastore that resides on the SAN LUN is actually destroyed
> when you issue a remove-datastore command to ESX(i). In other words, the
> contents of the SAN LUN are destroyed.****
>
> ** **
>
> ESX(i) datastores exist at the datacenter level.****
>
> ** **
>
> The XenServer behavior (not destroying the contents of the LUN when the SR
> is removed) is desirable; the ESX(i) behavior (destroying the contents of
> the LUN when the datastore is removed) is not desirable.****
>
> ** **
>
> I can get around the ESX(i) behavior by removing the referenced IQN from
> each host in the cluster when the CS storage framework notifies my plug-in
> of a detach-volume event. When the CS volume is later attached again
> (either to the same or a different ESX(i) cluster), I can have each host in
> the relevant ESX(i) cluster reference the IQN and the datastore will be
> back again.****
>
> ** **
>
> This sequence of events is critical for my plug-in to support both
> environments:****
>
> *XenServer flow (events for Driver class):*****
>
> createAsync (called right before first attach) = creates SAN volume****
>
> preAttachVolume = creates SR and creates VDI or just introduces SR****
>
> (CS volume gets attached and later detached)****
>
> postDetachVolume = removes SR (this does not delete data on SAN volume)***
> *
>
> deleteAsync (can be called when volume is not attached) = deletes the SAN
> volume****
>
> *VMware flow (events for Driver class):*****
>
> createAsync (called right before first attach) = creates SAN volume,
> creates datastore, and creates VMDK file****
>
> preAttachVolume = adds iSCSI target to each host in cluster****
>
> (CS volume gets attached and later detached)****
>
> postDetachVolume = removes iSCSI target from each host in cluster****
>
> deleteAsync (can be called when volume is not attached) = deletes
> datastore and deletes the SAN volume****
>
> ************************
>
> So, the way I have it written up here, the driver is talking to the
> hypervisor. As John has pointed out, this may not be ideal.****
>
> ** **
>
> Whatever we decide on, it needs to be able to support this kind of flow. :)
> ****
>
> ** **
>
> We could go John's route, where the driver says if it's managed or not.***
> *
>
> ** **
>
> If it is, then the storage framework could send the necessary messages to
> the hypervisor in question at the right times (just migrate the hypervisor
> logic above from the driver to the right parts of the storage framework).*
> ***
>
> ** **
>
> This is fine, except that it does not resolve Edison's use case of
> desiring to have the storage LUN potentially connected directly to a VM.**
> **
>
> ** **
>
> I'm not sure we should design around that use case, though. Is it common
> enough? Has anyone asked for it? Is it even desirable? How are usage
> statistics gathered for such volumes? It's nice to offer the flexibility,
> but is it a good idea in this case?****
>
> ** **
>
> Thanks!****
>
> ** **
>
> ** **
>
> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:***
> *
>
>
>
> > -----Original Message-----
> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > Sent: Friday, June 07, 2013 4:08 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > Hi John,
> >
> > Yes, you are correct that Xen must layer an SR on top of the iSCSI volume
> > to use it. Same story for VMware: ESX(i) must layer a datastore on top of
> > the iSCSI volume to use it. I look at it like they are layering a
> clustered
> > file system on the SAN volume so the hypervisors can share access to the
> > contents of the volume.
> >
> > In 4.1, by the time the hypervisor-attach-volume logic is called, the SR
> or
> > datastore has already been created (usually manually by an admin).
> >
> > This pre-setup of the, say, SR is, of course, not acceptable in an
> > environment where each CloudStack volume that a user creates is mapped
> > to a
> > single SAN volume (via the SR).
> >
> > The question comes down to who should allocate the SR dynamically.
> >
> > We could have the storage framework ask the storage plug-in if it is
> > managed. If it is, then the storage framework could send a message to the
> > hypervisor in question to create (let's talk Xen here) the SR ahead of
> > time. Then, when the storage framework next sends the attach-volume
> > command
> > to the hypervisor, it should work without changes to that attach-volume
> > logic (because - from the point of view of the attach logic - the SR is
> > already existent, as expected).
> >
> > Now, as Edison has pointed out, this limits the power of the storage
> > plug-in. A storage plug-in in this model cannot directly attach storage
> to
> > a VM (it must go through the hypervisor). Perhaps that is OK. We need to
> > make a call on that.
>
>
> Yes, I think we need to address this usage case, and won't limit the power
> of the storage plug-in.
> John, what's your idea to fit this usage case into your abstraction?
>
>
> >
> > If we want to give more power to the storage vendor, we would have to
> > have
> > the storage framework call into the storage plug-in to send the
> appropriate
> > attach commands to the hypervisor.
> >
> > Let's discuss and come to a consensus (or at least agree on a path) as
> soon
> > as we can. :)
> >
> > Thanks!
> >
> >
> >
> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com> wrote:
> >
> > > Mike,
> > >
> > > My understanding is the Xen expects an SR structure on all ISCSI
> devices
> > > -- at least that is how I read the code in your patch. Is my
> understanding
> > > correct? If so, the Xen plugin should be able to query the storage
> device
> > > to determine the presence of the SR structure and create it if it does
> not
> > > exist. Am I missing something in the implementation that makes that
> type
> > > of implementation impossible?
> > >
> > > Thanks,
> > > -John
> > >
> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> > > wrote:
> > >
> > > > Yeah, if a storage vendor wanted to attach a volume directly to a VM
> in
> > > 4.2
> > > > today, it would probably fail because the attach-volume logic assumes
> > the
> > > > existence of the necessary hypervisor data structure (ex. SR on Xen).
> > > >
> > > > If we wanted to enable such an attach, we could do it the way Edison
> > > > suggests.
> > > >
> > > >
> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
> wrote:
> > > >
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > >>> Sent: Friday, June 07, 2013 2:37 PM
> > > >>> To: dev@cloudstack.apache.org
> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > > >>>
> > > >>> As we only have three weeks until feature freeze, we should come to
> > a
> > > >>> consensus on this design point as soon as possible.
> > > >>>
> > > >>> Right now, if the storage framework asks my driver it is is
> managed, it
> > > >>> will say 'yes.' This means the framework will tell the driver to
> > > perform
> > > >>> its management activities. This then means the driver will call
> into
> > > the
> > > >>> host (it doesn't know which hypervisor, by the way) to perform the
> > > >> activity
> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> > > >>>
> > > >>> The driver doesn't know which hypervisor it's talking to, it just
> > > sends a
> > > >>> message to the host to perform the necessary pre-attach work.
> > > >>
> > > >> Could we just expose a method like "attachVolume/dettachVolume" on
> > the
> > > >> PrimaryDataStoreDriver driver?
> > > >> For most of cases, the implementation of each driver would just send
> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
> > the
> > > >> implementation in a base class, so that can be shared by all of this
> > > >> drivers), and in each hypervisor resource code, which may just call
> > > >> hypervisor's api to attach the volume to VM, while for certain
> storage,
> > > >> like SolidFire, may need to create a SR first, and create volume on
> it,
> > > >> then call hypervisor's API to attach volume to VM.
> > > >> While for some other storage vendor, may want to bypass hypervisor
> > > during
> > > >> attaching volume, so inside driver's attachvolume implementation,
> the
> > > >> driver can do some magic, such as, directly talk to an agent inside
> VM
> > > >> instance, then create a disk inside VM.
> > > >>
> > > >> How do you guys think?
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> > > wrote:
> > > >>>
> > > >>>>
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> > > >>>>> To: dev@cloudstack.apache.org
> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > > >>>>>
> > > >>>>> Hi John,
> > > >>>>>
> > > >>>>> How's about this:
> > > >>>>>
> > > >>>>> The driver can implement an isManaged() method. The
> > > >>> VolumeManagerImpl
> > > >>>>> can
> > > >>>>> call into the driver to see if its managed. If it is, the
> > > >>>> VolumeManagerImpl
> > > >>>>> (which is responsible for calling into the hypervisor to attach
> the
> > > >> disk)
> > > >>>>> can call into the hypervisor to create the necessary hypervisor
> data
> > > >>>>> structure (ex. for XenServer, a storage repository).
> > > >>>>
> > > >>>> The problem here is that storage vendor may work differently with
> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> > > >> other
> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> > directly
> > > >> to
> > > >>>> VM instance, see the discuss(
> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> > > >>>
> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> > > >>> %3E
> > > >>>> ).
> > > >>>> So I would let storage provider to implement attach disk to VM,
> > > >> instead of
> > > >>>> implemented by cloudstack itself.
> > > >>>>
> > > >>>>
> > > >>>>>
> > > >>>>> If that's what you're going for, that works for me. By the way,
> > > >> Edison's
> > > >>>>> default storage plug-in (which handles the default storage
> behavior
> > > >> in
> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks
> to
> > > >>>>> hypervisors. You might want to contact him and inform him of your
> > > >>>> concerns
> > > >>>>> or that logic (as is) will make it to production.
> > > >>>>>
> > > >>>>> Please let me know if what I wrote in above (for my solution) is
> OK
> > > >> with
> > > >>>>> you. :)
> > > >>>>>
> > > >>>>> Thanks!
> > > >>>>>
> > > >>>>>
> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jburwell@basho.com
> >
> > > >>> wrote:
> > > >>>>>
> > > >>>>>> Mike,
> > > >>>>>>
> > > >>>>>> Please see my responses in-line below.
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> -John
> > > >>>>>>
> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > > >>>> mike.tutkowski@solidfire.com>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hey John,
> > > >>>>>>>
> > > >>>>>>> I still have a bit more testing I'd like to do before I build
> up
> > > >> a
> > > >>>> patch
> > > >>>>>>> file, but this is the gist of what I've done:
> > > >>>>>>>
> > > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
> > tells
> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if
> it
> > > >>>>> managed.
> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> > > >> whatever
> > > >>>>>>> activity is required. In my case, this includes sending a
> > > >> message to
> > > >>>> the
> > > >>>>>>> host where the VM is running to have, say XenServer, add a
> > > >> storage
> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
> > > >> SAN
> > > >>>>>> volume,
> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
> can
> > > >> on
> > > >>>> the
> > > >>>>>>> storage repository). After this, the normal attach-volume
> > > >> message is
> > > >>>> sent
> > > >>>>>>> to the host by VolumeManagerImpl.
> > > >>>>>>
> > > >>>>>> There should be **no** code from a storage driver to a
> hypervisor.
> > > >> I
> > > >>>>>> apologize for the repetition, but we simply can not have
> hypervisor
> > > >>>>>> specific code in the storage layer. The circular dependencies
> > > >> between
> > > >>>> the
> > > >>>>>> two layers are not sustainable in the long term. Either the
> > > >>>> VirtualManager
> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> > > >>> coordinate
> > > >>>>>> volume creation and then populating the SR. Ideally, we can
> > > >>>> generalize the
> > > >>>>>> process flow for attaching volumes such that the Xen hypervisor
> > > >> plugin
> > > >>>>>> would only implement callbacks to perform the attach action and
> > > >> create
> > > >>>>> the
> > > >>>>>> structure and SR. To my mind, the SolidFire driver should only
> be
> > > >>>>>> allocating space and providing information about contents (e.g.
> > > >> space
> > > >>>>>> available, space consumed, streams to a URI, file handle for a
> URI,
> > > >>>> etc)
> > > >>>>>> and capabilities.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> * The reverse is performed for a detach-volume command.
> > > >>>>>>>
> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
> driver.
> > > >>>>>> Edison's
> > > >>>>>>> default driver simply does a "return false;". We could add a
> new
> > > >>>>>> parameter
> > > >>>>>>> to the createStoragePool API command, if we want, to remove
> > the
> > > >>>>>> hard-coded
> > > >>>>>>> return values in the drivers (although my driver will probably
> > > >> just
> > > >>>>>> ignore
> > > >>>>>>> this parameter and always return true since it wouldn't make
> > > >> sense
> > > >>>> for it
> > > >>>>>>> to ever return false). We'd need another column in the
> > > >> storage_pool
> > > >>>>> table
> > > >>>>>>> to store this value.
> > > >>>>>>
> > > >>>>>> Yes, I think we should have a parameter added to the
> > > >>> createStoragePool
> > > >>>>>> surfaced to the HTTP API that allows DataStores to be configured
> > > >> for
> > > >>>>>> management when their underlying drivers support it. To
> simplify
> > > >>>> things,
> > > >>>>>> this flag should only be mutable when the DataStore is created.
> It
> > > >>>> would be
> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
> > > >>>> creation.
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> Sound like I'm in sync with what you were thinking?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > > >>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> I agree, John. Just wanted to point out that I have a working
> > > >> GUI
> > > >>>> for
> > > >>>>>> you
> > > >>>>>>>> to review (in that document), if you'd like to check it out.
> > > >>>>>>>>
> > > >>>>>>>> Thanks!
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> > > >> jburwell@basho.com>
> > > >>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Mike,
> > > >>>>>>>>>
> > > >>>>>>>>> I would like the UIs of two features reviewed together to
> > > >> ensure
> > > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
> > > >> and
> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
> > > >>>> confusion,
> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
> > > >> help
> > > >>>>>>>>> minimize any potential confusion.
> > > >>>>>>>>>
> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
> itself
> > > >> if
> > > >>>> it
> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> > > >>>> throttled
> > > >>>>>>>>> IOPs or storage provisioned IOPs.
> > > >>>>>>>>>
> > > >>>>>>>>> Thanks,
> > > >>>>>>>>> -John
> > > >>>>>>>>>
> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > > >>>>>>>>> <mi...@solidfire.com> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hi John,
> > > >>>>>>>>>>
> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
> > to
> > > >>> you
> > > >>>>> the
> > > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
> > > >> IOPS.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> > > >> and we
> > > >>>>>>>>> decided
> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> > Offering
> > > >>>> dialog.
> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> > should
> > > >>> be
> > > >>>>> able
> > > >>>>>> to
> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only use
> > > >> Max
> > > >>>>>> IOPS).
> > > >>>>>>>>>> These fields are optional and can be ignored for storage
> that
> > > >>>> does not
> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> > > >>>> admin can
> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
> > > >> IOPS.
> > > >>>>>>>>>>
> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
> > > >> mine,
> > > >>>> as
> > > >>>>>> is
> > > >>>>>>>>>> decided by the admin).
> > > >>>>>>>>>>
> > > >>>>>>>>>> I'm not sure what we can do about these two features
> > > >> expressing
> > > >>>> the
> > > >>>>>>>>> speed
> > > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS
> for
> > > >>>> QoS in
> > > >>>>>>>>> any
> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
> read/write
> > > >>>> IOPS).
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thanks
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > > >>> <jb...@basho.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Wei,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> > > >>> Storage/Hypervisor
> > > >>>>>> layer
> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
> > > >>>> hypervisor
> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> > > >> following
> > > >>>>>> issues
> > > >>>>>>>>>>> outstanding:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > > >>> read/write
> > > >>>>>>>>> values
> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
> > > >> IOPS.
> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> > > >>> storage
> > > >>>>>> volumes
> > > >>>>>>>>>>> with provisioned IOPS?
> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> > > >>>> provisioned
> > > >>>>>> IOPS
> > > >>>>>>>>>>> that will permit providers to discriminate these guaranteed
> > > >>>>>> operations
> > > >>>>>>>>> in
> > > >>>>>>>>>>> the event they want to bill for it differently?
> > > >>>>>>>>>>> 4. What is the user experience for throttled and
> provisioned
> > > >>> IOPS
> > > >>>>>> that
> > > >>>>>>>>>>> minimizes confusion of these concepts?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> > > >>>> throttled
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> solve
> > > >>>> items 1
> > > >>>>>>>>> and 2.
> > > >>>>>>>>>>> Since the two facilities would not be permitted to operate
> > > >>>> together,
> > > >>>>>>>>> they
> > > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
> > > >> can be
> > > >>>>>>>>> resolved
> > > >>>>>>>>>>> with an additional flag or two on the usage records. As
> for
> > > >>>> Item 4,
> > > >>>>>> I
> > > >>>>>>>>> am
> > > >>>>>>>>>>> not familiar with how these two enhancements are
> > depicted in
> > > >>> the
> > > >>>>> user
> > > >>>>>>>>>>> interface. I think we need to review the UI enhancements
> > for
> > > >>>> both
> > > >>>>>>>>>>> enhancements and ensure they are consistent.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Do these solutions make sense?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks,
> > > >>>>>>>>>>> -John
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> > <us...@gmail.com>
> > > >>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> John and Mike,
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> > 2780/2729,
> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > > >>> CLOUDSTACK-
> > > >>>>> 2875 ) this
> > > >>>>>>>>> week.
> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
> > > >> ask
> > > >>>> for
> > > >>>>>>>>> second
> > > >>>>>>>>>>>> merge request after testing.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
> John,
> > > >> and
> > > >>>>> have
> > > >>>>>>>>> been
> > > >>>>>>>>>>>>> re-working this section of code today.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks for the comments,
> > > >>>>>>>>>>>>> Mike
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > > >>>>> <jb...@basho.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> See my responses in-line below.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks for the response.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> > whether or
> > > >>>>> not a
> > > >>>>>>>>> Volume
> > > >>>>>>>>>>> is
> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> > > >>> actual
> > > >>>>>>>>> isManaged
> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> > > >> resources
> > > >>>> for
> > > >>>>>> the
> > > >>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>> if it is managed."
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> > method
> > > >>>>> added to
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> > > >> then the
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
> > > >>> method
> > > >>>>> (which
> > > >>>>>>>>>>> would
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
> > and
> > > >>> this
> > > >>>>>> would
> > > >>>>>>>>>>> call
> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
> > on
> > > >>>>>>>>> XenServer,
> > > >>>>>>>>>>> an
> > > >>>>>>>>>>>>> SR?
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> > > >>>>> DataStoreDriver.
> > > >>>>>>>>> Since
> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
> > I
> > > >>>> would
> > > >>>>>> add
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> > straw
> > > >>> man
> > > >>>>> term
> > > >>>>>>>>> --
> > > >>>>>>>>>>>>>> something other than volume) methods to
> > > >>>>>>>>>>> allocate/create/deallocate/delete
> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> > > >>> property
> > > >>>>> of
> > > >>>>>>>>>>>>> DataStore
> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> > > >>> DataStoreDriver
> > > >>>>> can be
> > > >>>>>>>>>>>>> managed.
> > > >>>>>>>>>>>>>> This approach allows operators to override
> > manageability
> > > >> of
> > > >>>>>> devices.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> > > >> plugin
> > > >>>>>> would
> > > >>>>>>>>> be
> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
> > to
> > > >>>>> create any
> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There should
> > > >> be no
> > > >>>>>>>>>>>>> dependencies
> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> > > >> earlier,
> > > >>>> such
> > > >>>>>>>>>>>>> circular
> > > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
> > mess.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks again, John
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > > >>>>> <jb...@basho.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
> > that
> > > >>>>>> supports n
> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> > > >> hypervisors.
> > > >>>>>> Such
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> > > >>> beast.
> > > >>>>>>>>>>>>> Therefore, my
> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
> > the
> > > >>>>> Storage
> > > >>>>>> layer
> > > >>>>>>>>>>>>> towards
> > > >>>>>>>>>>>>>>> this separation.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> > whether
> > > >>> or
> > > >>>>> not a
> > > >>>>>>>>> Volume
> > > >>>>>>>>>>> is
> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
> > an
> > > >>> actual
> > > >>>>>>>>>>> isManaged
> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> > > >> resources
> > > >>>> for
> > > >>>>>> the
> > > >>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
> > > >> to
> > > >>>>>> indicate
> > > >>>>>>>>>>>>> whether
> > > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
> > > >> think
> > > >>>> we
> > > >>>>>>>>> need to
> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to
> have
> > > >>>>>> something
> > > >>>>>>>>>>>>> that is
> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> > > >>> managing
> > > >>>>> the
> > > >>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> > unmanaged).
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
> > in
> > > >>> the
> > > >>>>> Xen
> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> > Storage
> > > >>>>> layer to
> > > >>>>>>>>>>> perform
> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
> > > >> file,
> > > >>>> etc).
> > > >>>>>>>>>>> This
> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> > > >>>> hypervisor, as
> > > >>>>>>>>> such,
> > > >>>>>>>>>>>>>>> should be confined to its implementation. From my
> > > >>>>> perspective,
> > > >>>>>>>>>>> nothing
> > > >>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
> > From
> > > >>> its
> > > >>>>>>>>>>>>> perspective,
> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
> > to
> > > >>>> query
> > > >>>>>> the
> > > >>>>>>>>>>> data
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
> > level
> > > >>>>> layers
> > > >>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> > composition
> > > >>> of
> > > >>>>>>>>>>> operations
> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> > Volume
> > > >>>>> storage
> > > >>>>>>>>>>> protocol
> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> > implemented
> > > >> in
> > > >>>> my
> > > >>>>>>>>> patch,
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>> could do the following:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> > > >>>>>>>>> VolumeManagerImpl
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> > > >>>>> hypervisor.
> > > >>>>>> Before
> > > >>>>>>>>>>> this
> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> > dealing
> > > >>> with
> > > >>>>>>>>> Dynamic
> > > >>>>>>>>>>> (or
> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> send a
> > > >>>> "Create
> > > >>>>>>>>> SR"
> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> > would
> > > >>> then
> > > >>>>>>>>> proceed
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
> > > >> add
> > > >>>>>> another
> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
> > > >> particular
> > > >>>>> type
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>> storage.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> What do you think?
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Hey John,
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
> > outlines
> > > >>> how
> > > >>>>> the
> > > >>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> > > >> use
> > > >>>> to
> > > >>>>>>>>> you, as
> > > >>>>>>>>>>>>> well,
> > > >>>>>>>>>>>>>>>> as you perform the code review.
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> I have shared this document with you (you should
> > have
> > > >>>>> received
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Talk to you later!
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > > >>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> > thoughts,
> > > >>> but
> > > >>>>>> want
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > > >>>>>>>>>>>>> disk_io_throttle,
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> > > >>>>> coherent
> > > >>>>>>>>> form
> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
> > the
> > > >>>> notion
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
> > > >> the
> > > >>>>> notion
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
> > was
> > > >>>>>>>>> incomplete,
> > > >>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>> I had
> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> > > >> device.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> > > >> this.
> > > >>>> :)
> > > >>>>> Do
> > > >>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>> have a
> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> > > >>> what
> > > >>>>> you're
> > > >>>>>>>>> looking
> > > >>>>>>>>>>>>>>>>>> for?
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > > >>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
> > the
> > > >>>> notion
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> > > >>>>> capabilities
> > > >>>>>>>>> (i.e.
> > > >>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> > > >>> managed
> > > >>>>> by
> > > >>>>>>>>> CloudStack)
> > > >>>>>>>>>>>>>>>>>> from the
> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
> > a
> > > >>>>> semantic
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
> > I
> > > >>> also
> > > >>>>> want
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
> > the
> > > >>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> definitions.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > > >>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> > questions
> > > >>> for
> > > >>>>> me
> > > >>>>>> with
> > > >>>>>>>>>>>>>>>>>> regards to
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
> > and
> > > >>> SAN
> > > >>>>>> volumes?
> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
> > (LUN
> > > >>> 0).
> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
> > and
> > > >>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> Volumes?
> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> > > >>> server or
> > > >>>>>>>>> externally
> > > >>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> > > >>> volume
> > > >>>>> (same
> > > >>>>>> as a
> > > >>>>>>>>>>> SAN
> > > >>>>>>>>>>>>>>>>>> LUN) is
> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
> > plug-in)
> > > >>>>> the
> > > >>>>>> first
> > > >>>>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
> > but
> > > >>>> still
> > > >>>>>>>>> want
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> use a
> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> > > >>> today
> > > >>>>>> (prior
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> 4.2). The
> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
> > in
> > > >>> this
> > > >>>>> case -
> > > >>>>>>>>>>>>>>>>>> multiple VMs
> > > >>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
> > you
> > > >>> can do
> > > >>>>> this
> > > >>>>>>>>>>>>> today,
> > > >>>>>>>>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
> > server
> > > >>> or
> > > >>>>> externally
> > > >>>>>>>>> by
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> > > >>> volumes
> > > >>>>> are
> > > >>>>>>>>>>>>> completely
> > > >>>>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
> > There
> > > >>> is
> > > >>>>> no
> > > >>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>> interaction.
> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> > > >>> volume
> > > >>>>> and a
> > > >>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > > >>>>> supports
> > > >>>>>> true
> > > >>>>>>>>>>> QoS.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> > > >> related
> > > >>>>> and
> > > >>>>>>>>>>>>> expected
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> operate.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> > multiple
> > > >>> VMs
> > > >>>>> with
> > > >>>>>>>>>>>>>>>>>> provisioned
> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> > > >>> single
> > > >>>>> SAN
> > > >>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>> (LUN) to a
> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> > > >>> multiple
> > > >>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> > guarantee
> > > >>> QoS.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> > volume
> > > >>> house
> > > >>>>> more
> > > >>>>>>>>> than
> > > >>>>>>>>>>>>> one
> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> > > >>> without
> > > >>>>> any of
> > > >>>>>> my
> > > >>>>>>>>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>>>> code.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> > volume
> > > >>> for
> > > >>>>>>>>>>> CloudStack's
> > > >>>>>>>>>>>>>>>>>> use
> > > >>>>>>>>>>>>>>>>>>>> onto
> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> > Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> > concerns me
> > > >>> --
> > > >>>>>> concepts
> > > >>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
> > today
> > > >>>> only
> > > >>>>>> used
> > > >>>>>>>>>>> (as
> > > >>>>>>>>>>>>>>>>>> far as
> > > >>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
> > not
> > > >>>>> PreSetup an
> > > >>>>>>>>> SR).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> > from
> > > >>>>> VMware, it
> > > >>>>>>>>> uses
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> vmfs
> > > >>>>>>>>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> > from
> > > >>> KVM,
> > > >>>>> it uses
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> > > >>> here
> > > >>>>> was to
> > > >>>>>>>>> make a
> > > >>>>>>>>>>>>>>>>>> new type
> > > >>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> > > >>> Edison
> > > >>>>> added
> > > >>>>>>>>> into
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> > > >>> specificy,
> > > >>>>> though:
> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> > > >>> Tutkowski
> > > >>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
> > know
> > > >>> or
> > > >>>>> care
> > > >>>>>> that
> > > >>>>>>>>> it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>> used
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > > >>>>> questions
> > > >>>>>>>>> about
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > > >>>>> SolidFire
> > > >>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>> needs to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> > > >>> hypervisor
> > > >>>>> logic
> > > >>>>>>>>>>> knows
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> treat
> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> > constraint
> > > >>>>> and just
> > > >>>>>>>>>>>>>>>>>> calling it
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
> > for
> > > >>> the
> > > >>>>> hypervisor
> > > >>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> > perform
> > > >>> the
> > > >>>>> necessary
> > > >>>>>>>>>>>>>>>>>> activities.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> > > >>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > > >>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
> > able to
> > > >>>>> manage a
> > > >>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > > >>> CloudStack
> > > >>>>> (e.g. the
> > > >>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>> shared
> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> > > >>> chosen
> > > >>>>> to
> > > >>>>>>>>> dedicate
> > > >>>>>>>>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>>>>>>>> portion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> > > >>> make
> > > >>>>> sense?"
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
> > SAN
> > > >>>>> volume via
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
> > would
> > > >>>>> work unless
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
> > an
> > > >>> SR.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > > >>>>> CloudStack, an
> > > >>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>> allocates
> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> > > >>> exposes it
> > > >>>>>> through
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> LUN. They
> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
> > to
> > > >>>>> CloudStack,
> > > >>>>>>>>> but
> > > >>>>>>>>>>>>>>>>>> not allow
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > > >>>>> changes, we
> > > >>>>>>>>> would
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>> passing
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > > >>>>> necessary
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>> (like
> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
> > have
> > > >>> the
> > > >>>>> storage
> > > >>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>> itself
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> > support
> > > >>> from
> > > >>>>> the
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> side and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
> > to
> > > >>> create
> > > >>>>> the
> > > >>>>>>>>> SR
> > > >>>>>>>>>>>>>>>>>> before the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
> > plugin
> > > >>>>> should
> > > >>>>>> have a
> > > >>>>>>>>>>>>> rich
> > > >>>>>>>>>>>>>>>>>> enough
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> > > >>> available
> > > >>>>> for
> > > >>>>>> volume
> > > >>>>>>>>>>>>>>>>>> storage.
> > > >>>>>>>>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > > >>> interrogation of
> > > >>>>> the
> > > >>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
> > itself
> > > >>>>> shouldn't
> > > >>>>>>>>> know
> > > >>>>>>>>>>>>>>>>>> or care
> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
> > able
> > > >>> to
> > > >>>>>> answer
> > > >>>>>>>>>>>>>>>>>> questions
> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> > > >>> driver
> > > >>>>> (which
> > > >>>>>>>>> is a
> > > >>>>>>>>>>>>>>>>>> type of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> > done, a
> > > >>> new
> > > >>>>> Primary
> > > >>>>>>>>>>>>>>>>>> Storage shows
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> > > >>> that
> > > >>>>>> references
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> tag
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> > CloudStack
> > > >>>>> volume. This
> > > >>>>>>>>> leads
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> a new
> > > >>>>>>>>>>>>>>>>>>>>>> row
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> > CloudStack
> > > >>>>> volume to a
> > > >>>>>>>>> VM
> > > >>>>>>>>>>>>>>>>>> (attach
> > > >>>>>>>>>>>>>>>>>>>>>> disk).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
> > calling
> > > >>> the
> > > >>>>> plug-in
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> > > >>> SAN).
> > > >>>>> The
> > > >>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>> also
> > > >>>>>>>>>>>>>>>>>>>>>> updates
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
> > data
> > > >>>>> (like the
> > > >>>>>>>>> IQN of
> > > >>>>>>>>>>>>>>>>>> the SAN
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
> > the
> > > >>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> volume is in
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
> > the
> > > >>>>> volume
> > > >>>>>>>>> will be
> > > >>>>>>>>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
> > the
> > > >>>>> plug-in
> > > >>>>>>>>> code
> > > >>>>>>>>>>>>>>>>>> will not
> > > >>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
> > and
> > > >>>>> detects
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
> > form
> > > >>> of a
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > > >>>>> terminology,
> > > >>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>> in these
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > > >>>>> allocation on
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> > > >>> mean
> > > >>>>> 'SAN
> > > >>>>>>>>> volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
> > user
> > > >>> can
> > > >>>>> make in
> > > >>>>>>>>>>>>>>>>>> CloudStack, I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
> > well
> > > >>> make
> > > >>>>> sense, but
> > > >>>>>>>>> I'm
> > > >>>>>>>>>>>>>>>>>> having a
> > > >>>>>>>>>>>>>>>>>>>>>> hard
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
> > in on
> > > >>>>> this
> > > >>>>>>>>> conversation
> > > >>>>>>>>>>>>>>>>>> as he
> > > >>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > > >>> approach I
> > > >>>>> took.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> > > >>> John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
> > two
> > > >>>>> distinct
> > > >>>>>>>>> concepts
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> > protocols.
> > > >>> In
> > > >>>>> both
> > > >>>>>>>>> cases, we
> > > >>>>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> > system to
> > > >>>>>> create/delete
> > > >>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
> > the
> > > >>>>> volume to
> > > >>>>>> be
> > > >>>>>>>>>>>>>>>>>> volume to be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> > context.
> > > >>> To
> > > >>>>> ensure
> > > >>>>>>>>> that
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>> are in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > > >>>>> definitions,
> > > >>>>>>>>> refers to
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> > correct?
> > > >>>>> Minimally,
> > > >>>>>> we
> > > >>>>>>>>>>>>> must
> > > >>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>> able
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
> > device
> > > >>> to
> > > >>>>> move bits
> > > >>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>> one place
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > > >>> Optionally, a
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> > Therefore,
> > > >>>>> we can have
> > > >>>>>> a
> > > >>>>>>>>>>>>>>>>>> unmanaged
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
> > and
> > > >>> we
> > > >>>>> can have a
> > > >>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> > > >>> capable
> > > >>>>> of
> > > >>>>>>>>> allocating
> > > >>>>>>>>>>>>>>>>>> LUNs and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> > CloudStack
> > > >>> may
> > > >>>>> be able
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> manage a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > > >>>>> unmanaged by
> > > >>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
> > the
> > > >>>>> operator has
> > > >>>>>>>>> chosen
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
> > Does
> > > >>> my
> > > >>>>>> reasoning
> > > >>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>> sense?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
> > are
> > > >>>>> reasonable, it
> > > >>>>>>>>> seems
> > > >>>>>>>>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> > concerns
> > > >>> from
> > > >>>>>>>>> StoragePoolType,
> > > >>>>>>>>>>>>>>>>>> add the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> > attached
> > > >>> driver
> > > >>>>> that
> > > >>>>>>>>>>> indicates
> > > >>>>>>>>>>>>>>>>>> whether
> > > >>>>>>>>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> > > >>> establish
> > > >>>>> a
> > > >>>>>>>>> abstraction
> > > >>>>>>>>>>>>>>>>>>>>>> representing a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
> > that
> > > >>> is
> > > >>>>>>>>> associated
> > > >>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>> volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> > hypervisor
> > > >>>>> drivers can
> > > >>>>>>>>>>>>> declare
> > > >>>>>>>>>>>>>>>>>> which
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> > encounter
> > > >>> a
> > > >>>>> device
> > > >>>>>> managed
> > > >>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> > exposed
> > > >>> by
> > > >>>>> the driver to
> > > >>>>>>>>>>>>> automate
> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> > > >>> sense,
> > > >>>>> then we can
> > > >>>>>>>>> sit
> > > >>>>>>>>>>>>>>>>>> down and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> > between
> > > >>> the
> > > >>>>> current
> > > >>>>>> iSCSI
> > > >>>>>>>>>>>>> type
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
> > go in
> > > >>> and
> > > >>>>> create
> > > >>>>>> a
> > > >>>>>>>>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
> > time,
> > > >>>>> the
> > > >>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>> volume must
> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> > allocated).
> > > >>>>> Future
> > > >>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
> > created
> > > >>>>> behind the
> > > >>>>>>>>>>>>> scenes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
> > to
> > > >>> go in
> > > >>>>> and
> > > >>>>>> create
> > > >>>>>>>>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
> > delete
> > > >>>>> volumes on
> > > >>>>>>>>> its
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
> > the
> > > >>>>> storage
> > > >>>>>>>>>>>>>>>>>> framework). When
> > > >>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
> > that
> > > >>> was
> > > >>>>> created,
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>>>> tells
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> > volume.
> > > >>> After
> > > >>>>> this is
> > > >>>>>>>>> done,
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
> > called.
> > > >>>>> No
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> > volume
> > > >>> was
> > > >>>>> just
> > > >>>>>>>>> created.
> > > >>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
> > PM,
> > > >>> Mike
> > > >>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
> > should
> > > >>>>> probably
> > > >>>>>> have
> > > >>>>>>>>>>>>>>>>>> defined them
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> > > >>> pre-
> > > >>>>> allocated
> > > >>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>> an admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> > > >>> hypervisor
> > > >>>>> data
> > > >>>>>>>>> structure
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
> > SR),
> > > >>> then
> > > >>>>> that
> > > >>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
> > Disks
> > > >>> (VDI)
> > > >>>>> are later
> > > >>>>>>>>>>>>> placed
> > > >>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> > > >>> these
> > > >>>>> cases, the
> > > >>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> > > >>> already
> > > >>>>> in place
> > > >>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> > structure
> > > >>> to
> > > >>>>> the VM in
> > > >>>>>>>>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
> > that is
> > > >>> not
> > > >>>>>>>>>>> pre-allocated.
> > > >>>>>>>>>>>>>>>>>> Instead
> > > >>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
> > could
> > > >>> be a
> > > >>>>> SAN
> > > >>>>>> (not
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> volume on
> > > >>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
> > data
> > > >>>>> structure
> > > >>>>>>>>> must be
> > > >>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> > because
> > > >>> these
> > > >>>>> types of
> > > >>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>> have not
> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > > >>>>> structure by an
> > > >>>>>>>>>>> admin.
> > > >>>>>>>>>>>>>>>>>> Once
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> > XenServer
> > > >>> for
> > > >>>>> this
> > > >>>>>>>>> volume,
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
> > VM
> > > >>> in
> > > >>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
> > PM,
> > > >>>>> John Burwell
> > > >>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> > implementation of
> > > >>> the
> > > >>>>> Dynamic
> > > >>>>>> type
> > > >>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>> behavior
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
> > why I
> > > >>> ask
> > > >>>>> about the
> > > >>>>>>>>>>>>>>>>>> difference.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
> > the
> > > >>>>> definition
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> > > >>> Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
> > type
> > > >>> is
> > > >>>>>> uniquely
> > > >>>>>>>>>>> used
> > > >>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> > > >>> Storage
> > > >>>>> that is
> > > >>>>>>>>> directly
> > > >>>>>>>>>>>>>>>>>> based on
> > > >>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> > > >>> step of
> > > >>>>> going
> > > >>>>>> to
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
> > on
> > > >>> that
> > > >>>>> iSCSI
> > > >>>>>>>>> target as
> > > >>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > > >>>>> supported for
> > > >>>>>>>>>>>>>>>>>> XenServer.
> > > >>>>>>>>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
> > first go
> > > >>> to
> > > >>>>> the
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>> perform
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
> > RBD is.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 2:13
> > > >>> PM,
> > > >>>>> John
> > > >>>>>> Burwell
> > > >>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
> > code,
> > > >>> what
> > > >>>>> is the
> > > >>>>>>>>>>> difference
> > > >>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
> > isn't
> > > >>> RBD
> > > >>>>> considered
> > > >>>>>>>>>>>>> Dynamic?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
> > PM,
> > > >>> Mike
> > > >>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> > storage is
> > > >>>>> defined in
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > > >>>>> StoragePoolType {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
> > local
> > > >>>>> directory
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > NetworkFilesystem(true),
> > > >>> //
> > > >>>>> NFS or CIFS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> > shared
> > > >>> LUN,
> > > >>>>> with a
> > > >>>>>>>>>>> clusterfs
> > > >>>>>>>>>>>>>>>>>>>> overlay
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
> > ZFS
> > > >>>>> Comstar
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
> > image
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> > XenServer
> > > >>> local
> > > >>>>> LVM SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > SharedMountPoint(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> > VMware
> > > >>> VMFS
> > > >>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > > >>>>> XenServer, Storage
> > > >>>>>>>>> Pool
> > > >>>>>>>>>>>>>>>>>> is set
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> > XenServer
> > > >>> local
> > > >>>>> EXT SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> > dynamic,
> > > >>>>> zone-wide
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > StoragePoolType(boolean
> > > >>>>> shared) {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> > isShared() {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 1:41
> > > >>> PM,
> > > >>>>> Mike
> > > >>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> > > >>> another
> > > >>>>> storage
> > > >>>>>>>>>>> company
> > > >>>>>>>>>>>>>>>>>> wants
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> > > >>> Quality
> > > >>>>> of
> > > >>>>>> Service
> > > >>>>>>>>>>>>>>>>>> feature. It
> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
> > well.
> > > >>>>> They would
> > > >>>>>>>>> need
> > > >>>>>>>>>>>>> only
> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
> > the
> > > >>>>> necessary
> > > >>>>>>>>> changes to
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
> > in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > > >>> 1:39
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
> > the
> > > >>>>> XenServer code
> > > >>>>>> is
> > > >>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> > attach
> > > >>> logic is
> > > >>>>> now aware
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > > >>>>> implementation of
> > > >>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> > > >>> with
> > > >>>>> Edison's
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
> > new
> > > >>>>> framework that
> > > >>>>>>>>>>> supported
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> > dynamically.
> > > >>>>> However, when I
> > > >>>>>>>>>>>>>>>>>> visited with
> > > >>>>>>>>>>>>>>>>>>>>>> him
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
> > that
> > > >>> it
> > > >>>>> was not
> > > >>>>>>>>>>>>>>>>>> complete. We
> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
> > could
> > > >>> do
> > > >>>>> with these
> > > >>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>> unless
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> > recognize
> > > >>> this
> > > >>>>> new type of
> > > >>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> > structure.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > > >>> 1:28
> > > >>>>> PM, John
> > > >>>>>>>>> Burwell
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
> > to me
> > > >>>>> that any
> > > >>>>>>>>>>> operation
> > > >>>>>>>>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> > > >>> about
> > > >>>>> details. I
> > > >>>>>>>>> expect
> > > >>>>>>>>>>>>>>>>>> to see
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> > > >>> operations
> > > >>>>> that can be
> > > >>>>>>>>>>>>>>>>>>>> composed/driven
> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > > >>>>> allocate
> > > >>>>>> space/create
> > > >>>>>>>>>>>>>>>>>> structures
> > > >>>>>>>>>>>>>>>>>>>>>> per
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > > >>>>> dependency, we are
> > > >>>>>>>>> going
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> end
> > > >>>>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
> > make
> > > >>> the
> > > >>>>> system
> > > >>>>>>>>>>>>> increasingly
> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > > >>>>> understanding that the
> > > >>>>>>>>> Xen
> > > >>>>>>>>>>>>>>>>>> specific
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > > >>>>> CitrixResourceBase
> > > >>>>>> class?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > > >>> 3:12
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
> > in a
> > > >>> bit
> > > >>>>> more
> > > >>>>>>>>> detail:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
> > aside
> > > >>> from
> > > >>>>> one setup
> > > >>>>>>>>>>>>> method
> > > >>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> > > >>> volume on
> > > >>>>> the
> > > >>>>>> storage
> > > >>>>>>>>>>>>>>>>>> system,
> > > >>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> > > >>> structure
> > > >>>>> to make
> > > >>>>>> use
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > > >>> XenServer
> > > >>>>> or a
> > > >>>>>> datastore
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> ESX(i)).
> > > >>>>>>>>>>>>>>>>>>>>>> VMs
> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> > storage
> > > >>>>> system's volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> > > >>> storage
> > > >>>>> framework,
> > > >>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> > > >>> create
> > > >>>>> a 1:1
> > > >>>>>>>>>>>>> relationship
> > > >>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
> > VM's
> > > >>> data
> > > >>>>> disk
> > > >>>>>>>>> (necessary
> > > >>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
> > a
> > > >>> plug-
> > > >>>>> in that is
> > > >>>>>>>>> called
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> > needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
> > the
> > > >>>>> storage
> > > >>>>>> framework
> > > >>>>>>>>> did
> > > >>>>>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>>>> address
> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > > >>> hypervisor-
> > > >>>>> specific data
> > > >>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>> when
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
> > case,
> > > >>> I've
> > > >>>>> been
> > > >>>>>>>>> enhancing
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>>>>>>>> so.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > > >>>>> submitted. I've got
> > > >>>>>>>>> ESX(i)
> > > >>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
> > 4.2
> > > >>>>> freeze date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
> > bit? :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > > >>>>> 1:03 PM, Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> > plug-in -
> > > >>> by
> > > >>>>> itself - is
> > > >>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
> > the
> > > >>>>> volume-attach
> > > >>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>> (in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
> > into
> > > >>> the
> > > >>>>> plug-in to
> > > >>>>>>>>> have
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> > > >>> comes to
> > > >>>>> attach the
> > > >>>>>>>>> volume
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> > > >>> enough
> > > >>>>> to recognize
> > > >>>>>>>>> it's
> > > >>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> > volume
> > > >>> has
> > > >>>>> just been
> > > >>>>>>>>> created)
> > > >>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> create,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > > >>>>> XenServer) or a
> > > >>>>>>>>> datastore
> > > >>>>>>>>>>>>>>>>>> (for
> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
> > just
> > > >>>>> created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> > spending
> > > >>> most
> > > >>>>> of my time
> > > >>>>>>>>>>>>> recently
> > > >>>>>>>>>>>>>>>>>> making
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> > code.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
> > up?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > > >>> at
> > > >>>>> 12:48 PM,
> > > >>>>>> John
> > > >>>>>>>>>>>>>>>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> > > >>> why
> > > >>>>> the the storage
> > > >>>>>>>>>>>>> driver
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> > at
> > > >>> 1:21
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> > ultimately I
> > > >>>>> would like to
> > > >>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
> > think
> > > >>> I'm
> > > >>>>> just out of
> > > >>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>> for 4.2
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> > > >>> plug-in
> > > >>>>> supports
> > > >>>>>>>>>>>>> XenServer.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> > > >>> feature
> > > >>>>> freeze, I
> > > >>>>>>>>> have
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> working
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
> > be
> > > >>> clear,
> > > >>>>> this is
> > > >>>>>> all
> > > >>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> regards
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > Compute
> > > >>>>> Offerings post
> > > >>>>>> 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > > >>> 2013
> > > >>>>> at 11:14 AM,
> > > >>>>>>>>> Kelcey
> > > >>>>>>>>>>>>>>>>>> Jamison
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
> > plan
> > > >>> on
> > > >>>>> supporting
> > > >>>>>>>>> KVM in
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > > >>>>> Message -----
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > > >>>>> Tutkowski" <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > > >>>>> dev@cloudstack.apache.org
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> > Monday,
> > > >>> June
> > > >>>>> 3, 2013
> > > >>>>>>>>> 10:12:32 AM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> > > >>> [MERGE]
> > > >>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> MASTER
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> > > >>> merging
> > > >>>>> Wei's feature
> > > >>>>>>>>>>> first,
> > > >>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>> mine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
> > is
> > > >>> for
> > > >>>>> KVM only,
> > > >>>>>>>>> then
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> is a
> > > >>>>>>>>>>>>>>>>>>>> non
> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> > 3,
> > > >>> 2013
> > > >>>>> at 8:55 AM,
> > > >>>>>> Wei
> > > >>>>>>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> > billing,
> > > >>> as
> > > >>>>> no one works
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> billing
> > > >>>>>>>>>>>>>>>>>>>> now,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
> > by
> > > >>>>> themselves. They
> > > >>>>>>>>> can
> > > >>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_offering of
> > > >>> a
> > > >>>>> VMs and
> > > >>>>>> volumes
> > > >>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
> > the
> > > >>>>> exact
> > > >>>>>> limitation
> > > >>>>>>>>>>>>> value
> > > >>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > limitation
> > > >>> for
> > > >>>>> nics as
> > > >>>>>> well.
> > > >>>>>>>>> I
> > > >>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
> > I/O
> > > >>>>> throttling?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > > >>>>> misunstand the second
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> > > >>> what
> > > >>>>> you said about
> > > >>>>>> the
> > > >>>>>>>>>>> two
> > > >>>>>>>>>>>>>>>>>>>> features.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> > 2013,
> > > >>> at
> > > >>>>> 2:13 AM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> > Mike's
> > > >>>>> aswer helps you.
> > > >>>>>>>>> I am
> > > >>>>>>>>>>>>>>>>>> trying
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> > think
> > > >>> billing
> > > >>>>> should
> > > >>>>>>>>> depend
> > > >>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> IO
> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > limitation.
> > > >>> Please
> > > >>>>> review
> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
> > > >>>>>>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
> > IO
> > > >>>>> statistics
> > > >>>>>> including
> > > >>>>>>>>>>>>>>>>>> bytes/iops
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> > > >>> machine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
> > the
> > > >>> AWS
> > > >>>>> model,
> > > >>>>>> customers
> > > >>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>> billed
> > > >>>>>>>>>>>>>>>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > provisioned
> > > >>> IOPS,
> > > >>>>> as well as,
> > > >>>>>> for
> > > >>>>>>>>>>>>> those
> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://aws.amazon.com/ebs/).
> > > >>>>>> I
> > > >>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>> imagine
> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> > > >>> employ
> > > >>>>> similar cost
> > > >>>>>>>>>>> models.
> > > >>>>>>>>>>>>>>>>>> Could
> > > >>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
> > model
> > > >>> in
> > > >>>>> the current
> > > >>>>>>>>> patch?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
> > you
> > > >>> mean
> > > >>>>> IOPS runtime
> > > >>>>>>>>>>> change?
> > > >>>>>>>>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > limitation for
> > > >>> a
> > > >>>>> running
> > > >>>>>> virtual
> > > >>>>>>>>>>>>>>>>>> machine
> > > >>>>>>>>>>>>>>>>>>>>>> through
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > CloudStack
> > > >>> does
> > > >>>>> not support
> > > >>>>>>>>>>> changing
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (computer
> > > >>>>> offering or disk
> > > >>>>>>>>>>>>> offering).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
> > the
> > > >>>>> Java interface
> > > >>>>>>>>> level.
> > > >>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>> apologize
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > > >>>>> generalize allocation
> > > >>>>>>>>>>>>>>>>>> algorithms
> > > >>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
> > the
> > > >>>>> service guarantees
> > > >>>>>>>>>>>>>>>>>> provided by a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> > > >>> good
> > > >>>>> question.
> > > >>>>>>>>> Maybe it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> disk_io_throttling as Mike
> > > >>>>>>>>> needs to
> > > >>>>>>>>>>>>>>>>>> consider
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
> > think.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> > expand
> > > >>> on
> > > >>>>> my thoughts
> > > >>>>>> in a
> > > >>>>>>>>>>>>> later
> > > >>>>>>>>>>>>>>>>>>>>>> response
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
> > points
> > > >>>>> between these two
> > > >>>>>>>>>>>>>>>>>> features. I
> > > >>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
> > to be
> > > >>>>> merged before
> > > >>>>>>>>>>>>>>>>>> SolidFire, but
> > > >>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > coordination
> > > >>>>> between the
> > > >>>>>> branches
> > > >>>>>>>>>>>>>>>>>> (possibly
> > > >>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> disk_io_throttling)
> > > >>>>> to
> > > >>>>>>>>> coordinate on
> > > >>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>> issue.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > > >>> John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > things I
> > > >>>>> want to
> > > >>>>>> understand
> > > >>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
> > there
> > > >>>>> value in
> > > >>>>>> capturing
> > > >>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>> policies
> > > >>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > model
> > > >>> (e.g.
> > > >>>>> for
> > > >>>>>>>>> billing/usage
> > > >>>>>>>>>>>>>>>>>>>> purposes,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> > Should
> > > >>> there
> > > >>>>> be a common
> > > >>>>>>>>>>>>>>>>>> interface model
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > provisioning
> > > >>> at
> > > >>>>> runtime?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
> > are
> > > >>>>> conflicting
> > > >>>>>>>>> provisioned
> > > >>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > hypervisor
> > > >>> and
> > > >>>>> storage device
> > > >>>>>>>>>>>>>>>>>> reconciled?
> > > >>>>>>>>>>>>>>>>>>>> In
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
> > to
> > > >>>>> believe (and
> > > >>>>>> billed)
> > > >>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>> device
> > > >>>>> has been
> > > >>>>>>>>> configured
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > consistent
> > > >>>>> configuration
> > > >>>>>>>>> between a
> > > >>>>>>>>>>>>>>>>>> VM and a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
> > but a
> > > >>>>> later
> > > >>>>>> modification
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> inconsistency.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
> > 2,
> > > >>> 2013,
> > > >>>>> at 8:38 PM,
> > > >>>>>>>>> Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > believe
> > > >>> Wei's
> > > >>>>> feature deals
> > > >>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>> controlling
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>> hypervisor
> > > >>>>> side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > feature
> > > >>> is
> > > >>>>> focused on
> > > >>>>>>>>>>>>> controlling
> > > >>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> > that
> > > >>>>> helps. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
> > Jun
> > > >>> 2,
> > > >>>>> 2013 at 6:35
> > > >>>>>> PM,
> > > >>>>>>>>>>>>> John
> > > >>>>>>>>>>>>>>>>>>>> Burwell
> > > >>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > > >>> opinion is
> > > >>>>> that no
> > > >>>>>> features
> > > >>>>>>>>>>>>>>>>>> should be
> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> > > >>> have
> > > >>>>> been resolved
> > > >>>>>> and
> > > >>>>>>>>> it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>> ready
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > Ops
> > > >>> vs
> > > >>>>> discrete
> > > >>>>>>>>> read/write
> > > >>>>>>>>>>>>>>>>>> ops issue
> > > >>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> > reviewed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > I
> > > >>> don't
> > > >>>>> think this
> > > >>>>>>>>> criteria
> > > >>>>>>>>>>>>>>>>>> has been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> > how
> > > >>>>> does this work
> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> > > >>>>>>>>>>>>>>>>>> )?
> > > >>>>>>>>>>>>>>>>>>>> As I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> > > >>> involves
> > > >>>>> provisioned
> > > >>>>>>>>> IOPS. I
> > > >>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > scenario
> > > >>>>> where provisioned
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> KVM and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > unnecessarily
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> incompatible.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Jun 1,
> > > >>> 2013,
> > > >>>>> at 6:47 AM,
> > > >>>>>>>>> Wei
> > > >>>>>>>>>>>>>>>>>> ZHOU <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
> > will
> > > >>>>> change it next
> > > >>>>>>>>> week.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/6/1
> > > >>>>> Wido den Hollander
> > > >>>>>> <
> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 06/01/2013 08:24 AM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Exactly. I
> > > >>> have
> > > >>>>> pushed the
> > > >>>>>>>>>>>>> features
> > > >>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > anyone
> > > >>>>> object thems for
> > > >>>>>>>>>>>>> technical
> > > >>>>>>>>>>>>>>>>>>>> reason
> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> > the
> > > >>> sake
> > > >>>>> of clarity I
> > > >>>>>> just
> > > >>>>>>>>>>>>> want
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> mention
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > total
> > > >>> IOps
> > > >>>>> to R/W IOps
> > > >>>>>>>>> asap
> > > >>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > total
> > > >>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
> > laid
> > > >>> the
> > > >>>>> groundwork for
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > however
> > > >>>>> prevent that we
> > > >>>>>> create
> > > >>>>>>>>>>>>>>>>>> legacy from
> > > >>>>>>>>>>>>>>>>>>>>>> day
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/5/31
> > > >>>>> Wido den
> > > >>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/31/2013 03:59 PM, John
> > > >>>>>>>>>>>>> Burwell
> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
> > this
> > > >>>>> enhancement must
> > > >>>>>> to
> > > >>>>>>>>>>>>>>>>>> discretely
> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > see
> > > >>> how
> > > >>>>> it could be
> > > >>>>>>>>> fixed
> > > >>>>>>>>>>>>>>>>>> later
> > > >>>>>>>>>>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > correctly
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
> > total
> > > >>> IOPS
> > > >>>>> into read
> > > >>>>>> and
> > > >>>>>>>>>>>>>>>>>> write.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>> unless/until we
> > > >>>>>> decided
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> break
> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
> > Wei
> > > >>>>> meant was merging
> > > >>>>>> it
> > > >>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>>>>>>>> now
> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > branch
> > > >>>>> and add Read /
> > > >>>>>>>>> Write
> > > >>>>>>>>>>>>> IOps
> > > >>>>>>>>>>>>>>>>>>>> before
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > > >>>>> released with Read
> > > >>>>>> and
> > > >>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>> instead
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
> > to
> > > >>>>> make the May 31st
> > > >>>>>>>>>>>>> feature
> > > >>>>>>>>>>>>>>>>>>>> freeze
> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
> > other
> > > >>>>> threads) then it
> > > >>>>>>>>> won't
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>> necessary
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > > >>>>> completely agree that
> > > >>>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>>> is no
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > I/O.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > May 31,
> > > >>>>> 2013, at 9:51 AM,
> > > >>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>> den
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/31/2013 03:13 PM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> > Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks.
> > > >>> Good
> > > >>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > thought
> > > >>>>> about at the
> > > >>>>>>>>>>> beginning.
> > > >>>>>>>>>>>>>>>>>>>> Finally I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > difference
> > > >>> of
> > > >>>>> read and write
> > > >>>>>>>>>>>>> mainly
> > > >>>>>>>>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
> > the
> > > >>>>> difference of sent
> > > >>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> received
> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > > >>>>> reasoning seems odd.
> > > >>>>>>>>>>>>>>>>>> Networking and
> > > >>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > I/O is
> > > >>>>> much more
> > > >>>>>>>>> expensive
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> most
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > bandwith.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> Implementing
> > > >>>>> it will be some
> > > >>>>>>>>>>>>>>>>>> copy-paste
> > > >>>>>>>>>>>>>>>>>>>>>> work.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> implemented
> > > >>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
> > days.
> > > >>> For
> > > >>>>> the deadline
> > > >>>>>> of
> > > >>>>>>>>>>>>>>>>>> feature
> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
> > if
> > > >>>>> needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> > think it's
> > > >>> a
> > > >>>>> feature we
> > > >>>>>>>>> can't
> > > >>>>>>>>>>>>>>>>>> miss. But
> > > >>>>>>>>>>>>>>>>>>>>>> if
> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > window
> > > >>> we
> > > >>>>> have to make sure
> > > >>>>>> we
> > > >>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
> > 4.3,
> > > >>> that
> > > >>>>> will confuse
> > > >>>>>>>>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/5/31
> > > >>>>> Wido den
> > > >>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/30/2013 06:03 PM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > would
> > > >>> like to
> > > >>>>> merge
> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > nobody
> > > >>>>> object, I will
> > > >>>>>> merge
> > > >>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>> purpose
> > > >>>>> is :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > > >>>>> machines are running
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> same
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > storage or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> > > >>> strage).
> > > >>>>> Because of
> > > >>>>>> the
> > > >>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>>>> limitation
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
> > if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
> > VM
> > > >>> has
> > > >>>>> large disk
> > > >>>>>>>>> operation,
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>> affect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > VMs
> > > >>>>> running on the
> > > >>>>>> same
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> device.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > >>> neccesary
> > > >>>>> to set the
> > > >>>>>>>>>>> maximum
> > > >>>>>>>>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Looking at
> > > >>> the
> > > >>>>> code I see
> > > >>>>>> you
> > > >>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
> > and
> > > >>>>> libvirt support
> > > >>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>> both a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > >>> which
> > > >>>>> could benefit a
> > > >>>>>>>>> lot of
> > > >>>>>>>>>>>>>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
> > also
> > > >>>>> strange, in the
> > > >>>>>>>>> polling
> > > >>>>>>>>>>>>>>>>>> side you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > but
> > > >>> on
> > > >>>>> the throttling
> > > >>>>>>>>> side
> > > >>>>>>>>>>>>>>>>>> you only
> > > >>>>>>>>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > IOps
> > > >>> are
> > > >>>>> usually much
> > > >>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>> expensive
> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
> > valid
> > > >>>>> use-case where
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > vs
> > > >>> Read
> > > >>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
> > this
> > > >>> only
> > > >>>>> supports KVM
> > > >>>>>>>>> at
> > > >>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>> point I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > to at
> > > >>>>> least have the
> > > >>>>>>>>>>>>>>>>>> mechanism in
> > > >>>>>>>>>>>>>>>>>>>>>> place
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> implementing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > later
> > > >>>>> would be a lot of
> > > >>>>>>>>>>> work.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> > > >>> hypervisor
> > > >>>>> doesn't
> > > >>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > you
> > > >>> can
> > > >>>>> always sum
> > > >>>>>> both
> > > >>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>> and set
> > > >>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > you
> > > >>>>> explain why you
> > > >>>>>>>>>>>>> implemented
> > > >>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>> feature
> > > >>>>> includes:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> > the
> > > >>>>> maximum rate of
> > > >>>>>>>>> VMs
> > > >>>>>>>>>>>>> (in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> configuration)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > change
> > > >>> the
> > > >>>>> maximum rate
> > > >>>>>> of
> > > >>>>>>>>>>> VMs
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > limit
> > > >>> the
> > > >>>>> disk rate
> > > >>>>>> (total
> > > >>>>>>>>>>> bps
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> iops)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
> > ticket:
> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> jira/browse/CLOUDSTACK-1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>> 1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 1192
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
> > will
> > > >>>>> update later) :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>> VM+Disk+IO+Throttling<https://
> <https://%0b>> > >>>>>>>>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>
> > > >>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > >>>>> rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Merge
> > > >>> check
> > > >>>>> list :-
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
> > you
> > > >>>>> check the branch's
> > > >>>>>>>>> RAT
> > > >>>>>>>>>>>>>>>>>> execution
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> > > >>> there
> > > >>>>> new dependencies
> > > >>>>>>>>>>>>>>>>>> introduced?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > What
> > > >>>>> automated testing
> > > >>>>>> (unit
> > > >>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > feature?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
> > tests
> > > >>> are
> > > >>>>> added.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > What
> > > >>>>> testing has been done
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> check for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> > the
> > > >>>>> bytes rate and
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>> rate on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > VM
> > > >>>>> operations, including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > deploy,
> > > >>> stop,
> > > >>>>> start, reboot,
> > > >>>>>>>>>>>>>>>>>> destroy,
> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > > >>> Volume
> > > >>>>> operations,
> > > >>>>>>>>> including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > > >>>>> Detach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> > review
> > > >>> the
> > > >>>>> code, you can
> > > >>>>>>>>> try
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> > > >>> regards,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>> VM+Disk+IO+Throttling<https://
> <https://%0b>> > >>>>>>>>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>
> > > >>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > >>>>> rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > > >>>>>>>>> refs/heads/disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>>>> 1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 1301
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/****jira/**browse/CLOUDSTACK-
> > > >>> 2071
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 2071
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <h**ttps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> (**CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > VM
> > > >>> Disk
> > > >>>>> I/O
> > > >>>>>> Throttling)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > >>>>> Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>>>> CloudStack Developer,
> > > >>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > > >>> 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Advancing
> > > >>> the
> > > >>>>> way the world
> > > >>>>>>>>> uses
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > >>> Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>> CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > > >>> way
> > > >>>>> the world uses
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>> CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > > >>> way
> > > >>>>> the world uses
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > >>> the
> > > >>>>> world uses the
> > > >>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > >>>>> Developer,
> > > >>>>>> SolidFire
> > > >>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > >>> the
> > > >>>>> world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>>>> Developer, SolidFire
> > > >>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > >>>>> world uses the
> > > >>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>> Developer,
> > > >>>>> SolidFire
> > > >>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > >>> world
> > > >>>>> uses the
> > > >>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>> Developer,
> > > >>>>> SolidFire
> > > >>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > > >>> world
> > > >>>>> uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>>>> SolidFire
> > > >>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > >>> uses
> > > >>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>>>> SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > >>> uses
> > > >>>>> the cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>>>> SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > >>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > >>> the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>> *(tm)*
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> *Mike Tutkowski*
> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>> o: 303.746.7302
> > > >>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>> *(tm)*
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> *Mike Tutkowski*
> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>> o: 303.746.7302
> > > >>>>>>> Advancing the way the world uses the
> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>> *(tm)*
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> *Mike Tutkowski*
> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>> e: mike.tutkowski@solidfire.com
> > > >>>>> o: 303.746.7302
> > > >>>>> Advancing the way the world uses the
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>> *(tm)*
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> *Mike Tutkowski*
> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> e: mike.tutkowski@solidfire.com
> > > >>> o: 303.746.7302
> > > >>> Advancing the way the world uses the
> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>> *(tm)*
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkowski@solidfire.com
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > *(tm)*
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*****
>
>
>
> ****
>
> ** **
>
> --
> *Mike Tutkowski*****
>
> *Senior CloudStack Developer, SolidFire Inc.*****
>
> e: mike.tutkowski@solidfire.com****
>
> o: 303.746.7302****
>
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*****
>
>
>
> ****
>
> ** **
>
> --
> *Mike Tutkowski*****
>
> *Senior CloudStack Developer, SolidFire Inc.*****
>
> e: mike.tutkowski@solidfire.com****
>
> o: 303.746.7302****
>
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*****
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
Ok, I am Ok with the way you are talking about in 4.2.
From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
Sent: Sunday, June 09, 2013 8:59 PM
To: dev@cloudstack.apache.org
Cc: Edison Su; John Burwell
Subject: Re: [MERGE] disk_io_throttling to MASTER
Another thing we might want to consider for 4.2 is that the way the code is currently implemented in master, I don't think it's even possible to write a plug-in that connects storage directly to a VM because the CloudStack storage framework will call the "attach" method of the hypervisor in use and this method will fail because it assumes there is (talking Xen here) an SR existent and in this situation there isn't one.
My thinking is that we should ignore the direct-attach-of-volume-to-VM use case for 4.2. I think it will require too much of a rewrite at this point in development (fewer than three weeks left until code freeze).
If the plug-in implements an isManaged() method (as John suggested), the storage framework can invoke the necessary commands on the hypervisor in use.
For example, the createAsync method is called. The storage plug-in creates a SAN volume and updates the CS DB. The storage framework then asks the driver if it's managed. If it is, the storage framework executes a command against the hypervisor in use to have it create, say, an SR (with a single VDI that takes up the entire SR). Now, when the storage framework executes the "attach" command against the hypervisor, this command will work, as well (because that logic assumes the prior existence of an SR and it will, in fact, be present).
On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <mi...@solidfire.com>> wrote:
Hi Edison and John,
I wanted to point out something I thought was relevant to this conversation.
My plug-in supports both XenServer and ESX(i).
These two hypervisors handle their respective hypervisor data structures (XenServer SR or ESX(i) datastore) quite differently.
In XenServer, the SR that resides on the SAN LUN is simply "forgotten" when you issue a remove-SR command to XenServer.
When you later want to access the contents of that data, it is still safely stored on the LUN and can be accessed from XenServer (from the same or different cluster (what XenServer calls a resource pool)) by creating a new SR that is based on the same IQN and LUN.
XenServer SRs exist at the cluster level.
In ESX(i), the datastore that resides on the SAN LUN is actually destroyed when you issue a remove-datastore command to ESX(i). In other words, the contents of the SAN LUN are destroyed.
ESX(i) datastores exist at the datacenter level.
The XenServer behavior (not destroying the contents of the LUN when the SR is removed) is desirable; the ESX(i) behavior (destroying the contents of the LUN when the datastore is removed) is not desirable.
I can get around the ESX(i) behavior by removing the referenced IQN from each host in the cluster when the CS storage framework notifies my plug-in of a detach-volume event. When the CS volume is later attached again (either to the same or a different ESX(i) cluster), I can have each host in the relevant ESX(i) cluster reference the IQN and the datastore will be back again.
This sequence of events is critical for my plug-in to support both environments:
XenServer flow (events for Driver class):
createAsync (called right before first attach) = creates SAN volume
preAttachVolume = creates SR and creates VDI or just introduces SR
(CS volume gets attached and later detached)
postDetachVolume = removes SR (this does not delete data on SAN volume)
deleteAsync (can be called when volume is not attached) = deletes the SAN volume
VMware flow (events for Driver class):
createAsync (called right before first attach) = creates SAN volume, creates datastore, and creates VMDK file
preAttachVolume = adds iSCSI target to each host in cluster
(CS volume gets attached and later detached)
postDetachVolume = removes iSCSI target from each host in cluster
deleteAsync (can be called when volume is not attached) = deletes datastore and deletes the SAN volume
********************
So, the way I have it written up here, the driver is talking to the hypervisor. As John has pointed out, this may not be ideal.
Whatever we decide on, it needs to be able to support this kind of flow. :)
We could go John's route, where the driver says if it's managed or not.
If it is, then the storage framework could send the necessary messages to the hypervisor in question at the right times (just migrate the hypervisor logic above from the driver to the right parts of the storage framework).
This is fine, except that it does not resolve Edison's use case of desiring to have the storage LUN potentially connected directly to a VM.
I'm not sure we should design around that use case, though. Is it common enough? Has anyone asked for it? Is it even desirable? How are usage statistics gathered for such volumes? It's nice to offer the flexibility, but is it a good idea in this case?
Thanks!
On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com>> wrote:
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> Sent: Friday, June 07, 2013 4:08 PM
> To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Hi John,
>
> Yes, you are correct that Xen must layer an SR on top of the iSCSI volume
> to use it. Same story for VMware: ESX(i) must layer a datastore on top of
> the iSCSI volume to use it. I look at it like they are layering a clustered
> file system on the SAN volume so the hypervisors can share access to the
> contents of the volume.
>
> In 4.1, by the time the hypervisor-attach-volume logic is called, the SR or
> datastore has already been created (usually manually by an admin).
>
> This pre-setup of the, say, SR is, of course, not acceptable in an
> environment where each CloudStack volume that a user creates is mapped
> to a
> single SAN volume (via the SR).
>
> The question comes down to who should allocate the SR dynamically.
>
> We could have the storage framework ask the storage plug-in if it is
> managed. If it is, then the storage framework could send a message to the
> hypervisor in question to create (let's talk Xen here) the SR ahead of
> time. Then, when the storage framework next sends the attach-volume
> command
> to the hypervisor, it should work without changes to that attach-volume
> logic (because - from the point of view of the attach logic - the SR is
> already existent, as expected).
>
> Now, as Edison has pointed out, this limits the power of the storage
> plug-in. A storage plug-in in this model cannot directly attach storage to
> a VM (it must go through the hypervisor). Perhaps that is OK. We need to
> make a call on that.
Yes, I think we need to address this usage case, and won't limit the power of the storage plug-in.
John, what's your idea to fit this usage case into your abstraction?
>
> If we want to give more power to the storage vendor, we would have to
> have
> the storage framework call into the storage plug-in to send the appropriate
> attach commands to the hypervisor.
>
> Let's discuss and come to a consensus (or at least agree on a path) as soon
> as we can. :)
>
> Thanks!
>
>
>
> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>> wrote:
>
> > Mike,
> >
> > My understanding is the Xen expects an SR structure on all ISCSI devices
> > -- at least that is how I read the code in your patch. Is my understanding
> > correct? If so, the Xen plugin should be able to query the storage device
> > to determine the presence of the SR structure and create it if it does not
> > exist. Am I missing something in the implementation that makes that type
> > of implementation impossible?
> >
> > Thanks,
> > -John
> >
> > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <mi...@solidfire.com>>
> > wrote:
> >
> > > Yeah, if a storage vendor wanted to attach a volume directly to a VM in
> > 4.2
> > > today, it would probably fail because the attach-volume logic assumes
> the
> > > existence of the necessary hypervisor data structure (ex. SR on Xen).
> > >
> > > If we wanted to enable such an attach, we could do it the way Edison
> > > suggests.
> > >
> > >
> > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>> wrote:
> > >
> > >>
> > >>
> > >>> -----Original Message-----
> > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> > >>> Sent: Friday, June 07, 2013 2:37 PM
> > >>> To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >>>
> > >>> As we only have three weeks until feature freeze, we should come to
> a
> > >>> consensus on this design point as soon as possible.
> > >>>
> > >>> Right now, if the storage framework asks my driver it is is managed, it
> > >>> will say 'yes.' This means the framework will tell the driver to
> > perform
> > >>> its management activities. This then means the driver will call into
> > the
> > >>> host (it doesn't know which hypervisor, by the way) to perform the
> > >> activity
> > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> > >>>
> > >>> The driver doesn't know which hypervisor it's talking to, it just
> > sends a
> > >>> message to the host to perform the necessary pre-attach work.
> > >>
> > >> Could we just expose a method like "attachVolume/dettachVolume" on
> the
> > >> PrimaryDataStoreDriver driver?
> > >> For most of cases, the implementation of each driver would just send
> > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
> the
> > >> implementation in a base class, so that can be shared by all of this
> > >> drivers), and in each hypervisor resource code, which may just call
> > >> hypervisor's api to attach the volume to VM, while for certain storage,
> > >> like SolidFire, may need to create a SR first, and create volume on it,
> > >> then call hypervisor's API to attach volume to VM.
> > >> While for some other storage vendor, may want to bypass hypervisor
> > during
> > >> attaching volume, so inside driver's attachvolume implementation, the
> > >> driver can do some magic, such as, directly talk to an agent inside VM
> > >> instance, then create a disk inside VM.
> > >>
> > >> How do you guys think?
> > >>
> > >>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>>
> > wrote:
> > >>>
> > >>>>
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com<ma...@solidfire.com>]
> > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> > >>>>> To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >>>>>
> > >>>>> Hi John,
> > >>>>>
> > >>>>> How's about this:
> > >>>>>
> > >>>>> The driver can implement an isManaged() method. The
> > >>> VolumeManagerImpl
> > >>>>> can
> > >>>>> call into the driver to see if its managed. If it is, the
> > >>>> VolumeManagerImpl
> > >>>>> (which is responsible for calling into the hypervisor to attach the
> > >> disk)
> > >>>>> can call into the hypervisor to create the necessary hypervisor data
> > >>>>> structure (ex. for XenServer, a storage repository).
> > >>>>
> > >>>> The problem here is that storage vendor may work differently with
> > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> > >> other
> > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> directly
> > >> to
> > >>>> VM instance, see the discuss(
> > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> > >>>
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com<ma...@mail.gmail.com>
> > >>> %3E
> > >>>> ).
> > >>>> So I would let storage provider to implement attach disk to VM,
> > >> instead of
> > >>>> implemented by cloudstack itself.
> > >>>>
> > >>>>
> > >>>>>
> > >>>>> If that's what you're going for, that works for me. By the way,
> > >> Edison's
> > >>>>> default storage plug-in (which handles the default storage behavior
> > >> in
> > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
> > >>>>> hypervisors. You might want to contact him and inform him of your
> > >>>> concerns
> > >>>>> or that logic (as is) will make it to production.
> > >>>>>
> > >>>>> Please let me know if what I wrote in above (for my solution) is OK
> > >> with
> > >>>>> you. :)
> > >>>>>
> > >>>>> Thanks!
> > >>>>>
> > >>>>>
> > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>>
> > >>> wrote:
> > >>>>>
> > >>>>>> Mike,
> > >>>>>>
> > >>>>>> Please see my responses in-line below.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> -John
> > >>>>>>
> > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > >>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hey John,
> > >>>>>>>
> > >>>>>>> I still have a bit more testing I'd like to do before I build up
> > >> a
> > >>>> patch
> > >>>>>>> file, but this is the gist of what I've done:
> > >>>>>>>
> > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
> tells
> > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
> > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> > >>>>> managed.
> > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> > >> whatever
> > >>>>>>> activity is required. In my case, this includes sending a
> > >> message to
> > >>>> the
> > >>>>>>> host where the VM is running to have, say XenServer, add a
> > >> storage
> > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
> > >> SAN
> > >>>>>> volume,
> > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can
> > >> on
> > >>>> the
> > >>>>>>> storage repository). After this, the normal attach-volume
> > >> message is
> > >>>> sent
> > >>>>>>> to the host by VolumeManagerImpl.
> > >>>>>>
> > >>>>>> There should be **no** code from a storage driver to a hypervisor.
> > >> I
> > >>>>>> apologize for the repetition, but we simply can not have hypervisor
> > >>>>>> specific code in the storage layer. The circular dependencies
> > >> between
> > >>>> the
> > >>>>>> two layers are not sustainable in the long term. Either the
> > >>>> VirtualManager
> > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> > >>> coordinate
> > >>>>>> volume creation and then populating the SR. Ideally, we can
> > >>>> generalize the
> > >>>>>> process flow for attaching volumes such that the Xen hypervisor
> > >> plugin
> > >>>>>> would only implement callbacks to perform the attach action and
> > >> create
> > >>>>> the
> > >>>>>> structure and SR. To my mind, the SolidFire driver should only be
> > >>>>>> allocating space and providing information about contents (e.g.
> > >> space
> > >>>>>> available, space consumed, streams to a URI, file handle for a URI,
> > >>>> etc)
> > >>>>>> and capabilities.
> > >>>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>> * The reverse is performed for a detach-volume command.
> > >>>>>>>
> > >>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
> > >>>>>> Edison's
> > >>>>>>> default driver simply does a "return false;". We could add a new
> > >>>>>> parameter
> > >>>>>>> to the createStoragePool API command, if we want, to remove
> the
> > >>>>>> hard-coded
> > >>>>>>> return values in the drivers (although my driver will probably
> > >> just
> > >>>>>> ignore
> > >>>>>>> this parameter and always return true since it wouldn't make
> > >> sense
> > >>>> for it
> > >>>>>>> to ever return false). We'd need another column in the
> > >> storage_pool
> > >>>>> table
> > >>>>>>> to store this value.
> > >>>>>>
> > >>>>>> Yes, I think we should have a parameter added to the
> > >>> createStoragePool
> > >>>>>> surfaced to the HTTP API that allows DataStores to be configured
> > >> for
> > >>>>>> management when their underlying drivers support it. To simplify
> > >>>> things,
> > >>>>>> this flag should only be mutable when the DataStore is created. It
> > >>>> would be
> > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
> > >>>> creation.
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Sound like I'm in sync with what you were thinking?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > >>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> I agree, John. Just wanted to point out that I have a working
> > >> GUI
> > >>>> for
> > >>>>>> you
> > >>>>>>>> to review (in that document), if you'd like to check it out.
> > >>>>>>>>
> > >>>>>>>> Thanks!
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> > >> jburwell@basho.com<ma...@basho.com>>
> > >>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Mike,
> > >>>>>>>>>
> > >>>>>>>>> I would like the UIs of two features reviewed together to
> > >> ensure
> > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
> > >> and
> > >>>>>>>>> storage device provisioned IOPs. I see the potential for
> > >>>> confusion,
> > >>>>>>>>> and I think a side-by-side Ui review of these features will
> > >> help
> > >>>>>>>>> minimize any potential confusion.
> > >>>>>>>>>
> > >>>>>>>>> As I mentioned, the term reconciliation issue will work itself
> > >> if
> > >>>> it
> > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> > >>>> throttled
> > >>>>>>>>> IOPs or storage provisioned IOPs.
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> -John
> > >>>>>>>>>
> > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > >>>>>>>>> <mi...@solidfire.com>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi John,
> > >>>>>>>>>>
> > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
> to
> > >>> you
> > >>>>> the
> > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
> > >> IOPS.
> > >>>>>>>>>>
> > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> > >> and we
> > >>>>>>>>> decided
> > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> Offering
> > >>>> dialog.
> > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> should
> > >>> be
> > >>>>> able
> > >>>>>> to
> > >>>>>>>>>> leverage this GUI going forward (even if they, say, only use
> > >> Max
> > >>>>>> IOPS).
> > >>>>>>>>>> These fields are optional and can be ignored for storage that
> > >>>> does not
> > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> > >>>> admin can
> > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
> > >> IOPS.
> > >>>>>>>>>>
> > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
> > >> mine,
> > >>>> as
> > >>>>>> is
> > >>>>>>>>>> decided by the admin).
> > >>>>>>>>>>
> > >>>>>>>>>> I'm not sure what we can do about these two features
> > >> expressing
> > >>>> the
> > >>>>>>>>> speed
> > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
> > >>>> QoS in
> > >>>>>>>>> any
> > >>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
> > >>>> IOPS).
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > >>> <jb...@basho.com>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Wei,
> > >>>>>>>>>>>
> > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> > >>> Storage/Hypervisor
> > >>>>>> layer
> > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
> > >>>> hypervisor
> > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> > >> following
> > >>>>>> issues
> > >>>>>>>>>>> outstanding:
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > >>> read/write
> > >>>>>>>>> values
> > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
> > >> IOPS.
> > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> > >>> storage
> > >>>>>> volumes
> > >>>>>>>>>>> with provisioned IOPS?
> > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> > >>>> provisioned
> > >>>>>> IOPS
> > >>>>>>>>>>> that will permit providers to discriminate these guaranteed
> > >>>>>> operations
> > >>>>>>>>> in
> > >>>>>>>>>>> the event they want to bill for it differently?
> > >>>>>>>>>>> 4. What is the user experience for throttled and provisioned
> > >>> IOPS
> > >>>>>> that
> > >>>>>>>>>>> minimizes confusion of these concepts?
> > >>>>>>>>>>>
> > >>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> > >>>> throttled
> > >>>>>>>>> IOPS
> > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
> > >>>> items 1
> > >>>>>>>>> and 2.
> > >>>>>>>>>>> Since the two facilities would not be permitted to operate
> > >>>> together,
> > >>>>>>>>> they
> > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
> > >> can be
> > >>>>>>>>> resolved
> > >>>>>>>>>>> with an additional flag or two on the usage records. As for
> > >>>> Item 4,
> > >>>>>> I
> > >>>>>>>>> am
> > >>>>>>>>>>> not familiar with how these two enhancements are
> depicted in
> > >>> the
> > >>>>> user
> > >>>>>>>>>>> interface. I think we need to review the UI enhancements
> for
> > >>>> both
> > >>>>>>>>>>> enhancements and ensure they are consistent.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Do these solutions make sense?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> -John
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> <us...@gmail.com>>
> > >>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> John and Mike,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> 2780/2729,
> > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > >>> CLOUDSTACK-
> > >>>>> 2875 ) this
> > >>>>>>>>> week.
> > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
> > >> ask
> > >>>> for
> > >>>>>>>>> second
> > >>>>>>>>>>>> merge request after testing.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> I believe I understand where you're going with this, John,
> > >> and
> > >>>>> have
> > >>>>>>>>> been
> > >>>>>>>>>>>>> re-working this section of code today.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for the comments,
> > >>>>>>>>>>>>> Mike
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > >>>>> <jb...@basho.com>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> See my responses in-line below.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks for the response.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> whether or
> > >>>>> not a
> > >>>>>>>>> Volume
> > >>>>>>>>>>> is
> > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> > >>> actual
> > >>>>>>>>> isManaged
> > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> resources
> > >>>> for
> > >>>>>> the
> > >>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>> if it is managed."
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> method
> > >>>>> added to
> > >>>>>>>>> the
> > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> > >> then the
> > >>>>>>>>> storage
> > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
> > >>> method
> > >>>>> (which
> > >>>>>>>>>>> would
> > >>>>>>>>>>>>> be
> > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
> and
> > >>> this
> > >>>>>> would
> > >>>>>>>>>>> call
> > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
> on
> > >>>>>>>>> XenServer,
> > >>>>>>>>>>> an
> > >>>>>>>>>>>>> SR?
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> > >>>>> DataStoreDriver.
> > >>>>>>>>> Since
> > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
> I
> > >>>> would
> > >>>>>> add
> > >>>>>>>>> a
> > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> straw
> > >>> man
> > >>>>> term
> > >>>>>>>>> --
> > >>>>>>>>>>>>>> something other than volume) methods to
> > >>>>>>>>>>> allocate/create/deallocate/delete
> > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> > >>> property
> > >>>>> of
> > >>>>>>>>>>>>> DataStore
> > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> > >>> DataStoreDriver
> > >>>>> can be
> > >>>>>>>>>>>>> managed.
> > >>>>>>>>>>>>>> This approach allows operators to override
> manageability
> > >> of
> > >>>>>> devices.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> > >> plugin
> > >>>>>> would
> > >>>>>>>>> be
> > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
> to
> > >>>>> create any
> > >>>>>>>>>>>>>> directories or files necessary for the SR. There should
> > >> be no
> > >>>>>>>>>>>>> dependencies
> > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> > >> earlier,
> > >>>> such
> > >>>>>>>>>>>>> circular
> > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
> mess.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks again, John
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > >>>>> <jb...@basho.com>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
> that
> > >>>>>> supports n
> > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> > >> hypervisors.
> > >>>>>> Such
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> > >>> beast.
> > >>>>>>>>>>>>> Therefore, my
> > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
> the
> > >>>>> Storage
> > >>>>>> layer
> > >>>>>>>>>>>>> towards
> > >>>>>>>>>>>>>>> this separation.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> whether
> > >>> or
> > >>>>> not a
> > >>>>>>>>> Volume
> > >>>>>>>>>>> is
> > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
> an
> > >>> actual
> > >>>>>>>>>>> isManaged
> > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> resources
> > >>>> for
> > >>>>>> the
> > >>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
> > >> to
> > >>>>>> indicate
> > >>>>>>>>>>>>> whether
> > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
> > >> think
> > >>>> we
> > >>>>>>>>> need to
> > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
> > >>>>>> something
> > >>>>>>>>>>>>> that is
> > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> > >>> managing
> > >>>>> the
> > >>>>>>>>>>> device,
> > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> unmanaged).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
> in
> > >>> the
> > >>>>> Xen
> > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> Storage
> > >>>>> layer to
> > >>>>>>>>>>> perform
> > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
> > >> file,
> > >>>> etc).
> > >>>>>>>>>>> This
> > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> > >>>> hypervisor, as
> > >>>>>>>>> such,
> > >>>>>>>>>>>>>>> should be confined to its implementation. From my
> > >>>>> perspective,
> > >>>>>>>>>>> nothing
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
> From
> > >>> its
> > >>>>>>>>>>>>> perspective,
> > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
> to
> > >>>> query
> > >>>>>> the
> > >>>>>>>>>>> data
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
> level
> > >>>>> layers
> > >>>>>>>>> (e.g.
> > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> composition
> > >>> of
> > >>>>>>>>>>> operations
> > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> Volume
> > >>>>> storage
> > >>>>>>>>>>> protocol
> > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> implemented
> > >> in
> > >>>> my
> > >>>>>>>>> patch,
> > >>>>>>>>>>> we
> > >>>>>>>>>>>>>>> could do the following:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> > >>>>>>>>> VolumeManagerImpl
> > >>>>>>>>>>> we
> > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> > >>>>> hypervisor.
> > >>>>>> Before
> > >>>>>>>>>>> this
> > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> dealing
> > >>> with
> > >>>>>>>>> Dynamic
> > >>>>>>>>>>> (or
> > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
> > >>>> "Create
> > >>>>>>>>> SR"
> > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> would
> > >>> then
> > >>>>>>>>> proceed
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
> > >> add
> > >>>>>> another
> > >>>>>>>>>>>>>>> command to the agent code that is called for this
> > >> particular
> > >>>>> type
> > >>>>>>>>> of
> > >>>>>>>>>>>>>>> storage.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hey John,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I created a document for a customer today that
> outlines
> > >>> how
> > >>>>> the
> > >>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> > >> use
> > >>>> to
> > >>>>>>>>> you, as
> > >>>>>>>>>>>>> well,
> > >>>>>>>>>>>>>>>> as you perform the code review.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I have shared this document with you (you should
> have
> > >>>>> received
> > >>>>>>>>> that
> > >>>>>>>>>>>>>>>> information in a separate e-mail).
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Talk to you later!
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > >>>>>> jburwell@basho.com<ma...@basho.com>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> thoughts,
> > >>> but
> > >>>>>> want
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > >>>>>>>>>>>>> disk_io_throttle,
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> > >>>>> coherent
> > >>>>>>>>> form
> > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
> the
> > >>>> notion
> > >>>>>> of
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
> > >> the
> > >>>>> notion
> > >>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
> was
> > >>>>>>>>> incomplete,
> > >>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>> I had
> > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> > >> device.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> > >> this.
> > >>>> :)
> > >>>>> Do
> > >>>>>>>>> you
> > >>>>>>>>>>>>>>>>>> have a
> > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> > >>> what
> > >>>>> you're
> > >>>>>>>>> looking
> > >>>>>>>>>>>>>>>>>> for?
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > >>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
> the
> > >>>> notion
> > >>>>>> of
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> > >>>>> capabilities
> > >>>>>>>>> (i.e.
> > >>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> > >>> managed
> > >>>>> by
> > >>>>>>>>> CloudStack)
> > >>>>>>>>>>>>>>>>>> from the
> > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
> a
> > >>>>> semantic
> > >>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
> I
> > >>> also
> > >>>>> want
> > >>>>>>>>> to
> > >>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
> the
> > >>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> definitions.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > >>>>>>>>>>> jburwell@basho.com<ma...@basho.com>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> questions
> > >>> for
> > >>>>> me
> > >>>>>> with
> > >>>>>>>>>>>>>>>>>> regards to
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
> and
> > >>> SAN
> > >>>>>> volumes?
> > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
> (LUN
> > >>> 0).
> > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
> and
> > >>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> Volumes?
> > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> > >>> server or
> > >>>>>>>>> externally
> > >>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> operator?
> > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> > >>> volume
> > >>>>> (same
> > >>>>>> as a
> > >>>>>>>>>>> SAN
> > >>>>>>>>>>>>>>>>>> LUN) is
> > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
> plug-in)
> > >>>>> the
> > >>>>>> first
> > >>>>>>>>>>> time
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
> but
> > >>>> still
> > >>>>>>>>> want
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> use a
> > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> > >>> today
> > >>>>>> (prior
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> 4.2). The
> > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
> in
> > >>> this
> > >>>>> case -
> > >>>>>>>>>>>>>>>>>> multiple VMs
> > >>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
> you
> > >>> can do
> > >>>>> this
> > >>>>>>>>>>>>> today,
> > >>>>>>>>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
> server
> > >>> or
> > >>>>> externally
> > >>>>>>>>> by
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> operator?
> > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> > >>> volumes
> > >>>>> are
> > >>>>>>>>>>>>> completely
> > >>>>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
> There
> > >>> is
> > >>>>> no
> > >>>>>> admin
> > >>>>>>>>>>>>>>>>>>>> interaction.
> > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> > >>> volume
> > >>>>> and a
> > >>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>> volume,
> > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > >>>>> supports
> > >>>>>> true
> > >>>>>>>>>>> QoS.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> > >> related
> > >>>>> and
> > >>>>>>>>>>>>> expected
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> operate.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> multiple
> > >>> VMs
> > >>>>> with
> > >>>>>>>>>>>>>>>>>> provisioned
> > >>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> > >>> single
> > >>>>> SAN
> > >>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>> (LUN) to a
> > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> > >>> multiple
> > >>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> guarantee
> > >>> QoS.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> volume
> > >>> house
> > >>>>> more
> > >>>>>>>>> than
> > >>>>>>>>>>>>> one
> > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> > >>> without
> > >>>>> any of
> > >>>>>> my
> > >>>>>>>>>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>>>> code.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> volume
> > >>> for
> > >>>>>>>>>>> CloudStack's
> > >>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>> onto
> > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> concerns me
> > >>> --
> > >>>>>> concepts
> > >>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
> today
> > >>>> only
> > >>>>>> used
> > >>>>>>>>>>> (as
> > >>>>>>>>>>>>>>>>>> far as
> > >>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
> not
> > >>>>> PreSetup an
> > >>>>>>>>> SR).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> from
> > >>>>> VMware, it
> > >>>>>>>>> uses
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> vmfs
> > >>>>>>>>>>>>>>>>>>>>>> type.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> from
> > >>> KVM,
> > >>>>> it uses
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> > >>> here
> > >>>>> was to
> > >>>>>>>>> make a
> > >>>>>>>>>>>>>>>>>> new type
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> > >>> Edison
> > >>>>> added
> > >>>>>>>>> into
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> > >>> specificy,
> > >>>>> though:
> > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> > >>> Tutkowski
> > >>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
> know
> > >>> or
> > >>>>> care
> > >>>>>> that
> > >>>>>>>>> it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>>>>> used
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > >>>>> questions
> > >>>>>>>>> about
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > >>>>> SolidFire
> > >>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>> needs to
> > >>>>>>>>>>>>>>>>>>>>>>>>> call
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> > >>> hypervisor
> > >>>>> logic
> > >>>>>>>>>>> knows
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>> treat
> > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> constraint
> > >>>>> and just
> > >>>>>>>>>>>>>>>>>> calling it
> > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
> for
> > >>> the
> > >>>>> hypervisor
> > >>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> perform
> > >>> the
> > >>>>> necessary
> > >>>>>>>>>>>>>>>>>> activities.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> > >>> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > >>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
> able to
> > >>>>> manage a
> > >>>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > >>> CloudStack
> > >>>>> (e.g. the
> > >>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>> shared
> > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> > >>> chosen
> > >>>>> to
> > >>>>>>>>> dedicate
> > >>>>>>>>>>>>>>>>>> only a
> > >>>>>>>>>>>>>>>>>>>>>> portion
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> > >>> make
> > >>>>> sense?"
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
> SAN
> > >>>>> volume via
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
> would
> > >>>>> work unless
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
> an
> > >>> SR.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > >>>>> CloudStack, an
> > >>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>> allocates
> > >>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> > >>> exposes it
> > >>>>>> through
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> LUN. They
> > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
> to
> > >>>>> CloudStack,
> > >>>>>>>>> but
> > >>>>>>>>>>>>>>>>>> not allow
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > >>>>> changes, we
> > >>>>>>>>> would
> > >>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>> passing
> > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > >>>>> necessary
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>> (like
> > >>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
> have
> > >>> the
> > >>>>> storage
> > >>>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>>>>>> itself
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> support
> > >>> from
> > >>>>> the
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> side and
> > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
> to
> > >>> create
> > >>>>> the
> > >>>>>>>>> SR
> > >>>>>>>>>>>>>>>>>> before the
> > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
> plugin
> > >>>>> should
> > >>>>>> have a
> > >>>>>>>>>>>>> rich
> > >>>>>>>>>>>>>>>>>> enough
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> > >>> available
> > >>>>> for
> > >>>>>> volume
> > >>>>>>>>>>>>>>>>>> storage.
> > >>>>>>>>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > >>> interrogation of
> > >>>>> the
> > >>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
> itself
> > >>>>> shouldn't
> > >>>>>>>>> know
> > >>>>>>>>>>>>>>>>>> or care
> > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
> able
> > >>> to
> > >>>>>> answer
> > >>>>>>>>>>>>>>>>>> questions
> > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> > >>> driver
> > >>>>> (which
> > >>>>>>>>> is a
> > >>>>>>>>>>>>>>>>>> type of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> done, a
> > >>> new
> > >>>>> Primary
> > >>>>>>>>>>>>>>>>>> Storage shows
> > >>>>>>>>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> > >>> that
> > >>>>>> references
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>> tag
> > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> CloudStack
> > >>>>> volume. This
> > >>>>>>>>> leads
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> a new
> > >>>>>>>>>>>>>>>>>>>>>> row
> > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> CloudStack
> > >>>>> volume to a
> > >>>>>>>>> VM
> > >>>>>>>>>>>>>>>>>> (attach
> > >>>>>>>>>>>>>>>>>>>>>> disk).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
> calling
> > >>> the
> > >>>>> plug-in
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> create a
> > >>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> > >>> SAN).
> > >>>>> The
> > >>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>> also
> > >>>>>>>>>>>>>>>>>>>>>> updates
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
> data
> > >>>>> (like the
> > >>>>>>>>> IQN of
> > >>>>>>>>>>>>>>>>>> the SAN
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
> the
> > >>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> volume is in
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
> the
> > >>>>> volume
> > >>>>>>>>> will be
> > >>>>>>>>>>>>>>>>>> in the
> > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
> the
> > >>>>> plug-in
> > >>>>>>>>> code
> > >>>>>>>>>>>>>>>>>> will not
> > >>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
> and
> > >>>>> detects
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
> form
> > >>> of a
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > >>>>> terminology,
> > >>>>>>>>> volume,
> > >>>>>>>>>>>>>>>>>> in these
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > >>>>> allocation on
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> > >>> mean
> > >>>>> 'SAN
> > >>>>>>>>> volume'.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
> user
> > >>> can
> > >>>>> make in
> > >>>>>>>>>>>>>>>>>> CloudStack, I
> > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
> well
> > >>> make
> > >>>>> sense, but
> > >>>>>>>>> I'm
> > >>>>>>>>>>>>>>>>>> having a
> > >>>>>>>>>>>>>>>>>>>>>> hard
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
> in on
> > >>>>> this
> > >>>>>>>>> conversation
> > >>>>>>>>>>>>>>>>>> as he
> > >>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > >>> approach I
> > >>>>> took.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> > >>> John
> > >>>>> Burwell <
> > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
> two
> > >>>>> distinct
> > >>>>>>>>> concepts
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> protocols.
> > >>> In
> > >>>>> both
> > >>>>>>>>> cases, we
> > >>>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> system to
> > >>>>>> create/delete
> > >>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
> the
> > >>>>> volume to
> > >>>>>> be
> > >>>>>>>>>>>>>>>>>> volume to be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> context.
> > >>> To
> > >>>>> ensure
> > >>>>>>>>> that
> > >>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>> are in
> > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > >>>>> definitions,
> > >>>>>>>>> refers to
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> physical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> correct?
> > >>>>> Minimally,
> > >>>>>> we
> > >>>>>>>>>>>>> must
> > >>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>> able
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
> device
> > >>> to
> > >>>>> move bits
> > >>>>>>>>> from
> > >>>>>>>>>>>>>>>>>> one place
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > >>> Optionally, a
> > >>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> Therefore,
> > >>>>> we can have
> > >>>>>> a
> > >>>>>>>>>>>>>>>>>> unmanaged
> > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
> and
> > >>> we
> > >>>>> can have a
> > >>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> > >>> capable
> > >>>>> of
> > >>>>>>>>> allocating
> > >>>>>>>>>>>>>>>>>> LUNs and
> > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> CloudStack
> > >>> may
> > >>>>> be able
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> manage a
> > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > >>>>> unmanaged by
> > >>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> (e.g.
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
> the
> > >>>>> operator has
> > >>>>>>>>> chosen
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> dedicate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
> Does
> > >>> my
> > >>>>>> reasoning
> > >>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>> sense?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
> are
> > >>>>> reasonable, it
> > >>>>>>>>> seems
> > >>>>>>>>>>>>>>>>>>>> appropriate
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> concerns
> > >>> from
> > >>>>>>>>> StoragePoolType,
> > >>>>>>>>>>>>>>>>>> add the
> > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> attached
> > >>> driver
> > >>>>> that
> > >>>>>>>>>>> indicates
> > >>>>>>>>>>>>>>>>>> whether
> > >>>>>>>>>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> > >>> establish
> > >>>>> a
> > >>>>>>>>> abstraction
> > >>>>>>>>>>>>>>>>>>>>>> representing a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
> that
> > >>> is
> > >>>>>>>>> associated
> > >>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>> volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> hypervisor
> > >>>>> drivers can
> > >>>>>>>>>>>>> declare
> > >>>>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> encounter
> > >>> a
> > >>>>> device
> > >>>>>> managed
> > >>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>> CloudStack,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> exposed
> > >>> by
> > >>>>> the driver to
> > >>>>>>>>>>>>> automate
> > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> > >>> sense,
> > >>>>> then we can
> > >>>>>>>>> sit
> > >>>>>>>>>>>>>>>>>> down and
> > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> between
> > >>> the
> > >>>>> current
> > >>>>>> iSCSI
> > >>>>>>>>>>>>> type
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
> go in
> > >>> and
> > >>>>> create
> > >>>>>> a
> > >>>>>>>>>>>>>>>>>> Primary
> > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
> time,
> > >>>>> the
> > >>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>> volume must
> > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> allocated).
> > >>>>> Future
> > >>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
> created
> > >>>>> behind the
> > >>>>>>>>>>>>> scenes.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
> to
> > >>> go in
> > >>>>> and
> > >>>>>> create
> > >>>>>>>>>>>>>>>>>> Primary
> > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
> delete
> > >>>>> volumes on
> > >>>>>>>>> its
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>> system
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
> the
> > >>>>> storage
> > >>>>>>>>>>>>>>>>>> framework). When
> > >>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>> user
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
> that
> > >>> was
> > >>>>> created,
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>>>>>>>> tells
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> volume.
> > >>> After
> > >>>>> this is
> > >>>>>>>>> done,
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
> called.
> > >>>>> No
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> volume
> > >>> was
> > >>>>> just
> > >>>>>>>>> created.
> > >>>>>>>>>>>>> The
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
> PM,
> > >>> Mike
> > >>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
> should
> > >>>>> probably
> > >>>>>> have
> > >>>>>>>>>>>>>>>>>> defined them
> > >>>>>>>>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> > >>> pre-
> > >>>>> allocated
> > >>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>> an admin
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> > >>> hypervisor
> > >>>>> data
> > >>>>>>>>> structure
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
> SR),
> > >>> then
> > >>>>> that
> > >>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
> Disks
> > >>> (VDI)
> > >>>>> are later
> > >>>>>>>>>>>>> placed
> > >>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> > >>> these
> > >>>>> cases, the
> > >>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> > >>> already
> > >>>>> in place
> > >>>>>>>>> and
> > >>>>>>>>>>>>>>>>>> simply
> > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> structure
> > >>> to
> > >>>>> the VM in
> > >>>>>>>>>>>>>>>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
> that is
> > >>> not
> > >>>>>>>>>>> pre-allocated.
> > >>>>>>>>>>>>>>>>>> Instead
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
> could
> > >>> be a
> > >>>>> SAN
> > >>>>>> (not
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> volume on
> > >>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
> data
> > >>>>> structure
> > >>>>>>>>> must be
> > >>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> because
> > >>> these
> > >>>>> types of
> > >>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>> have not
> > >>>>>>>>>>>>>>>>>>>>>>>>> been
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > >>>>> structure by an
> > >>>>>>>>>>> admin.
> > >>>>>>>>>>>>>>>>>> Once
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> XenServer
> > >>> for
> > >>>>> this
> > >>>>>>>>> volume,
> > >>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>> attaches
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
> VM
> > >>> in
> > >>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
> PM,
> > >>>>> John Burwell
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> implementation of
> > >>> the
> > >>>>> Dynamic
> > >>>>>> type
> > >>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>> behavior
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
> why I
> > >>> ask
> > >>>>> about the
> > >>>>>>>>>>>>>>>>>> difference.
> > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
> the
> > >>>>> definition
> > >>>>>> of
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> Dynamic
> > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> > >>> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
> type
> > >>> is
> > >>>>>> uniquely
> > >>>>>>>>>>> used
> > >>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> > >>> Storage
> > >>>>> that is
> > >>>>>>>>> directly
> > >>>>>>>>>>>>>>>>>> based on
> > >>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> > >>> step of
> > >>>>> going
> > >>>>>> to
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
> on
> > >>> that
> > >>>>> iSCSI
> > >>>>>>>>> target as
> > >>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>> does
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > >>>>> supported for
> > >>>>>>>>>>>>>>>>>> XenServer.
> > >>>>>>>>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>>>>>>>> all
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
> first go
> > >>> to
> > >>>>> the
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> perform
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
> RBD is.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 2:13
> > >>> PM,
> > >>>>> John
> > >>>>>> Burwell
> > >>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
> code,
> > >>> what
> > >>>>> is the
> > >>>>>>>>>>> difference
> > >>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
> isn't
> > >>> RBD
> > >>>>> considered
> > >>>>>>>>>>>>> Dynamic?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
> PM,
> > >>> Mike
> > >>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> storage is
> > >>>>> defined in
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > >>>>> StoragePoolType {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
> local
> > >>>>> directory
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> NetworkFilesystem(true),
> > >>> //
> > >>>>> NFS or CIFS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> shared
> > >>> LUN,
> > >>>>> with a
> > >>>>>>>>>>> clusterfs
> > >>>>>>>>>>>>>>>>>>>> overlay
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
> ZFS
> > >>>>> Comstar
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
> image
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> XenServer
> > >>> local
> > >>>>> LVM SR
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> SharedMountPoint(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> VMware
> > >>> VMFS
> > >>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > >>>>> XenServer, Storage
> > >>>>>>>>> Pool
> > >>>>>>>>>>>>>>>>>> is set
> > >>>>>>>>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> XenServer
> > >>> local
> > >>>>> EXT SR
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> dynamic,
> > >>>>> zone-wide
> > >>>>>>>>> storage
> > >>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> StoragePoolType(boolean
> > >>>>> shared) {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> isShared() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 1:41
> > >>> PM,
> > >>>>> Mike
> > >>>>>>>>> Tutkowski
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> > >>> another
> > >>>>> storage
> > >>>>>>>>>>> company
> > >>>>>>>>>>>>>>>>>> wants
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> > >>> Quality
> > >>>>> of
> > >>>>>> Service
> > >>>>>>>>>>>>>>>>>> feature. It
> > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
> well.
> > >>>>> They would
> > >>>>>>>>> need
> > >>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
> the
> > >>>>> necessary
> > >>>>>>>>> changes to
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
> in.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> > >>> 1:39
> > >>>>> PM, Mike
> > >>>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
> the
> > >>>>> XenServer code
> > >>>>>> is
> > >>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> attach
> > >>> logic is
> > >>>>> now aware
> > >>>>>>>>> of
> > >>>>>>>>>>>>>>>>>> dynamic,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > >>>>> implementation of
> > >>>>>>>>> this
> > >>>>>>>>>>>>>>>>>> kind of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> > >>> with
> > >>>>> Edison's
> > >>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
> new
> > >>>>> framework that
> > >>>>>>>>>>> supported
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> dynamically.
> > >>>>> However, when I
> > >>>>>>>>>>>>>>>>>> visited with
> > >>>>>>>>>>>>>>>>>>>>>> him
> > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
> that
> > >>> it
> > >>>>> was not
> > >>>>>>>>>>>>>>>>>> complete. We
> > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
> could
> > >>> do
> > >>>>> with these
> > >>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>> unless
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> recognize
> > >>> this
> > >>>>> new type of
> > >>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> create
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> structure.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> > >>> 1:28
> > >>>>> PM, John
> > >>>>>>>>> Burwell
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
> to me
> > >>>>> that any
> > >>>>>>>>>>> operation
> > >>>>>>>>>>>>>>>>>> in the
> > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> > >>> about
> > >>>>> details. I
> > >>>>>>>>> expect
> > >>>>>>>>>>>>>>>>>> to see
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> > >>> operations
> > >>>>> that can be
> > >>>>>>>>>>>>>>>>>>>> composed/driven
> > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > >>>>> allocate
> > >>>>>> space/create
> > >>>>>>>>>>>>>>>>>> structures
> > >>>>>>>>>>>>>>>>>>>>>> per
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > >>>>> dependency, we are
> > >>>>>>>>> going
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> end
> > >>>>>>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
> make
> > >>> the
> > >>>>> system
> > >>>>>>>>>>>>> increasingly
> > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > >>>>> understanding that the
> > >>>>>>>>> Xen
> > >>>>>>>>>>>>>>>>>> specific
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > >>>>> CitrixResourceBase
> > >>>>>> class?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013 at
> > >>> 3:12
> > >>>>> PM, Mike
> > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
> in a
> > >>> bit
> > >>>>> more
> > >>>>>>>>> detail:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
> aside
> > >>> from
> > >>>>> one setup
> > >>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> > >>> volume on
> > >>>>> the
> > >>>>>> storage
> > >>>>>>>>>>>>>>>>>> system,
> > >>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> > >>> structure
> > >>>>> to make
> > >>>>>> use
> > >>>>>>>>> of
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > >>> XenServer
> > >>>>> or a
> > >>>>>> datastore
> > >>>>>>>>> on
> > >>>>>>>>>>>>>>>>>> ESX(i)).
> > >>>>>>>>>>>>>>>>>>>>>> VMs
> > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> storage
> > >>>>> system's volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> > >>> storage
> > >>>>> framework,
> > >>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> > >>> create
> > >>>>> a 1:1
> > >>>>>>>>>>>>> relationship
> > >>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
> VM's
> > >>> data
> > >>>>> disk
> > >>>>>>>>> (necessary
> > >>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
> a
> > >>> plug-
> > >>>>> in that is
> > >>>>>>>>> called
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> needed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
> the
> > >>>>> storage
> > >>>>>> framework
> > >>>>>>>>> did
> > >>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>> address
> > >>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > >>> hypervisor-
> > >>>>> specific data
> > >>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>> when
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
> case,
> > >>> I've
> > >>>>> been
> > >>>>>>>>> enhancing
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> to do
> > >>>>>>>>>>>>>>>>>>>> so.
> > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > >>>>> submitted. I've got
> > >>>>>>>>> ESX(i)
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>>>>>>>> sandbox
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
> 4.2
> > >>>>> freeze date.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
> bit? :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013 at
> > >>>>> 1:03 PM, Mike
> > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> plug-in -
> > >>> by
> > >>>>> itself - is
> > >>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
> the
> > >>>>> volume-attach
> > >>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>> (in the
> > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
> into
> > >>> the
> > >>>>> plug-in to
> > >>>>>>>>> have
> > >>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>> create a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> > >>> comes to
> > >>>>> attach the
> > >>>>>>>>> volume
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> > >>> enough
> > >>>>> to recognize
> > >>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> volume
> > >>> has
> > >>>>> just been
> > >>>>>>>>> created)
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>> create,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > >>>>> XenServer) or a
> > >>>>>>>>> datastore
> > >>>>>>>>>>>>>>>>>> (for
> > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
> just
> > >>>>> created.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> spending
> > >>> most
> > >>>>> of my time
> > >>>>>>>>>>>>> recently
> > >>>>>>>>>>>>>>>>>> making
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> code.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
> up?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> > >>> at
> > >>>>> 12:48 PM,
> > >>>>>> John
> > >>>>>>>>>>>>>>>>>> Burwell <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> > >>> why
> > >>>>> the the storage
> > >>>>>>>>>>>>> driver
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> at
> > >>> 1:21
> > >>>>> PM, Mike
> > >>>>>>>>>>>>> Tutkowski
> > >>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> ultimately I
> > >>>>> would like to
> > >>>>>>>>> support
> > >>>>>>>>>>>>>>>>>> all
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
> think
> > >>> I'm
> > >>>>> just out of
> > >>>>>>>>> time
> > >>>>>>>>>>>>>>>>>> for 4.2
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> get
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> > >>> plug-in
> > >>>>> supports
> > >>>>>>>>>>>>> XenServer.
> > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> > >>> feature
> > >>>>> freeze, I
> > >>>>>>>>> have
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> working
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
> be
> > >>> clear,
> > >>>>> this is
> > >>>>>> all
> > >>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> regards
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> Compute
> > >>>>> Offerings post
> > >>>>>> 4.2.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > >>> 2013
> > >>>>> at 11:14 AM,
> > >>>>>>>>> Kelcey
> > >>>>>>>>>>>>>>>>>> Jamison
> > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca<ma...@bbits.ca>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
> plan
> > >>> on
> > >>>>> supporting
> > >>>>>>>>> KVM in
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > >>>>> Message -----
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > >>>>> Tutkowski" <
> > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > >>>>> dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> Monday,
> > >>> June
> > >>>>> 3, 2013
> > >>>>>>>>> 10:12:32 AM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> > >>> [MERGE]
> > >>>>>>>>>>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>> MASTER
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> > >>> merging
> > >>>>> Wei's feature
> > >>>>>>>>>>> first,
> > >>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>> mine.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
> is
> > >>> for
> > >>>>> KVM only,
> > >>>>>>>>> then
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> is a
> > >>>>>>>>>>>>>>>>>>>> non
> > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> 3,
> > >>> 2013
> > >>>>> at 8:55 AM,
> > >>>>>> Wei
> > >>>>>>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com<ma...@gmail.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> billing,
> > >>> as
> > >>>>> no one works
> > >>>>>>>>> on
> > >>>>>>>>>>>>>>>>>> billing
> > >>>>>>>>>>>>>>>>>>>> now,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
> by
> > >>>>> themselves. They
> > >>>>>>>>> can
> > >>>>>>>>>>>>> get
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk_offering of
> > >>> a
> > >>>>> VMs and
> > >>>>>> volumes
> > >>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
> the
> > >>>>> exact
> > >>>>>> limitation
> > >>>>>>>>>>>>> value
> > >>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> limitation
> > >>> for
> > >>>>> nics as
> > >>>>>> well.
> > >>>>>>>>> I
> > >>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
> I/O
> > >>>>> throttling?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > >>>>> misunstand the second
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> > >>> what
> > >>>>> you said about
> > >>>>>> the
> > >>>>>>>>>>> two
> > >>>>>>>>>>>>>>>>>>>> features.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> John
> > >>>>> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> 2013,
> > >>> at
> > >>>>> 2:13 AM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com<ma...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> Mike
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> Mike's
> > >>>>> aswer helps you.
> > >>>>>>>>> I am
> > >>>>>>>>>>>>>>>>>> trying
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> think
> > >>> billing
> > >>>>> should
> > >>>>>>>>> depend
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>> IO
> > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> limitation.
> > >>> Please
> > >>>>> review
> > >>>>>>>>>>>>>>>>>> disk_io_stat if
> > >>>>>>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
> IO
> > >>>>> statistics
> > >>>>>> including
> > >>>>>>>>>>>>>>>>>> bytes/iops
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> > >>> machine.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
> the
> > >>> AWS
> > >>>>> model,
> > >>>>>> customers
> > >>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>> billed
> > >>>>>>>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> provisioned
> > >>> IOPS,
> > >>>>> as well as,
> > >>>>>> for
> > >>>>>>>>>>>>> those
> > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://aws.amazon.com/ebs/).
> > >>>>>> I
> > >>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>> imagine
> > >>>>>>>>>>>>>>>>>>>>>>>>> our
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> > >>> employ
> > >>>>> similar cost
> > >>>>>>>>>>> models.
> > >>>>>>>>>>>>>>>>>> Could
> > >>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
> model
> > >>> in
> > >>>>> the current
> > >>>>>>>>> patch?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
> you
> > >>> mean
> > >>>>> IOPS runtime
> > >>>>>>>>>>> change?
> > >>>>>>>>>>>>>>>>>> KVM
> > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> limitation for
> > >>> a
> > >>>>> running
> > >>>>>> virtual
> > >>>>>>>>>>>>>>>>>> machine
> > >>>>>>>>>>>>>>>>>>>>>> through
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> CloudStack
> > >>> does
> > >>>>> not support
> > >>>>>>>>>>> changing
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (computer
> > >>>>> offering or disk
> > >>>>>>>>>>>>> offering).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
> the
> > >>>>> Java interface
> > >>>>>>>>> level.
> > >>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>> apologize
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > >>>>> generalize allocation
> > >>>>>>>>>>>>>>>>>> algorithms
> > >>>>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>> set
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
> the
> > >>>>> service guarantees
> > >>>>>>>>>>>>>>>>>> provided by a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> > >>> good
> > >>>>> question.
> > >>>>>>>>> Maybe it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>> better
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> disk_io_throttling as Mike
> > >>>>>>>>> needs to
> > >>>>>>>>>>>>>>>>>> consider
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
> think.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> expand
> > >>> on
> > >>>>> my thoughts
> > >>>>>> in a
> > >>>>>>>>>>>>> later
> > >>>>>>>>>>>>>>>>>>>>>> response
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
> points
> > >>>>> between these two
> > >>>>>>>>>>>>>>>>>> features. I
> > >>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
> to be
> > >>>>> merged before
> > >>>>>>>>>>>>>>>>>> SolidFire, but
> > >>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> coordination
> > >>>>> between the
> > >>>>>> branches
> > >>>>>>>>>>>>>>>>>> (possibly
> > >>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> disk_io_throttling)
> > >>>>> to
> > >>>>>>>>> coordinate on
> > >>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>> issue.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > >>> John
> > >>>>> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com<ma...@basho.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> things I
> > >>>>> want to
> > >>>>>> understand
> > >>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
> there
> > >>>>> value in
> > >>>>>> capturing
> > >>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>> policies
> > >>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> model
> > >>> (e.g.
> > >>>>> for
> > >>>>>>>>> billing/usage
> > >>>>>>>>>>>>>>>>>>>> purposes,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> Should
> > >>> there
> > >>>>> be a common
> > >>>>>>>>>>>>>>>>>> interface model
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> provisioning
> > >>> at
> > >>>>> runtime?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
> are
> > >>>>> conflicting
> > >>>>>>>>> provisioned
> > >>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> hypervisor
> > >>> and
> > >>>>> storage device
> > >>>>>>>>>>>>>>>>>> reconciled?
> > >>>>>>>>>>>>>>>>>>>> In
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
> to
> > >>>>> believe (and
> > >>>>>> billed)
> > >>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>> device
> > >>>>> has been
> > >>>>>>>>> configured
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> deliver.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> consistent
> > >>>>> configuration
> > >>>>>>>>> between a
> > >>>>>>>>>>>>>>>>>> VM and a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
> but a
> > >>>>> later
> > >>>>>> modification
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> inconsistency.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
> 2,
> > >>> 2013,
> > >>>>> at 8:38 PM,
> > >>>>>>>>> Mike
> > >>>>>>>>>>>>>>>>>> Tutkowski
> > >>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> believe
> > >>> Wei's
> > >>>>> feature deals
> > >>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>> controlling
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>> hypervisor
> > >>>>> side.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> feature
> > >>> is
> > >>>>> focused on
> > >>>>>>>>>>>>> controlling
> > >>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> that
> > >>>>> helps. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
> Jun
> > >>> 2,
> > >>>>> 2013 at 6:35
> > >>>>>> PM,
> > >>>>>>>>>>>>> John
> > >>>>>>>>>>>>>>>>>>>> Burwell
> > >>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> jburwell@basho.com<ma...@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > >>> opinion is
> > >>>>> that no
> > >>>>>> features
> > >>>>>>>>>>>>>>>>>> should be
> > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> > >>> have
> > >>>>> been resolved
> > >>>>>> and
> > >>>>>>>>> it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>> ready
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> Ops
> > >>> vs
> > >>>>> discrete
> > >>>>>>>>> read/write
> > >>>>>>>>>>>>>>>>>> ops issue
> > >>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> reviewed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> I
> > >>> don't
> > >>>>> think this
> > >>>>>>>>> criteria
> > >>>>>>>>>>>>>>>>>> has been
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> how
> > >>>>> does this work
> > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> > >>>>>>>>>>>>>>>>>> )?
> > >>>>>>>>>>>>>>>>>>>> As I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> > >>> involves
> > >>>>> provisioned
> > >>>>>>>>> IOPS. I
> > >>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> scenario
> > >>>>> where provisioned
> > >>>>>>>>> IOPS
> > >>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> KVM and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> unnecessarily
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> incompatible.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Jun 1,
> > >>> 2013,
> > >>>>> at 6:47 AM,
> > >>>>>>>>> Wei
> > >>>>>>>>>>>>>>>>>> ZHOU <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com<ma...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
> will
> > >>>>> change it next
> > >>>>>>>>> week.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/6/1
> > >>>>> Wido den Hollander
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl<ma...@widodh.nl>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 06/01/2013 08:24 AM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Exactly. I
> > >>> have
> > >>>>> pushed the
> > >>>>>>>>>>>>> features
> > >>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> anyone
> > >>>>> object thems for
> > >>>>>>>>>>>>> technical
> > >>>>>>>>>>>>>>>>>>>> reason
> > >>>>>>>>>>>>>>>>>>>>>>>>> till
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> the
> > >>> sake
> > >>>>> of clarity I
> > >>>>>> just
> > >>>>>>>>>>>>> want
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> mention
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> total
> > >>> IOps
> > >>>>> to R/W IOps
> > >>>>>>>>> asap
> > >>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>>>>>>>>>> never
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> total
> > >>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
> laid
> > >>> the
> > >>>>> groundwork for
> > >>>>>>>>> the
> > >>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>> throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> however
> > >>>>> prevent that we
> > >>>>>> create
> > >>>>>>>>>>>>>>>>>> legacy from
> > >>>>>>>>>>>>>>>>>>>>>> day
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/5/31
> > >>>>> Wido den
> > >>>>>> Hollander <
> > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl<ma...@widodh.nl>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 05/31/2013 03:59 PM, John
> > >>>>>>>>>>>>> Burwell
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
> this
> > >>>>> enhancement must
> > >>>>>> to
> > >>>>>>>>>>>>>>>>>> discretely
> > >>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> see
> > >>> how
> > >>>>> it could be
> > >>>>>>>>> fixed
> > >>>>>>>>>>>>>>>>>> later
> > >>>>>>>>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> correctly
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
> total
> > >>> IOPS
> > >>>>> into read
> > >>>>>> and
> > >>>>>>>>>>>>>>>>>> write.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>> unless/until we
> > >>>>>> decided
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> break
> > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
> Wei
> > >>>>> meant was merging
> > >>>>>> it
> > >>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>> master
> > >>>>>>>>>>>>>>>>>>>>>> now
> > >>>>>>>>>>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> branch
> > >>>>> and add Read /
> > >>>>>>>>> Write
> > >>>>>>>>>>>>> IOps
> > >>>>>>>>>>>>>>>>>>>> before
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > >>>>> released with Read
> > >>>>>> and
> > >>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>> instead
> > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
> to
> > >>>>> make the May 31st
> > >>>>>>>>>>>>> feature
> > >>>>>>>>>>>>>>>>>>>> freeze
> > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
> other
> > >>>>> threads) then it
> > >>>>>>>>> won't
> > >>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>> necessary
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > >>>>> completely agree that
> > >>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>>> is no
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> I/O.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> May 31,
> > >>>>> 2013, at 9:51 AM,
> > >>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>> den
> > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl<ma...@widodh.nl>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 05/31/2013 03:13 PM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks.
> > >>> Good
> > >>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> thought
> > >>>>> about at the
> > >>>>>>>>>>> beginning.
> > >>>>>>>>>>>>>>>>>>>> Finally I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> difference
> > >>> of
> > >>>>> read and write
> > >>>>>>>>>>>>> mainly
> > >>>>>>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
> the
> > >>>>> difference of sent
> > >>>>>>>>> and
> > >>>>>>>>>>>>>>>>>> received
> > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > >>>>> reasoning seems odd.
> > >>>>>>>>>>>>>>>>>> Networking and
> > >>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> I/O is
> > >>>>> much more
> > >>>>>>>>> expensive
> > >>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> most
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> bandwith.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> Implementing
> > >>>>> it will be some
> > >>>>>>>>>>>>>>>>>> copy-paste
> > >>>>>>>>>>>>>>>>>>>>>> work.
> > >>>>>>>>>>>>>>>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> implemented
> > >>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
> days.
> > >>> For
> > >>>>> the deadline
> > >>>>>> of
> > >>>>>>>>>>>>>>>>>> feature
> > >>>>>>>>>>>>>>>>>>>>>> freeze,
> > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
> if
> > >>>>> needed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> think it's
> > >>> a
> > >>>>> feature we
> > >>>>>>>>> can't
> > >>>>>>>>>>>>>>>>>> miss. But
> > >>>>>>>>>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> window
> > >>> we
> > >>>>> have to make sure
> > >>>>>> we
> > >>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
> 4.3,
> > >>> that
> > >>>>> will confuse
> > >>>>>>>>>>>>> users.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/5/31
> > >>>>> Wido den
> > >>>>>> Hollander <
> > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl<ma...@widodh.nl>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 05/30/2013 06:03 PM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> would
> > >>> like to
> > >>>>> merge
> > >>>>>>>>>>>>>>>>>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> nobody
> > >>>>> object, I will
> > >>>>>> merge
> > >>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>> master
> > >>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>> purpose
> > >>>>> is :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > >>>>> machines are running
> > >>>>>>>>> on
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> same
> > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> storage or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> > >>> strage).
> > >>>>> Because of
> > >>>>>> the
> > >>>>>>>>>>> rate
> > >>>>>>>>>>>>>>>>>>>>>> limitation
> > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
> if
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
> VM
> > >>> has
> > >>>>> large disk
> > >>>>>>>>> operation,
> > >>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>>>>>>>> affect
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> VMs
> > >>>>> running on the
> > >>>>>> same
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>> device.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > >>> neccesary
> > >>>>> to set the
> > >>>>>>>>>>> maximum
> > >>>>>>>>>>>>>>>>>> rate
> > >>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Looking at
> > >>> the
> > >>>>> code I see
> > >>>>>> you
> > >>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
> and
> > >>>>> libvirt support
> > >>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>> both a
> > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > >>> which
> > >>>>> could benefit a
> > >>>>>>>>> lot of
> > >>>>>>>>>>>>>>>>>> users.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
> also
> > >>>>> strange, in the
> > >>>>>>>>> polling
> > >>>>>>>>>>>>>>>>>> side you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> but
> > >>> on
> > >>>>> the throttling
> > >>>>>>>>> side
> > >>>>>>>>>>>>>>>>>> you only
> > >>>>>>>>>>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> IOps
> > >>> are
> > >>>>> usually much
> > >>>>>>>>> more
> > >>>>>>>>>>>>>>>>>> expensive
> > >>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
> valid
> > >>>>> use-case where
> > >>>>>>>>> that
> > >>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>> admin
> > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> vs
> > >>> Read
> > >>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
> this
> > >>> only
> > >>>>> supports KVM
> > >>>>>>>>> at
> > >>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>> point I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> to at
> > >>>>> least have the
> > >>>>>>>>>>>>>>>>>> mechanism in
> > >>>>>>>>>>>>>>>>>>>>>> place
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> implementing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> later
> > >>>>> would be a lot of
> > >>>>>>>>>>> work.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> > >>> hypervisor
> > >>>>> doesn't
> > >>>>>>>>> support
> > >>>>>>>>>>>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> you
> > >>> can
> > >>>>> always sum
> > >>>>>> both
> > >>>>>>>>> up
> > >>>>>>>>>>>>>>>>>> and set
> > >>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> you
> > >>>>> explain why you
> > >>>>>>>>>>>>> implemented
> > >>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>> feature
> > >>>>> includes:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> the
> > >>>>> maximum rate of
> > >>>>>>>>> VMs
> > >>>>>>>>>>>>> (in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> configuration)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> change
> > >>> the
> > >>>>> maximum rate
> > >>>>>> of
> > >>>>>>>>>>> VMs
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> limit
> > >>> the
> > >>>>> disk rate
> > >>>>>> (total
> > >>>>>>>>>>> bps
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>> iops)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
> ticket:
> > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> issues.apache.org/****<http://issues.apache.org/****>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> jira/browse/CLOUDSTACK-1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1192>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-<http://issues.apache.org/**jira/**browse/CLOUDSTACK->
> > >>> 1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> 1192
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> 1192>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
> will
> > >>>>> update later) :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>> VM+Disk+IO+Throttling<https://
<https://%0b>> > >>>>>>>>>>>>> ****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> cwiki.apache.org/confluence/****<http://cwiki.apache.org/confluence/****>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>
> > >>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<http://apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >>>>> rottling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Merge
> > >>> check
> > >>>>> list :-
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
> you
> > >>>>> check the branch's
> > >>>>>>>>> RAT
> > >>>>>>>>>>>>>>>>>> execution
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> > >>> there
> > >>>>> new dependencies
> > >>>>>>>>>>>>>>>>>> introduced?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> What
> > >>>>> automated testing
> > >>>>>> (unit
> > >>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> feature?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
> tests
> > >>> are
> > >>>>> added.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> What
> > >>>>> testing has been done
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> check for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> the
> > >>>>> bytes rate and
> > >>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>> rate on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> VM
> > >>>>> operations, including
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> deploy,
> > >>> stop,
> > >>>>> start, reboot,
> > >>>>>>>>>>>>>>>>>> destroy,
> > >>>>>>>>>>>>>>>>>>>>>> expunge.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > >>> Volume
> > >>>>> operations,
> > >>>>>>>>> including
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > >>>>> Detach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> review
> > >>> the
> > >>>>> code, you can
> > >>>>>>>>> try
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> > >>> regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>> VM+Disk+IO+Throttling<https://
<https://%0b>> > >>>>>>>>>>>>> ****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> cwiki.apache.org/confluence/****<http://cwiki.apache.org/confluence/****>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>
> > >>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<http://apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >>>>> rottling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > >>>>>>>>> refs/heads/disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >>>>> 1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-<http://issues.apache.org/****jira/browse/CLOUDSTACK->
> > >>> 1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1301>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-<http://issues.apache.org/**jira/**browse/CLOUDSTACK->
> > >>> 1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> 1301
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> 1301>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> issues.apache.org/****jira/**browse/CLOUDSTACK-<http://issues.apache.org/****jira/**browse/CLOUDSTACK->
> > >>> 2071
> > >>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >>>>> 2071
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >>>>> 2071
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> 2071
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >>>>> 2071
> > >>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 2071>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <h**ttps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071><
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> 2071>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> (**CLOUDSTACK-1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> VM
> > >>> Disk
> > >>>>> I/O
> > >>>>>> Throttling)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >>>>> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >>>>> CloudStack Developer,
> > >>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>>>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >>> 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> > >>> the
> > >>>>> way the world
> > >>>>>>>>> uses
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >>> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >>> CloudStack
> > >>>>> Developer,
> > >>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> > >>> way
> > >>>>> the world uses
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >>> CloudStack
> > >>>>> Developer,
> > >>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > >>> way
> > >>>>> the world uses
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> > >>>>> Developer,
> > >>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> > >>> the
> > >>>>> world uses the
> > >>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> > >>>>> Developer,
> > >>>>>> SolidFire
> > >>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> > >>> the
> > >>>>> world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >>>>> Developer, SolidFire
> > >>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> > >>>>> world uses the
> > >>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >>> Developer,
> > >>>>> SolidFire
> > >>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> > >>> world
> > >>>>> uses the
> > >>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >>> Developer,
> > >>>>> SolidFire
> > >>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > >>> world
> > >>>>> uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > >>>>> SolidFire
> > >>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> > >>> uses
> > >>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > >>>>> SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> > >>> uses
> > >>>>> the cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>>>> SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> > >>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302<tel:303.746.7302>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> > >>> the
> > >>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > >>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > >>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > >>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > >>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> > >>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>> *(tm)*
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> *Mike Tutkowski*
> > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>>> o: 303.746.7302
> > >>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>> *(tm)*
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> *Mike Tutkowski*
> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>>>> o: 303.746.7302
> > >>>>>>> Advancing the way the world uses the
> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>> *(tm)*
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> *Mike Tutkowski*
> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>>>> o: 303.746.7302
> > >>>>> Advancing the way the world uses the
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>> *(tm)*
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> *Mike Tutkowski*
> > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > >>> o: 303.746.7302
> > >>> Advancing the way the world uses the
> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>> *(tm)*
> > >>
> > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *(tm)*
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
--
Mike Tutkowski
Senior CloudStack Developer, SolidFire Inc.
e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
o: 303.746.7302
Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm)
--
Mike Tutkowski
Senior CloudStack Developer, SolidFire Inc.
e: mike.tutkowski@solidfire.com<ma...@solidfire.com>
o: 303.746.7302
Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm)
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Another thing we might want to consider for 4.2 is that the way the code is
currently implemented in master, I don't think it's even possible to write
a plug-in that connects storage directly to a VM because the CloudStack
storage framework will call the "attach" method of the hypervisor in use
and this method will fail because it assumes there is (talking Xen here) an
SR existent and in this situation there isn't one.
My thinking is that we should ignore the direct-attach-of-volume-to-VM use
case for 4.2. I think it will require too much of a rewrite at this point
in development (fewer than three weeks left until code freeze).
If the plug-in implements an isManaged() method (as John suggested), the
storage framework can invoke the necessary commands on the hypervisor in
use.
For example, the createAsync method is called. The storage plug-in creates
a SAN volume and updates the CS DB. The storage framework then asks the
driver if it's managed. If it is, the storage framework executes a command
against the hypervisor in use to have it create, say, an SR (with a single
VDI that takes up the entire SR). Now, when the storage framework executes
the "attach" command against the hypervisor, this command will work, as
well (because that logic assumes the prior existence of an SR and it will,
in fact, be present).
On Sun, Jun 9, 2013 at 12:37 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Hi Edison and John,
>
> I wanted to point out something I thought was relevant to this
> conversation.
>
> My plug-in supports both XenServer and ESX(i).
>
> These two hypervisors handle their respective hypervisor data structures
> (XenServer SR or ESX(i) datastore) quite differently.
>
> In XenServer, the SR that resides on the SAN LUN is simply "forgotten"
> when you issue a remove-SR command to XenServer.
>
> When you later want to access the contents of that data, it is still
> safely stored on the LUN and can be accessed from XenServer (from the same
> or different cluster (what XenServer calls a resource pool)) by creating a
> new SR that is based on the same IQN and LUN.
>
> XenServer SRs exist at the cluster level.
>
> In ESX(i), the datastore that resides on the SAN LUN is actually destroyed
> when you issue a remove-datastore command to ESX(i). In other words, the
> contents of the SAN LUN are destroyed.
>
> ESX(i) datastores exist at the datacenter level.
>
> The XenServer behavior (not destroying the contents of the LUN when the SR
> is removed) is desirable; the ESX(i) behavior (destroying the contents of
> the LUN when the datastore is removed) is not desirable.
>
> I can get around the ESX(i) behavior by removing the referenced IQN from
> each host in the cluster when the CS storage framework notifies my plug-in
> of a detach-volume event. When the CS volume is later attached again
> (either to the same or a different ESX(i) cluster), I can have each host in
> the relevant ESX(i) cluster reference the IQN and the datastore will be
> back again.
>
> This sequence of events is critical for my plug-in to support both
> environments:
>
> *XenServer flow (events for Driver class):*
>
> createAsync (called right before first attach) = creates SAN volume
>
> preAttachVolume = creates SR and creates VDI or just introduces SR
>
> (CS volume gets attached and later detached)
>
> postDetachVolume = removes SR (this does not delete data on SAN volume)
>
> deleteAsync (can be called when volume is not attached) = deletes the SAN
> volume
>
> *VMware flow (events for Driver class):*
>
> createAsync (called right before first attach) = creates SAN volume,
> creates datastore, and creates VMDK file
>
> preAttachVolume = adds iSCSI target to each host in cluster
>
> (CS volume gets attached and later detached)
>
> postDetachVolume = removes iSCSI target from each host in cluster
>
> deleteAsync (can be called when volume is not attached) = deletes
> datastore and deletes the SAN volume
>
> ********************
> So, the way I have it written up here, the driver is talking to the
> hypervisor. As John has pointed out, this may not be ideal.
>
> Whatever we decide on, it needs to be able to support this kind of flow. :)
>
> We could go John's route, where the driver says if it's managed or not.
>
> If it is, then the storage framework could send the necessary messages to
> the hypervisor in question at the right times (just migrate the hypervisor
> logic above from the driver to the right parts of the storage framework).
>
> This is fine, except that it does not resolve Edison's use case of
> desiring to have the storage LUN potentially connected directly to a VM.
>
> I'm not sure we should design around that use case, though. Is it common
> enough? Has anyone asked for it? Is it even desirable? How are usage
> statistics gathered for such volumes? It's nice to offer the flexibility,
> but is it a good idea in this case?
>
> Thanks!
>
>
>
> On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:
>
>>
>>
>> > -----Original Message-----
>> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > Sent: Friday, June 07, 2013 4:08 PM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >
>> > Hi John,
>> >
>> > Yes, you are correct that Xen must layer an SR on top of the iSCSI
>> volume
>> > to use it. Same story for VMware: ESX(i) must layer a datastore on top
>> of
>> > the iSCSI volume to use it. I look at it like they are layering a
>> clustered
>> > file system on the SAN volume so the hypervisors can share access to the
>> > contents of the volume.
>> >
>> > In 4.1, by the time the hypervisor-attach-volume logic is called, the
>> SR or
>> > datastore has already been created (usually manually by an admin).
>> >
>> > This pre-setup of the, say, SR is, of course, not acceptable in an
>> > environment where each CloudStack volume that a user creates is mapped
>> > to a
>> > single SAN volume (via the SR).
>> >
>> > The question comes down to who should allocate the SR dynamically.
>> >
>> > We could have the storage framework ask the storage plug-in if it is
>> > managed. If it is, then the storage framework could send a message to
>> the
>> > hypervisor in question to create (let's talk Xen here) the SR ahead of
>> > time. Then, when the storage framework next sends the attach-volume
>> > command
>> > to the hypervisor, it should work without changes to that attach-volume
>> > logic (because - from the point of view of the attach logic - the SR is
>> > already existent, as expected).
>> >
>> > Now, as Edison has pointed out, this limits the power of the storage
>> > plug-in. A storage plug-in in this model cannot directly attach storage
>> to
>> > a VM (it must go through the hypervisor). Perhaps that is OK. We need to
>> > make a call on that.
>>
>>
>> Yes, I think we need to address this usage case, and won't limit the
>> power of the storage plug-in.
>> John, what's your idea to fit this usage case into your abstraction?
>>
>>
>> >
>> > If we want to give more power to the storage vendor, we would have to
>> > have
>> > the storage framework call into the storage plug-in to send the
>> appropriate
>> > attach commands to the hypervisor.
>> >
>> > Let's discuss and come to a consensus (or at least agree on a path) as
>> soon
>> > as we can. :)
>> >
>> > Thanks!
>> >
>> >
>> >
>> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> > > Mike,
>> > >
>> > > My understanding is the Xen expects an SR structure on all ISCSI
>> devices
>> > > -- at least that is how I read the code in your patch. Is my
>> understanding
>> > > correct? If so, the Xen plugin should be able to query the storage
>> device
>> > > to determine the presence of the SR structure and create it if it
>> does not
>> > > exist. Am I missing something in the implementation that makes that
>> type
>> > > of implementation impossible?
>> > >
>> > > Thanks,
>> > > -John
>> > >
>> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> > > wrote:
>> > >
>> > > > Yeah, if a storage vendor wanted to attach a volume directly to a
>> VM in
>> > > 4.2
>> > > > today, it would probably fail because the attach-volume logic
>> assumes
>> > the
>> > > > existence of the necessary hypervisor data structure (ex. SR on
>> Xen).
>> > > >
>> > > > If we wanted to enable such an attach, we could do it the way Edison
>> > > > suggests.
>> > > >
>> > > >
>> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
>> wrote:
>> > > >
>> > > >>
>> > > >>
>> > > >>> -----Original Message-----
>> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > > >>> Sent: Friday, June 07, 2013 2:37 PM
>> > > >>> To: dev@cloudstack.apache.org
>> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> > > >>>
>> > > >>> As we only have three weeks until feature freeze, we should come
>> to
>> > a
>> > > >>> consensus on this design point as soon as possible.
>> > > >>>
>> > > >>> Right now, if the storage framework asks my driver it is is
>> managed, it
>> > > >>> will say 'yes.' This means the framework will tell the driver to
>> > > perform
>> > > >>> its management activities. This then means the driver will call
>> into
>> > > the
>> > > >>> host (it doesn't know which hypervisor, by the way) to perform the
>> > > >> activity
>> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
>> > > >>>
>> > > >>> The driver doesn't know which hypervisor it's talking to, it just
>> > > sends a
>> > > >>> message to the host to perform the necessary pre-attach work.
>> > > >>
>> > > >> Could we just expose a method like "attachVolume/dettachVolume" on
>> > the
>> > > >> PrimaryDataStoreDriver driver?
>> > > >> For most of cases, the implementation of each driver would just
>> send
>> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
>> > the
>> > > >> implementation in a base class, so that can be shared by all of
>> this
>> > > >> drivers), and in each hypervisor resource code, which may just call
>> > > >> hypervisor's api to attach the volume to VM, while for certain
>> storage,
>> > > >> like SolidFire, may need to create a SR first, and create volume
>> on it,
>> > > >> then call hypervisor's API to attach volume to VM.
>> > > >> While for some other storage vendor, may want to bypass hypervisor
>> > > during
>> > > >> attaching volume, so inside driver's attachvolume implementation,
>> the
>> > > >> driver can do some magic, such as, directly talk to an agent
>> inside VM
>> > > >> instance, then create a disk inside VM.
>> > > >>
>> > > >> How do you guys think?
>> > > >>
>> > > >>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
>> > > wrote:
>> > > >>>
>> > > >>>>
>> > > >>>>
>> > > >>>>> -----Original Message-----
>> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
>> > > >>>>> To: dev@cloudstack.apache.org
>> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> > > >>>>>
>> > > >>>>> Hi John,
>> > > >>>>>
>> > > >>>>> How's about this:
>> > > >>>>>
>> > > >>>>> The driver can implement an isManaged() method. The
>> > > >>> VolumeManagerImpl
>> > > >>>>> can
>> > > >>>>> call into the driver to see if its managed. If it is, the
>> > > >>>> VolumeManagerImpl
>> > > >>>>> (which is responsible for calling into the hypervisor to attach
>> the
>> > > >> disk)
>> > > >>>>> can call into the hypervisor to create the necessary hypervisor
>> data
>> > > >>>>> structure (ex. for XenServer, a storage repository).
>> > > >>>>
>> > > >>>> The problem here is that storage vendor may work differently with
>> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while
>> maybe
>> > > >> other
>> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
>> > directly
>> > > >> to
>> > > >>>> VM instance, see the discuss(
>> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>> > > >>>
>> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>> > > >>> %3E
>> > > >>>> ).
>> > > >>>> So I would let storage provider to implement attach disk to VM,
>> > > >> instead of
>> > > >>>> implemented by cloudstack itself.
>> > > >>>>
>> > > >>>>
>> > > >>>>>
>> > > >>>>> If that's what you're going for, that works for me. By the way,
>> > > >> Edison's
>> > > >>>>> default storage plug-in (which handles the default storage
>> behavior
>> > > >> in
>> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that
>> talks to
>> > > >>>>> hypervisors. You might want to contact him and inform him of
>> your
>> > > >>>> concerns
>> > > >>>>> or that logic (as is) will make it to production.
>> > > >>>>>
>> > > >>>>> Please let me know if what I wrote in above (for my solution)
>> is OK
>> > > >> with
>> > > >>>>> you. :)
>> > > >>>>>
>> > > >>>>> Thanks!
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <
>> jburwell@basho.com>
>> > > >>> wrote:
>> > > >>>>>
>> > > >>>>>> Mike,
>> > > >>>>>>
>> > > >>>>>> Please see my responses in-line below.
>> > > >>>>>>
>> > > >>>>>> Thanks,
>> > > >>>>>> -John
>> > > >>>>>>
>> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>> > > >>>> mike.tutkowski@solidfire.com>
>> > > >>>>>> wrote:
>> > > >>>>>>
>> > > >>>>>>> Hey John,
>> > > >>>>>>>
>> > > >>>>>>> I still have a bit more testing I'd like to do before I build
>> up
>> > > >> a
>> > > >>>> patch
>> > > >>>>>>> file, but this is the gist of what I've done:
>> > > >>>>>>>
>> > > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
>> > tells
>> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
>> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if
>> it
>> > > >>>>> managed.
>> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
>> > > >> whatever
>> > > >>>>>>> activity is required. In my case, this includes sending a
>> > > >> message to
>> > > >>>> the
>> > > >>>>>>> host where the VM is running to have, say XenServer, add a
>> > > >> storage
>> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
>> > > >> SAN
>> > > >>>>>> volume,
>> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
>> can
>> > > >> on
>> > > >>>> the
>> > > >>>>>>> storage repository). After this, the normal attach-volume
>> > > >> message is
>> > > >>>> sent
>> > > >>>>>>> to the host by VolumeManagerImpl.
>> > > >>>>>>
>> > > >>>>>> There should be **no** code from a storage driver to a
>> hypervisor.
>> > > >> I
>> > > >>>>>> apologize for the repetition, but we simply can not have
>> hypervisor
>> > > >>>>>> specific code in the storage layer. The circular dependencies
>> > > >> between
>> > > >>>> the
>> > > >>>>>> two layers are not sustainable in the long term. Either the
>> > > >>>> VirtualManager
>> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>> > > >>> coordinate
>> > > >>>>>> volume creation and then populating the SR. Ideally, we can
>> > > >>>> generalize the
>> > > >>>>>> process flow for attaching volumes such that the Xen hypervisor
>> > > >> plugin
>> > > >>>>>> would only implement callbacks to perform the attach action and
>> > > >> create
>> > > >>>>> the
>> > > >>>>>> structure and SR. To my mind, the SolidFire driver should
>> only be
>> > > >>>>>> allocating space and providing information about contents (e.g.
>> > > >> space
>> > > >>>>>> available, space consumed, streams to a URI, file handle for a
>> URI,
>> > > >>>> etc)
>> > > >>>>>> and capabilities.
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>>>
>> > > >>>>>>> * The reverse is performed for a detach-volume command.
>> > > >>>>>>>
>> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
>> driver.
>> > > >>>>>> Edison's
>> > > >>>>>>> default driver simply does a "return false;". We could add a
>> new
>> > > >>>>>> parameter
>> > > >>>>>>> to the createStoragePool API command, if we want, to remove
>> > the
>> > > >>>>>> hard-coded
>> > > >>>>>>> return values in the drivers (although my driver will probably
>> > > >> just
>> > > >>>>>> ignore
>> > > >>>>>>> this parameter and always return true since it wouldn't make
>> > > >> sense
>> > > >>>> for it
>> > > >>>>>>> to ever return false). We'd need another column in the
>> > > >> storage_pool
>> > > >>>>> table
>> > > >>>>>>> to store this value.
>> > > >>>>>>
>> > > >>>>>> Yes, I think we should have a parameter added to the
>> > > >>> createStoragePool
>> > > >>>>>> surfaced to the HTTP API that allows DataStores to be
>> configured
>> > > >> for
>> > > >>>>>> management when their underlying drivers support it. To
>> simplify
>> > > >>>> things,
>> > > >>>>>> this flag should only be mutable when the DataStore is
>> created. It
>> > > >>>> would be
>> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
>> > > >>>> creation.
>> > > >>>>>>
>> > > >>>>>>>
>> > > >>>>>>> Sound like I'm in sync with what you were thinking?
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>> > > >>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>> wrote:
>> > > >>>>>>>
>> > > >>>>>>>> I agree, John. Just wanted to point out that I have a working
>> > > >> GUI
>> > > >>>> for
>> > > >>>>>> you
>> > > >>>>>>>> to review (in that document), if you'd like to check it out.
>> > > >>>>>>>>
>> > > >>>>>>>> Thanks!
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>> > > >> jburwell@basho.com>
>> > > >>>>>> wrote:
>> > > >>>>>>>>
>> > > >>>>>>>>> Mike,
>> > > >>>>>>>>>
>> > > >>>>>>>>> I would like the UIs of two features reviewed together to
>> > > >> ensure
>> > > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
>> > > >> and
>> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
>> > > >>>> confusion,
>> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
>> > > >> help
>> > > >>>>>>>>> minimize any potential confusion.
>> > > >>>>>>>>>
>> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
>> itself
>> > > >> if
>> > > >>>> it
>> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
>> > > >>>> throttled
>> > > >>>>>>>>> IOPs or storage provisioned IOPs.
>> > > >>>>>>>>>
>> > > >>>>>>>>> Thanks,
>> > > >>>>>>>>> -John
>> > > >>>>>>>>>
>> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>> > > >>>>>>>>> <mi...@solidfire.com> wrote:
>> > > >>>>>>>>>
>> > > >>>>>>>>>> Hi John,
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
>> > to
>> > > >>> you
>> > > >>>>> the
>> > > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
>> > > >> IOPS.
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
>> > > >> and we
>> > > >>>>>>>>> decided
>> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
>> > Offering
>> > > >>>> dialog.
>> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
>> > should
>> > > >>> be
>> > > >>>>> able
>> > > >>>>>> to
>> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only
>> use
>> > > >> Max
>> > > >>>>>> IOPS).
>> > > >>>>>>>>>> These fields are optional and can be ignored for storage
>> that
>> > > >>>> does not
>> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field,
>> the
>> > > >>>> admin can
>> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
>> > > >> IOPS.
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
>> > > >> mine,
>> > > >>>> as
>> > > >>>>>> is
>> > > >>>>>>>>>> decided by the admin).
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> I'm not sure what we can do about these two features
>> > > >> expressing
>> > > >>>> the
>> > > >>>>>>>>> speed
>> > > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS
>> for
>> > > >>>> QoS in
>> > > >>>>>>>>> any
>> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
>> read/write
>> > > >>>> IOPS).
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> Thanks
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>> > > >>> <jb...@basho.com>
>> > > >>>>>>>>> wrote:
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
>> > > >>> Storage/Hypervisor
>> > > >>>>>> layer
>> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
>> > > >>>> hypervisor
>> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
>> > > >> following
>> > > >>>>>> issues
>> > > >>>>>>>>>>> outstanding:
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>> > > >>> read/write
>> > > >>>>>>>>> values
>> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
>> > > >> IOPS.
>> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
>> > > >>> storage
>> > > >>>>>> volumes
>> > > >>>>>>>>>>> with provisioned IOPS?
>> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
>> > > >>>> provisioned
>> > > >>>>>> IOPS
>> > > >>>>>>>>>>> that will permit providers to discriminate these
>> guaranteed
>> > > >>>>>> operations
>> > > >>>>>>>>> in
>> > > >>>>>>>>>>> the event they want to bill for it differently?
>> > > >>>>>>>>>>> 4. What is the user experience for throttled and
>> provisioned
>> > > >>> IOPS
>> > > >>>>>> that
>> > > >>>>>>>>>>> minimizes confusion of these concepts?
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> My thinking is that a VM can have either utilize
>> hypervisor
>> > > >>>> throttled
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
>> solve
>> > > >>>> items 1
>> > > >>>>>>>>> and 2.
>> > > >>>>>>>>>>> Since the two facilities would not be permitted to operate
>> > > >>>> together,
>> > > >>>>>>>>> they
>> > > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
>> > > >> can be
>> > > >>>>>>>>> resolved
>> > > >>>>>>>>>>> with an additional flag or two on the usage records. As
>> for
>> > > >>>> Item 4,
>> > > >>>>>> I
>> > > >>>>>>>>> am
>> > > >>>>>>>>>>> not familiar with how these two enhancements are
>> > depicted in
>> > > >>> the
>> > > >>>>> user
>> > > >>>>>>>>>>> interface. I think we need to review the UI enhancements
>> > for
>> > > >>>> both
>> > > >>>>>>>>>>> enhancements and ensure they are consistent.
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> Do these solutions make sense?
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>> -John
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>> > <us...@gmail.com>
>> > > >>>>> wrote:
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>> John and Mike,
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
>> > 2780/2729,
>> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>> > > >>> CLOUDSTACK-
>> > > >>>>> 2875 ) this
>> > > >>>>>>>>> week.
>> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
>> > > >> ask
>> > > >>>> for
>> > > >>>>>>>>> second
>> > > >>>>>>>>>>>> merge request after testing.
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
>> John,
>> > > >> and
>> > > >>>>> have
>> > > >>>>>>>>> been
>> > > >>>>>>>>>>>>> re-working this section of code today.
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> Thanks for the comments,
>> > > >>>>>>>>>>>>> Mike
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>> > > >>>>> <jb...@basho.com>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> See my responses in-line below.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks for the response.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
>> > whether or
>> > > >>>>> not a
>> > > >>>>>>>>> Volume
>> > > >>>>>>>>>>> is
>> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>> > > >>> actual
>> > > >>>>>>>>> isManaged
>> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
>> > > >> resources
>> > > >>>> for
>> > > >>>>>> the
>> > > >>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>> if it is managed."
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
>> > method
>> > > >>>>> added to
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
>> > > >> then the
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
>> > > >>> method
>> > > >>>>> (which
>> > > >>>>>>>>>>> would
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
>> > and
>> > > >>> this
>> > > >>>>>> would
>> > > >>>>>>>>>>> call
>> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
>> > on
>> > > >>>>>>>>> XenServer,
>> > > >>>>>>>>>>> an
>> > > >>>>>>>>>>>>> SR?
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>> > > >>>>> DataStoreDriver.
>> > > >>>>>>>>> Since
>> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
>> > I
>> > > >>>> would
>> > > >>>>>> add
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
>> > straw
>> > > >>> man
>> > > >>>>> term
>> > > >>>>>>>>> --
>> > > >>>>>>>>>>>>>> something other than volume) methods to
>> > > >>>>>>>>>>> allocate/create/deallocate/delete
>> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
>> > > >>> property
>> > > >>>>> of
>> > > >>>>>>>>>>>>> DataStore
>> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
>> > > >>> DataStoreDriver
>> > > >>>>> can be
>> > > >>>>>>>>>>>>> managed.
>> > > >>>>>>>>>>>>>> This approach allows operators to override
>> > manageability
>> > > >> of
>> > > >>>>>> devices.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
>> > > >> plugin
>> > > >>>>>> would
>> > > >>>>>>>>> be
>> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
>> > to
>> > > >>>>> create any
>> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There
>> should
>> > > >> be no
>> > > >>>>>>>>>>>>> dependencies
>> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
>> > > >> earlier,
>> > > >>>> such
>> > > >>>>>>>>>>>>> circular
>> > > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
>> > mess.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Thanks again, John
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>> > > >>>>> <jb...@basho.com>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
>> > that
>> > > >>>>>> supports n
>> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
>> > > >> hypervisors.
>> > > >>>>>> Such
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
>> > > >>> beast.
>> > > >>>>>>>>>>>>> Therefore, my
>> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
>> > the
>> > > >>>>> Storage
>> > > >>>>>> layer
>> > > >>>>>>>>>>>>> towards
>> > > >>>>>>>>>>>>>>> this separation.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
>> > whether
>> > > >>> or
>> > > >>>>> not a
>> > > >>>>>>>>> Volume
>> > > >>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
>> > an
>> > > >>> actual
>> > > >>>>>>>>>>> isManaged
>> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
>> > > >> resources
>> > > >>>> for
>> > > >>>>>> the
>> > > >>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver
>> needs
>> > > >> to
>> > > >>>>>> indicate
>> > > >>>>>>>>>>>>> whether
>> > > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
>> > > >> think
>> > > >>>> we
>> > > >>>>>>>>> need to
>> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to
>> have
>> > > >>>>>> something
>> > > >>>>>>>>>>>>> that is
>> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
>> > > >>> managing
>> > > >>>>> the
>> > > >>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
>> > unmanaged).
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
>> > in
>> > > >>> the
>> > > >>>>> Xen
>> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
>> > Storage
>> > > >>>>> layer to
>> > > >>>>>>>>>>> perform
>> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
>> > > >> file,
>> > > >>>> etc).
>> > > >>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>> > > >>>> hypervisor, as
>> > > >>>>>>>>> such,
>> > > >>>>>>>>>>>>>>> should be confined to its implementation. From my
>> > > >>>>> perspective,
>> > > >>>>>>>>>>> nothing
>> > > >>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
>> > From
>> > > >>> its
>> > > >>>>>>>>>>>>> perspective,
>> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
>> > to
>> > > >>>> query
>> > > >>>>>> the
>> > > >>>>>>>>>>> data
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
>> > level
>> > > >>>>> layers
>> > > >>>>>>>>> (e.g.
>> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
>> > composition
>> > > >>> of
>> > > >>>>>>>>>>> operations
>> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
>> > Volume
>> > > >>>>> storage
>> > > >>>>>>>>>>> protocol
>> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
>> > implemented
>> > > >> in
>> > > >>>> my
>> > > >>>>>>>>> patch,
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>> could do the following:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>> > > >>>>>>>>> VolumeManagerImpl
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
>> > > >>>>> hypervisor.
>> > > >>>>>> Before
>> > > >>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
>> > dealing
>> > > >>> with
>> > > >>>>>>>>> Dynamic
>> > > >>>>>>>>>>> (or
>> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
>> send a
>> > > >>>> "Create
>> > > >>>>>>>>> SR"
>> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
>> > would
>> > > >>> then
>> > > >>>>>>>>> proceed
>> > > >>>>>>>>>>> to
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
>> > > >> add
>> > > >>>>>> another
>> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
>> > > >> particular
>> > > >>>>> type
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>> storage.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> What do you think?
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> Hey John,
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
>> > outlines
>> > > >>> how
>> > > >>>>> the
>> > > >>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be
>> of
>> > > >> use
>> > > >>>> to
>> > > >>>>>>>>> you, as
>> > > >>>>>>>>>>>>> well,
>> > > >>>>>>>>>>>>>>>> as you perform the code review.
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> I have shared this document with you (you should
>> > have
>> > > >>>>> received
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> Talk to you later!
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>> > > >>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
>> > thoughts,
>> > > >>> but
>> > > >>>>>> want
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the
>> solidfire,
>> > > >>>>>>>>>>>>> disk_io_throttle,
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
>> > > >>>>> coherent
>> > > >>>>>>>>> form
>> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
>> > the
>> > > >>>> notion
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
>> > > >> the
>> > > >>>>> notion
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
>> > was
>> > > >>>>>>>>> incomplete,
>> > > >>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>> I had
>> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
>> > > >> device.
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
>> > > >> this.
>> > > >>>> :)
>> > > >>>>> Do
>> > > >>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>> have a
>> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
>> > > >>> what
>> > > >>>>> you're
>> > > >>>>>>>>> looking
>> > > >>>>>>>>>>>>>>>>>> for?
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>> > > >>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
>> > the
>> > > >>>> notion
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
>> > > >>>>> capabilities
>> > > >>>>>>>>> (i.e.
>> > > >>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
>> > > >>> managed
>> > > >>>>> by
>> > > >>>>>>>>> CloudStack)
>> > > >>>>>>>>>>>>>>>>>> from the
>> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
>> > a
>> > > >>>>> semantic
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
>> > I
>> > > >>> also
>> > > >>>>> want
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
>> > the
>> > > >>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> definitions.
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>> > > >>>>>>>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
>> > questions
>> > > >>> for
>> > > >>>>> me
>> > > >>>>>> with
>> > > >>>>>>>>>>>>>>>>>> regards to
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
>> > and
>> > > >>> SAN
>> > > >>>>>> volumes?
>> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
>> > (LUN
>> > > >>> 0).
>> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
>> > and
>> > > >>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> Volumes?
>> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
>> > > >>> server or
>> > > >>>>>>>>> externally
>> > > >>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> operator?
>> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
>> > > >>> volume
>> > > >>>>> (same
>> > > >>>>>> as a
>> > > >>>>>>>>>>> SAN
>> > > >>>>>>>>>>>>>>>>>> LUN) is
>> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
>> > plug-in)
>> > > >>>>> the
>> > > >>>>>> first
>> > > >>>>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
>> > but
>> > > >>>> still
>> > > >>>>>>>>> want
>> > > >>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> use a
>> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
>> > > >>> today
>> > > >>>>>> (prior
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> 4.2). The
>> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
>> > in
>> > > >>> this
>> > > >>>>> case -
>> > > >>>>>>>>>>>>>>>>>> multiple VMs
>> > > >>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
>> > you
>> > > >>> can do
>> > > >>>>> this
>> > > >>>>>>>>>>>>> today,
>> > > >>>>>>>>>>>>>>>>>> it is
>> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
>> > server
>> > > >>> or
>> > > >>>>> externally
>> > > >>>>>>>>> by
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> operator?
>> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
>> > > >>> volumes
>> > > >>>>> are
>> > > >>>>>>>>>>>>> completely
>> > > >>>>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
>> > There
>> > > >>> is
>> > > >>>>> no
>> > > >>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>> interaction.
>> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
>> > > >>> volume
>> > > >>>>> and a
>> > > >>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
>> > > >>>>> supports
>> > > >>>>>> true
>> > > >>>>>>>>>>> QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
>> > > >> related
>> > > >>>>> and
>> > > >>>>>>>>>>>>> expected
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> operate.
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
>> > multiple
>> > > >>> VMs
>> > > >>>>> with
>> > > >>>>>>>>>>>>>>>>>> provisioned
>> > > >>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
>> > > >>> single
>> > > >>>>> SAN
>> > > >>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>> (LUN) to a
>> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
>> > > >>> multiple
>> > > >>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
>> > guarantee
>> > > >>> QoS.
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
>> > volume
>> > > >>> house
>> > > >>>>> more
>> > > >>>>>>>>> than
>> > > >>>>>>>>>>>>> one
>> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
>> > > >>> without
>> > > >>>>> any of
>> > > >>>>>> my
>> > > >>>>>>>>>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>>>> code.
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
>> > Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
>> > volume
>> > > >>> for
>> > > >>>>>>>>>>> CloudStack's
>> > > >>>>>>>>>>>>>>>>>> use
>> > > >>>>>>>>>>>>>>>>>>>> onto
>> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
>> > Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
>> > concerns me
>> > > >>> --
>> > > >>>>>> concepts
>> > > >>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
>> > Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
>> > today
>> > > >>>> only
>> > > >>>>>> used
>> > > >>>>>>>>>>> (as
>> > > >>>>>>>>>>>>>>>>>> far as
>> > > >>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
>> > not
>> > > >>>>> PreSetup an
>> > > >>>>>>>>> SR).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
>> > from
>> > > >>>>> VMware, it
>> > > >>>>>>>>> uses
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> vmfs
>> > > >>>>>>>>>>>>>>>>>>>>>> type.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
>> > from
>> > > >>> KVM,
>> > > >>>>> it uses
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
>> > > >>> here
>> > > >>>>> was to
>> > > >>>>>>>>> make a
>> > > >>>>>>>>>>>>>>>>>> new type
>> > > >>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
>> > > >>> Edison
>> > > >>>>> added
>> > > >>>>>>>>> into
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
>> > > >>> specificy,
>> > > >>>>> though:
>> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
>> > > >>> Tutkowski
>> > > >>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
>> > know
>> > > >>> or
>> > > >>>>> care
>> > > >>>>>> that
>> > > >>>>>>>>> it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>> used
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
>> > > >>>>> questions
>> > > >>>>>>>>> about
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
>> > > >>>>> SolidFire
>> > > >>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>> needs to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
>> > > >>> hypervisor
>> > > >>>>> logic
>> > > >>>>>>>>>>> knows
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>> treat
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
>> > constraint
>> > > >>>>> and just
>> > > >>>>>>>>>>>>>>>>>> calling it
>> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
>> > for
>> > > >>> the
>> > > >>>>> hypervisor
>> > > >>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
>> > perform
>> > > >>> the
>> > > >>>>> necessary
>> > > >>>>>>>>>>>>>>>>>> activities.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
>> > > >>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>> > > >>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
>> > able to
>> > > >>>>> manage a
>> > > >>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>> > > >>> CloudStack
>> > > >>>>> (e.g. the
>> > > >>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>> shared
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
>> > > >>> chosen
>> > > >>>>> to
>> > > >>>>>>>>> dedicate
>> > > >>>>>>>>>>>>>>>>>> only a
>> > > >>>>>>>>>>>>>>>>>>>>>> portion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
>> > > >>> make
>> > > >>>>> sense?"
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
>> > SAN
>> > > >>>>> volume via
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
>> > would
>> > > >>>>> work unless
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
>> > an
>> > > >>> SR.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
>> > > >>>>> CloudStack, an
>> > > >>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>> allocates
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
>> > > >>> exposes it
>> > > >>>>>> through
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> LUN. They
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
>> > to
>> > > >>>>> CloudStack,
>> > > >>>>>>>>> but
>> > > >>>>>>>>>>>>>>>>>> not allow
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
>> > > >>>>> changes, we
>> > > >>>>>>>>> would
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>> passing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
>> > > >>>>> necessary
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>> (like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
>> > have
>> > > >>> the
>> > > >>>>> storage
>> > > >>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>> itself
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
>> > support
>> > > >>> from
>> > > >>>>> the
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> side and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
>> > to
>> > > >>> create
>> > > >>>>> the
>> > > >>>>>>>>> SR
>> > > >>>>>>>>>>>>>>>>>> before the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
>> > plugin
>> > > >>>>> should
>> > > >>>>>> have a
>> > > >>>>>>>>>>>>> rich
>> > > >>>>>>>>>>>>>>>>>> enough
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
>> > > >>> available
>> > > >>>>> for
>> > > >>>>>> volume
>> > > >>>>>>>>>>>>>>>>>> storage.
>> > > >>>>>>>>>>>>>>>>>>>> For
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>> > > >>> interrogation of
>> > > >>>>> the
>> > > >>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
>> > itself
>> > > >>>>> shouldn't
>> > > >>>>>>>>> know
>> > > >>>>>>>>>>>>>>>>>> or care
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
>> > able
>> > > >>> to
>> > > >>>>>> answer
>> > > >>>>>>>>>>>>>>>>>> questions
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
>> > > >>> driver
>> > > >>>>> (which
>> > > >>>>>>>>> is a
>> > > >>>>>>>>>>>>>>>>>> type of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
>> > done, a
>> > > >>> new
>> > > >>>>> Primary
>> > > >>>>>>>>>>>>>>>>>> Storage shows
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
>> > > >>> that
>> > > >>>>>> references
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>> tag
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
>> > CloudStack
>> > > >>>>> volume. This
>> > > >>>>>>>>> leads
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> a new
>> > > >>>>>>>>>>>>>>>>>>>>>> row
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
>> > CloudStack
>> > > >>>>> volume to a
>> > > >>>>>>>>> VM
>> > > >>>>>>>>>>>>>>>>>> (attach
>> > > >>>>>>>>>>>>>>>>>>>>>> disk).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
>> > calling
>> > > >>> the
>> > > >>>>> plug-in
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> create a
>> > > >>>>>>>>>>>>>>>>>>>> new
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
>> > > >>> SAN).
>> > > >>>>> The
>> > > >>>>>>>>> plug-in
>> > > >>>>>>>>>>>>>>>>>> also
>> > > >>>>>>>>>>>>>>>>>>>>>> updates
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
>> > data
>> > > >>>>> (like the
>> > > >>>>>>>>> IQN of
>> > > >>>>>>>>>>>>>>>>>> the SAN
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
>> > the
>> > > >>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> volume is in
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
>> > the
>> > > >>>>> volume
>> > > >>>>>>>>> will be
>> > > >>>>>>>>>>>>>>>>>> in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
>> > the
>> > > >>>>> plug-in
>> > > >>>>>>>>> code
>> > > >>>>>>>>>>>>>>>>>> will not
>> > > >>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
>> > and
>> > > >>>>> detects
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
>> > form
>> > > >>> of a
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
>> > > >>>>> terminology,
>> > > >>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>> in these
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
>> > > >>>>> allocation on
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
>> > > >>> mean
>> > > >>>>> 'SAN
>> > > >>>>>>>>> volume'.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
>> > user
>> > > >>> can
>> > > >>>>> make in
>> > > >>>>>>>>>>>>>>>>>> CloudStack, I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
>> > well
>> > > >>> make
>> > > >>>>> sense, but
>> > > >>>>>>>>> I'm
>> > > >>>>>>>>>>>>>>>>>> having a
>> > > >>>>>>>>>>>>>>>>>>>>>> hard
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
>> > in on
>> > > >>>>> this
>> > > >>>>>>>>> conversation
>> > > >>>>>>>>>>>>>>>>>> as he
>> > > >>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>> > > >>> approach I
>> > > >>>>> took.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
>> > > >>> John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
>> > two
>> > > >>>>> distinct
>> > > >>>>>>>>> concepts
>> > > >>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
>> > protocols.
>> > > >>> In
>> > > >>>>> both
>> > > >>>>>>>>> cases, we
>> > > >>>>>>>>>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
>> > system to
>> > > >>>>>> create/delete
>> > > >>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
>> > the
>> > > >>>>> volume to
>> > > >>>>>> be
>> > > >>>>>>>>>>>>>>>>>> volume to be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
>> > context.
>> > > >>> To
>> > > >>>>> ensure
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>> are in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
>> > > >>>>> definitions,
>> > > >>>>>>>>> refers to
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>> physical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
>> > correct?
>> > > >>>>> Minimally,
>> > > >>>>>> we
>> > > >>>>>>>>>>>>> must
>> > > >>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>> able
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
>> > device
>> > > >>> to
>> > > >>>>> move bits
>> > > >>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>> one place
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>> > > >>> Optionally, a
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
>> > Therefore,
>> > > >>>>> we can have
>> > > >>>>>> a
>> > > >>>>>>>>>>>>>>>>>> unmanaged
>> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
>> > and
>> > > >>> we
>> > > >>>>> can have a
>> > > >>>>>>>>>>>>> managed
>> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
>> > > >>> capable
>> > > >>>>> of
>> > > >>>>>>>>> allocating
>> > > >>>>>>>>>>>>>>>>>> LUNs and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
>> > CloudStack
>> > > >>> may
>> > > >>>>> be able
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> manage a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
>> > > >>>>> unmanaged by
>> > > >>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>> (e.g.
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
>> > the
>> > > >>>>> operator has
>> > > >>>>>>>>> chosen
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
>> > Does
>> > > >>> my
>> > > >>>>>> reasoning
>> > > >>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>> sense?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
>> > are
>> > > >>>>> reasonable, it
>> > > >>>>>>>>> seems
>> > > >>>>>>>>>>>>>>>>>>>> appropriate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
>> > concerns
>> > > >>> from
>> > > >>>>>>>>> StoragePoolType,
>> > > >>>>>>>>>>>>>>>>>> add the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
>> > attached
>> > > >>> driver
>> > > >>>>> that
>> > > >>>>>>>>>>> indicates
>> > > >>>>>>>>>>>>>>>>>> whether
>> > > >>>>>>>>>>>>>>>>>>>>>> or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
>> > > >>> establish
>> > > >>>>> a
>> > > >>>>>>>>> abstraction
>> > > >>>>>>>>>>>>>>>>>>>>>> representing a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
>> > that
>> > > >>> is
>> > > >>>>>>>>> associated
>> > > >>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>> volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
>> > hypervisor
>> > > >>>>> drivers can
>> > > >>>>>>>>>>>>> declare
>> > > >>>>>>>>>>>>>>>>>> which
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
>> > encounter
>> > > >>> a
>> > > >>>>> device
>> > > >>>>>> managed
>> > > >>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
>> > exposed
>> > > >>> by
>> > > >>>>> the driver to
>> > > >>>>>>>>>>>>> automate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
>> > > >>> sense,
>> > > >>>>> then we can
>> > > >>>>>>>>> sit
>> > > >>>>>>>>>>>>>>>>>> down and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
>> > Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
>> > between
>> > > >>> the
>> > > >>>>> current
>> > > >>>>>> iSCSI
>> > > >>>>>>>>>>>>> type
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
>> > go in
>> > > >>> and
>> > > >>>>> create
>> > > >>>>>> a
>> > > >>>>>>>>>>>>>>>>>> Primary
>> > > >>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
>> > time,
>> > > >>>>> the
>> > > >>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>> volume must
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
>> > allocated).
>> > > >>>>> Future
>> > > >>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
>> > created
>> > > >>>>> behind the
>> > > >>>>>>>>>>>>> scenes.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
>> > to
>> > > >>> go in
>> > > >>>>> and
>> > > >>>>>> create
>> > > >>>>>>>>>>>>>>>>>> Primary
>> > > >>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
>> > delete
>> > > >>>>> volumes on
>> > > >>>>>>>>> its
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
>> > the
>> > > >>>>> storage
>> > > >>>>>>>>>>>>>>>>>> framework). When
>> > > >>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
>> > that
>> > > >>> was
>> > > >>>>> created,
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>>>> tells
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
>> > volume.
>> > > >>> After
>> > > >>>>> this is
>> > > >>>>>>>>> done,
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
>> > called.
>> > > >>>>> No
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
>> > volume
>> > > >>> was
>> > > >>>>> just
>> > > >>>>>>>>> created.
>> > > >>>>>>>>>>>>> The
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
>> > PM,
>> > > >>> Mike
>> > > >>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
>> > should
>> > > >>>>> probably
>> > > >>>>>> have
>> > > >>>>>>>>>>>>>>>>>> defined them
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
>> > > >>> pre-
>> > > >>>>> allocated
>> > > >>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>> an admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
>> > > >>> hypervisor
>> > > >>>>> data
>> > > >>>>>>>>> structure
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
>> > SR),
>> > > >>> then
>> > > >>>>> that
>> > > >>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
>> > Disks
>> > > >>> (VDI)
>> > > >>>>> are later
>> > > >>>>>>>>>>>>> placed
>> > > >>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
>> > > >>> these
>> > > >>>>> cases, the
>> > > >>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
>> > > >>> already
>> > > >>>>> in place
>> > > >>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> simply
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
>> > structure
>> > > >>> to
>> > > >>>>> the VM in
>> > > >>>>>>>>>>>>>>>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
>> > that is
>> > > >>> not
>> > > >>>>>>>>>>> pre-allocated.
>> > > >>>>>>>>>>>>>>>>>> Instead
>> > > >>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
>> > could
>> > > >>> be a
>> > > >>>>> SAN
>> > > >>>>>> (not
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> volume on
>> > > >>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
>> > data
>> > > >>>>> structure
>> > > >>>>>>>>> must be
>> > > >>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
>> > because
>> > > >>> these
>> > > >>>>> types of
>> > > >>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>> have not
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
>> > > >>>>> structure by an
>> > > >>>>>>>>>>> admin.
>> > > >>>>>>>>>>>>>>>>>> Once
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
>> > XenServer
>> > > >>> for
>> > > >>>>> this
>> > > >>>>>>>>> volume,
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> attaches
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
>> > VM
>> > > >>> in
>> > > >>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
>> > PM,
>> > > >>>>> John Burwell
>> > > >>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
>> > implementation of
>> > > >>> the
>> > > >>>>> Dynamic
>> > > >>>>>> type
>> > > >>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>> behavior
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
>> > why I
>> > > >>> ask
>> > > >>>>> about the
>> > > >>>>>>>>>>>>>>>>>> difference.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
>> > the
>> > > >>>>> definition
>> > > >>>>>> of
>> > > >>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>> Dynamic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
>> > > >>> Mike
>> > > >>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
>> > type
>> > > >>> is
>> > > >>>>>> uniquely
>> > > >>>>>>>>>>> used
>> > > >>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
>> > > >>> Storage
>> > > >>>>> that is
>> > > >>>>>>>>> directly
>> > > >>>>>>>>>>>>>>>>>> based on
>> > > >>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
>> > > >>> step of
>> > > >>>>> going
>> > > >>>>>> to
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
>> > on
>> > > >>> that
>> > > >>>>> iSCSI
>> > > >>>>>>>>> target as
>> > > >>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
>> > > >>>>> supported for
>> > > >>>>>>>>>>>>>>>>>> XenServer.
>> > > >>>>>>>>>>>>>>>>>>>> For
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
>> > first go
>> > > >>> to
>> > > >>>>> the
>> > > >>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>> perform
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
>> > RBD is.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> > 2:13
>> > > >>> PM,
>> > > >>>>> John
>> > > >>>>>> Burwell
>> > > >>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
>> > code,
>> > > >>> what
>> > > >>>>> is the
>> > > >>>>>>>>>>> difference
>> > > >>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
>> > isn't
>> > > >>> RBD
>> > > >>>>> considered
>> > > >>>>>>>>>>>>> Dynamic?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
>> > PM,
>> > > >>> Mike
>> > > >>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
>> > storage is
>> > > >>>>> defined in
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>> > > >>>>> StoragePoolType {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
>> > local
>> > > >>>>> directory
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > NetworkFilesystem(true),
>> > > >>> //
>> > > >>>>> NFS or CIFS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
>> > shared
>> > > >>> LUN,
>> > > >>>>> with a
>> > > >>>>>>>>>>> clusterfs
>> > > >>>>>>>>>>>>>>>>>>>> overlay
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
>> > ZFS
>> > > >>>>> Comstar
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
>> > image
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
>> > XenServer
>> > > >>> local
>> > > >>>>> LVM SR
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > SharedMountPoint(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
>> > VMware
>> > > >>> VMFS
>> > > >>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
>> > > >>>>> XenServer, Storage
>> > > >>>>>>>>> Pool
>> > > >>>>>>>>>>>>>>>>>> is set
>> > > >>>>>>>>>>>>>>>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
>> > XenServer
>> > > >>> local
>> > > >>>>> EXT SR
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
>> > dynamic,
>> > > >>>>> zone-wide
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > StoragePoolType(boolean
>> > > >>>>> shared) {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
>> > isShared() {
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> > 1:41
>> > > >>> PM,
>> > > >>>>> Mike
>> > > >>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
>> > > >>> another
>> > > >>>>> storage
>> > > >>>>>>>>>>> company
>> > > >>>>>>>>>>>>>>>>>> wants
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
>> > > >>> Quality
>> > > >>>>> of
>> > > >>>>>> Service
>> > > >>>>>>>>>>>>>>>>>> feature. It
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
>> > well.
>> > > >>>>> They would
>> > > >>>>>>>>> need
>> > > >>>>>>>>>>>>> only
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
>> > the
>> > > >>>>> necessary
>> > > >>>>>>>>> changes to
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
>> > in.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> > at
>> > > >>> 1:39
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
>> > the
>> > > >>>>> XenServer code
>> > > >>>>>> is
>> > > >>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
>> > attach
>> > > >>> logic is
>> > > >>>>> now aware
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>> dynamic,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
>> > > >>>>> implementation of
>> > > >>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>> kind of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
>> > > >>> with
>> > > >>>>> Edison's
>> > > >>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> framework
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
>> > new
>> > > >>>>> framework that
>> > > >>>>>>>>>>> supported
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
>> > dynamically.
>> > > >>>>> However, when I
>> > > >>>>>>>>>>>>>>>>>> visited with
>> > > >>>>>>>>>>>>>>>>>>>>>> him
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
>> > that
>> > > >>> it
>> > > >>>>> was not
>> > > >>>>>>>>>>>>>>>>>> complete. We
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
>> > could
>> > > >>> do
>> > > >>>>> with these
>> > > >>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>> unless
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
>> > recognize
>> > > >>> this
>> > > >>>>> new type of
>> > > >>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>> create
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>> > structure.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> > at
>> > > >>> 1:28
>> > > >>>>> PM, John
>> > > >>>>>>>>> Burwell
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
>> > to me
>> > > >>>>> that any
>> > > >>>>>>>>>>> operation
>> > > >>>>>>>>>>>>>>>>>> in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
>> > > >>> about
>> > > >>>>> details. I
>> > > >>>>>>>>> expect
>> > > >>>>>>>>>>>>>>>>>> to see
>> > > >>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
>> > > >>> operations
>> > > >>>>> that can be
>> > > >>>>>>>>>>>>>>>>>>>> composed/driven
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
>> > > >>>>> allocate
>> > > >>>>>> space/create
>> > > >>>>>>>>>>>>>>>>>> structures
>> > > >>>>>>>>>>>>>>>>>>>>>> per
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
>> > > >>>>> dependency, we are
>> > > >>>>>>>>> going
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> end
>> > > >>>>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
>> > make
>> > > >>> the
>> > > >>>>> system
>> > > >>>>>>>>>>>>> increasingly
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
>> > > >>>>> understanding that the
>> > > >>>>>>>>> Xen
>> > > >>>>>>>>>>>>>>>>>> specific
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>> > > >>>>> CitrixResourceBase
>> > > >>>>>> class?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013 at
>> > > >>> 3:12
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
>> > in a
>> > > >>> bit
>> > > >>>>> more
>> > > >>>>>>>>> detail:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
>> > aside
>> > > >>> from
>> > > >>>>> one setup
>> > > >>>>>>>>>>>>> method
>> > > >>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
>> > > >>> volume on
>> > > >>>>> the
>> > > >>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> system,
>> > > >>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>> go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
>> > > >>> structure
>> > > >>>>> to make
>> > > >>>>>> use
>> > > >>>>>>>>> of
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>> > > >>> XenServer
>> > > >>>>> or a
>> > > >>>>>> datastore
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> ESX(i)).
>> > > >>>>>>>>>>>>>>>>>>>>>> VMs
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
>> > storage
>> > > >>>>> system's volume.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
>> > > >>> storage
>> > > >>>>> framework,
>> > > >>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>> need
>> > > >>>>>>>>>>>>>>>>>>>> no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
>> > > >>> create
>> > > >>>>> a 1:1
>> > > >>>>>>>>>>>>> relationship
>> > > >>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
>> > VM's
>> > > >>> data
>> > > >>>>> disk
>> > > >>>>>>>>> (necessary
>> > > >>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
>> > a
>> > > >>> plug-
>> > > >>>>> in that is
>> > > >>>>>>>>> called
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
>> > needed.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
>> > the
>> > > >>>>> storage
>> > > >>>>>> framework
>> > > >>>>>>>>> did
>> > > >>>>>>>>>>>>>>>>>> not
>> > > >>>>>>>>>>>>>>>>>>>>>> address
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>> > > >>> hypervisor-
>> > > >>>>> specific data
>> > > >>>>>>>>>>>>>>>>>> structure
>> > > >>>>>>>>>>>>>>>>>>>> when
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
>> > case,
>> > > >>> I've
>> > > >>>>> been
>> > > >>>>>>>>> enhancing
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> to do
>> > > >>>>>>>>>>>>>>>>>>>> so.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
>> > > >>>>> submitted. I've got
>> > > >>>>>>>>> ESX(i)
>> > > >>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> my
>> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
>> > 4.2
>> > > >>>>> freeze date.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
>> > bit? :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013 at
>> > > >>>>> 1:03 PM, Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
>> > plug-in -
>> > > >>> by
>> > > >>>>> itself - is
>> > > >>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
>> > the
>> > > >>>>> volume-attach
>> > > >>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>> (in the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
>> > into
>> > > >>> the
>> > > >>>>> plug-in to
>> > > >>>>>>>>> have
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> create a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
>> > > >>> comes to
>> > > >>>>> attach the
>> > > >>>>>>>>> volume
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
>> > > >>> enough
>> > > >>>>> to recognize
>> > > >>>>>>>>> it's
>> > > >>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
>> > volume
>> > > >>> has
>> > > >>>>> just been
>> > > >>>>>>>>> created)
>> > > >>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> create,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
>> > > >>>>> XenServer) or a
>> > > >>>>>>>>> datastore
>> > > >>>>>>>>>>>>>>>>>> (for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
>> > just
>> > > >>>>> created.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
>> > spending
>> > > >>> most
>> > > >>>>> of my time
>> > > >>>>>>>>>>>>> recently
>> > > >>>>>>>>>>>>>>>>>> making
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
>> > code.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
>> > up?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > 2013
>> > > >>> at
>> > > >>>>> 12:48 PM,
>> > > >>>>>> John
>> > > >>>>>>>>>>>>>>>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
>> > > >>> why
>> > > >>>>> the the storage
>> > > >>>>>>>>>>>>> driver
>> > > >>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
>> > at
>> > > >>> 1:21
>> > > >>>>> PM, Mike
>> > > >>>>>>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
>> > ultimately I
>> > > >>>>> would like to
>> > > >>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>> all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
>> > think
>> > > >>> I'm
>> > > >>>>> just out of
>> > > >>>>>>>>> time
>> > > >>>>>>>>>>>>>>>>>> for 4.2
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
>> > > >>> plug-in
>> > > >>>>> supports
>> > > >>>>>>>>>>>>> XenServer.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
>> > > >>> feature
>> > > >>>>> freeze, I
>> > > >>>>>>>>> have
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> working
>> > > >>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
>> > be
>> > > >>> clear,
>> > > >>>>> this is
>> > > >>>>>> all
>> > > >>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> regards
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > Compute
>> > > >>>>> Offerings post
>> > > >>>>>> 4.2.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> > > >>> 2013
>> > > >>>>> at 11:14 AM,
>> > > >>>>>>>>> Kelcey
>> > > >>>>>>>>>>>>>>>>>> Jamison
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
>> > plan
>> > > >>> on
>> > > >>>>> supporting
>> > > >>>>>>>>> KVM in
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
>> > > >>>>> Message -----
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
>> > > >>>>> Tutkowski" <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>> > > >>>>> dev@cloudstack.apache.org
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
>> > Monday,
>> > > >>> June
>> > > >>>>> 3, 2013
>> > > >>>>>>>>> 10:12:32 AM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
>> > > >>> [MERGE]
>> > > >>>>>>>>>>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>> MASTER
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
>> > > >>> merging
>> > > >>>>> Wei's feature
>> > > >>>>>>>>>>> first,
>> > > >>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>> mine.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
>> > is
>> > > >>> for
>> > > >>>>> KVM only,
>> > > >>>>>>>>> then
>> > > >>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> is a
>> > > >>>>>>>>>>>>>>>>>>>> non
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
>> > 3,
>> > > >>> 2013
>> > > >>>>> at 8:55 AM,
>> > > >>>>>> Wei
>> > > >>>>>>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
>> > billing,
>> > > >>> as
>> > > >>>>> no one works
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> billing
>> > > >>>>>>>>>>>>>>>>>>>> now,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
>> > by
>> > > >>>>> themselves. They
>> > > >>>>>>>>> can
>> > > >>>>>>>>>>>>> get
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > disk_offering of
>> > > >>> a
>> > > >>>>> VMs and
>> > > >>>>>> volumes
>> > > >>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
>> > the
>> > > >>>>> exact
>> > > >>>>>> limitation
>> > > >>>>>>>>>>>>> value
>> > > >>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>> > limitation
>> > > >>> for
>> > > >>>>> nics as
>> > > >>>>>> well.
>> > > >>>>>>>>> I
>> > > >>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
>> > I/O
>> > > >>>>> throttling?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
>> > > >>>>> misunstand the second
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
>> > > >>> what
>> > > >>>>> you said about
>> > > >>>>>> the
>> > > >>>>>>>>>>> two
>> > > >>>>>>>>>>>>>>>>>>>> features.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>> > John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>> > 2013,
>> > > >>> at
>> > > >>>>> 2:13 AM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > Mike
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
>> > Mike's
>> > > >>>>> aswer helps you.
>> > > >>>>>>>>> I am
>> > > >>>>>>>>>>>>>>>>>> trying
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
>> > think
>> > > >>> billing
>> > > >>>>> should
>> > > >>>>>>>>> depend
>> > > >>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>>> IO
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > limitation.
>> > > >>> Please
>> > > >>>>> review
>> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
>> > > >>>>>>>>>>>>>>>>>>>> you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
>> > IO
>> > > >>>>> statistics
>> > > >>>>>> including
>> > > >>>>>>>>>>>>>>>>>> bytes/iops
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
>> > > >>> machine.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
>> > the
>> > > >>> AWS
>> > > >>>>> model,
>> > > >>>>>> customers
>> > > >>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>> billed
>> > > >>>>>>>>>>>>>>>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > provisioned
>> > > >>> IOPS,
>> > > >>>>> as well as,
>> > > >>>>>> for
>> > > >>>>>>>>>>>>> those
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://aws.amazon.com/ebs/).
>> > > >>>>>> I
>> > > >>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>> imagine
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
>> > > >>> employ
>> > > >>>>> similar cost
>> > > >>>>>>>>>>> models.
>> > > >>>>>>>>>>>>>>>>>> Could
>> > > >>>>>>>>>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
>> > model
>> > > >>> in
>> > > >>>>> the current
>> > > >>>>>>>>> patch?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
>> > you
>> > > >>> mean
>> > > >>>>> IOPS runtime
>> > > >>>>>>>>>>> change?
>> > > >>>>>>>>>>>>>>>>>> KVM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > limitation for
>> > > >>> a
>> > > >>>>> running
>> > > >>>>>> virtual
>> > > >>>>>>>>>>>>>>>>>> machine
>> > > >>>>>>>>>>>>>>>>>>>>>> through
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > CloudStack
>> > > >>> does
>> > > >>>>> not support
>> > > >>>>>>>>>>> changing
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > (computer
>> > > >>>>> offering or disk
>> > > >>>>>>>>>>>>> offering).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
>> > the
>> > > >>>>> Java interface
>> > > >>>>>>>>> level.
>> > > >>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>> apologize
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
>> > > >>>>> generalize allocation
>> > > >>>>>>>>>>>>>>>>>> algorithms
>> > > >>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
>> > the
>> > > >>>>> service guarantees
>> > > >>>>>>>>>>>>>>>>>> provided by a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
>> > > >>> good
>> > > >>>>> question.
>> > > >>>>>>>>> Maybe it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>> better
>> > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> disk_io_throttling as Mike
>> > > >>>>>>>>> needs to
>> > > >>>>>>>>>>>>>>>>>> consider
>> > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
>> > think.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
>> > expand
>> > > >>> on
>> > > >>>>> my thoughts
>> > > >>>>>> in a
>> > > >>>>>>>>>>>>> later
>> > > >>>>>>>>>>>>>>>>>>>>>> response
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
>> > points
>> > > >>>>> between these two
>> > > >>>>>>>>>>>>>>>>>> features. I
>> > > >>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
>> > to be
>> > > >>>>> merged before
>> > > >>>>>>>>>>>>>>>>>> SolidFire, but
>> > > >>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > coordination
>> > > >>>>> between the
>> > > >>>>>> branches
>> > > >>>>>>>>>>>>>>>>>> (possibly
>> > > >>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> disk_io_throttling)
>> > > >>>>> to
>> > > >>>>>>>>> coordinate on
>> > > >>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>> issue.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>> > > >>> John
>> > > >>>>> Burwell <
>> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > things I
>> > > >>>>> want to
>> > > >>>>>> understand
>> > > >>>>>>>>>>> are
>> > > >>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
>> > there
>> > > >>>>> value in
>> > > >>>>>> capturing
>> > > >>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>> policies
>> > > >>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> > model
>> > > >>> (e.g.
>> > > >>>>> for
>> > > >>>>>>>>> billing/usage
>> > > >>>>>>>>>>>>>>>>>>>> purposes,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
>> > Should
>> > > >>> there
>> > > >>>>> be a common
>> > > >>>>>>>>>>>>>>>>>> interface model
>> > > >>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > provisioning
>> > > >>> at
>> > > >>>>> runtime?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
>> > are
>> > > >>>>> conflicting
>> > > >>>>>>>>> provisioned
>> > > >>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > hypervisor
>> > > >>> and
>> > > >>>>> storage device
>> > > >>>>>>>>>>>>>>>>>> reconciled?
>> > > >>>>>>>>>>>>>>>>>>>> In
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
>> > to
>> > > >>>>> believe (and
>> > > >>>>>> billed)
>> > > >>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>> device
>> > > >>>>> has been
>> > > >>>>>>>>> configured
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > consistent
>> > > >>>>> configuration
>> > > >>>>>>>>> between a
>> > > >>>>>>>>>>>>>>>>>> VM and a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
>> > but a
>> > > >>>>> later
>> > > >>>>>> modification
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> inconsistency.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
>> > 2,
>> > > >>> 2013,
>> > > >>>>> at 8:38 PM,
>> > > >>>>>>>>> Mike
>> > > >>>>>>>>>>>>>>>>>> Tutkowski
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> mike.tutkowski@solidfire.com>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > believe
>> > > >>> Wei's
>> > > >>>>> feature deals
>> > > >>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>> controlling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>> hypervisor
>> > > >>>>> side.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> > feature
>> > > >>> is
>> > > >>>>> focused on
>> > > >>>>>>>>>>>>> controlling
>> > > >>>>>>>>>>>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>>>>>> from
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
>> > that
>> > > >>>>> helps. :)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
>> > Jun
>> > > >>> 2,
>> > > >>>>> 2013 at 6:35
>> > > >>>>>> PM,
>> > > >>>>>>>>>>>>> John
>> > > >>>>>>>>>>>>>>>>>>>> Burwell
>> > > >>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > jburwell@basho.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> > > >>> opinion is
>> > > >>>>> that no
>> > > >>>>>> features
>> > > >>>>>>>>>>>>>>>>>> should be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
>> > > >>> have
>> > > >>>>> been resolved
>> > > >>>>>> and
>> > > >>>>>>>>> it
>> > > >>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>> ready
>> > > >>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > Ops
>> > > >>> vs
>> > > >>>>> discrete
>> > > >>>>>>>>> read/write
>> > > >>>>>>>>>>>>>>>>>> ops issue
>> > > >>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
>> > reviewed
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > I
>> > > >>> don't
>> > > >>>>> think this
>> > > >>>>>>>>> criteria
>> > > >>>>>>>>>>>>>>>>>> has been
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
>> > how
>> > > >>>>> does this work
>> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>> > > >>>>>>>>>>>>>>>>>> )?
>> > > >>>>>>>>>>>>>>>>>>>> As I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>> > > >>> involves
>> > > >>>>> provisioned
>> > > >>>>>>>>> IOPS. I
>> > > >>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>> like
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > scenario
>> > > >>>>> where provisioned
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> KVM and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > unnecessarily
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> incompatible.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > Jun 1,
>> > > >>> 2013,
>> > > >>>>> at 6:47 AM,
>> > > >>>>>>>>> Wei
>> > > >>>>>>>>>>>>>>>>>> ZHOU <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
>> > will
>> > > >>>>> change it next
>> > > >>>>>>>>> week.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/6/1
>> > > >>>>> Wido den Hollander
>> > > >>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 06/01/2013 08:24 AM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Exactly. I
>> > > >>> have
>> > > >>>>> pushed the
>> > > >>>>>>>>>>>>> features
>> > > >>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > anyone
>> > > >>>>> object thems for
>> > > >>>>>>>>>>>>> technical
>> > > >>>>>>>>>>>>>>>>>>>> reason
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
>> > the
>> > > >>> sake
>> > > >>>>> of clarity I
>> > > >>>>>> just
>> > > >>>>>>>>>>>>> want
>> > > >>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>> mention
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > total
>> > > >>> IOps
>> > > >>>>> to R/W IOps
>> > > >>>>>>>>> asap
>> > > >>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>> that we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>> > total
>> > > >>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
>> > laid
>> > > >>> the
>> > > >>>>> groundwork for
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>> throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > however
>> > > >>>>> prevent that we
>> > > >>>>>> create
>> > > >>>>>>>>>>>>>>>>>> legacy from
>> > > >>>>>>>>>>>>>>>>>>>>>> day
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/5/31
>> > > >>>>> Wido den
>> > > >>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/31/2013 03:59 PM, John
>> > > >>>>>>>>>>>>> Burwell
>> > > >>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
>> > this
>> > > >>>>> enhancement must
>> > > >>>>>> to
>> > > >>>>>>>>>>>>>>>>>> discretely
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> > see
>> > > >>> how
>> > > >>>>> it could be
>> > > >>>>>>>>> fixed
>> > > >>>>>>>>>>>>>>>>>> later
>> > > >>>>>>>>>>>>>>>>>>>>>> because
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > correctly
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
>> > total
>> > > >>> IOPS
>> > > >>>>> into read
>> > > >>>>>> and
>> > > >>>>>>>>>>>>>>>>>> write.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>> unless/until we
>> > > >>>>>> decided
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> break
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
>> > Wei
>> > > >>>>> meant was merging
>> > > >>>>>> it
>> > > >>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>> master
>> > > >>>>>>>>>>>>>>>>>>>>>> now
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > branch
>> > > >>>>> and add Read /
>> > > >>>>>>>>> Write
>> > > >>>>>>>>>>>>> IOps
>> > > >>>>>>>>>>>>>>>>>>>> before
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
>> > > >>>>> released with Read
>> > > >>>>>> and
>> > > >>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>> instead
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
>> > to
>> > > >>>>> make the May 31st
>> > > >>>>>>>>>>>>> feature
>> > > >>>>>>>>>>>>>>>>>>>> freeze
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
>> > other
>> > > >>>>> threads) then it
>> > > >>>>>>>>> won't
>> > > >>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>> necessary
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
>> > > >>>>> completely agree that
>> > > >>>>>>>>>>> there
>> > > >>>>>>>>>>>>>>>>>> is no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > I/O.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > May 31,
>> > > >>>>> 2013, at 9:51 AM,
>> > > >>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>> den
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/31/2013 03:13 PM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>> > Wido,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Thanks.
>> > > >>> Good
>> > > >>>>> question.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > thought
>> > > >>>>> about at the
>> > > >>>>>>>>>>> beginning.
>> > > >>>>>>>>>>>>>>>>>>>> Finally I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > difference
>> > > >>> of
>> > > >>>>> read and write
>> > > >>>>>>>>>>>>> mainly
>> > > >>>>>>>>>>>>>>>>>>>> because
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
>> > the
>> > > >>>>> difference of sent
>> > > >>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>> received
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>> > > >>>>> reasoning seems odd.
>> > > >>>>>>>>>>>>>>>>>> Networking and
>> > > >>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>> > I/O is
>> > > >>>>> much more
>> > > >>>>>>>>> expensive
>> > > >>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>> most
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > bandwith.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> Implementing
>> > > >>>>> it will be some
>> > > >>>>>>>>>>>>>>>>>> copy-paste
>> > > >>>>>>>>>>>>>>>>>>>>>> work.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> implemented
>> > > >>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
>> > days.
>> > > >>> For
>> > > >>>>> the deadline
>> > > >>>>>> of
>> > > >>>>>>>>>>>>>>>>>> feature
>> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
>> > if
>> > > >>>>> needed.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
>> > think it's
>> > > >>> a
>> > > >>>>> feature we
>> > > >>>>>>>>> can't
>> > > >>>>>>>>>>>>>>>>>> miss. But
>> > > >>>>>>>>>>>>>>>>>>>>>> if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > window
>> > > >>> we
>> > > >>>>> have to make sure
>> > > >>>>>> we
>> > > >>>>>>>>>>>>> don't
>> > > >>>>>>>>>>>>>>>>>>>> release
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
>> > 4.3,
>> > > >>> that
>> > > >>>>> will confuse
>> > > >>>>>>>>>>>>> users.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > 2013/5/31
>> > > >>>>> Wido den
>> > > >>>>>> Hollander <
>> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>> > > >>>>> 05/30/2013 06:03 PM, Wei
>> > > >>>>>>>>> ZHOU
>> > > >>>>>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> > would
>> > > >>> like to
>> > > >>>>> merge
>> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> > nobody
>> > > >>>>> object, I will
>> > > >>>>>> merge
>> > > >>>>>>>>>>>>> into
>> > > >>>>>>>>>>>>>>>>>> master
>> > > >>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>> purpose
>> > > >>>>> is :
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
>> > > >>>>> machines are running
>> > > >>>>>>>>> on
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>> same
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > storage or
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
>> > > >>> strage).
>> > > >>>>> Because of
>> > > >>>>>> the
>> > > >>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>>>> limitation
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
>> > if
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
>> > VM
>> > > >>> has
>> > > >>>>> large disk
>> > > >>>>>>>>> operation,
>> > > >>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>> may
>> > > >>>>>>>>>>>>>>>>>>>>>> affect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>> > VMs
>> > > >>>>> running on the
>> > > >>>>>> same
>> > > >>>>>>>>>>>>>>>>>> storage
>> > > >>>>>>>>>>>>>>>>>>>>>> device.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> > > >>> neccesary
>> > > >>>>> to set the
>> > > >>>>>>>>>>> maximum
>> > > >>>>>>>>>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Looking at
>> > > >>> the
>> > > >>>>> code I see
>> > > >>>>>> you
>> > > >>>>>>>>>>> make
>> > > >>>>>>>>>>>>>>>>>> no
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
>> > and
>> > > >>>>> libvirt support
>> > > >>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>> both a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > > >>> which
>> > > >>>>> could benefit a
>> > > >>>>>>>>> lot of
>> > > >>>>>>>>>>>>>>>>>> users.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
>> > also
>> > > >>>>> strange, in the
>> > > >>>>>>>>> polling
>> > > >>>>>>>>>>>>>>>>>> side you
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>> > but
>> > > >>> on
>> > > >>>>> the throttling
>> > > >>>>>>>>> side
>> > > >>>>>>>>>>>>>>>>>> you only
>> > > >>>>>>>>>>>>>>>>>>>> go
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> > IOps
>> > > >>> are
>> > > >>>>> usually much
>> > > >>>>>>>>> more
>> > > >>>>>>>>>>>>>>>>>> expensive
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
>> > valid
>> > > >>>>> use-case where
>> > > >>>>>>>>> that
>> > > >>>>>>>>>>>>> an
>> > > >>>>>>>>>>>>>>>>>> admin
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> > vs
>> > > >>> Read
>> > > >>>>> IOps.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
>> > this
>> > > >>> only
>> > > >>>>> supports KVM
>> > > >>>>>>>>> at
>> > > >>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>> point I
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>> > to at
>> > > >>>>> least have the
>> > > >>>>>>>>>>>>>>>>>> mechanism in
>> > > >>>>>>>>>>>>>>>>>>>>>> place
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> implementing
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > later
>> > > >>>>> would be a lot of
>> > > >>>>>>>>>>> work.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
>> > > >>> hypervisor
>> > > >>>>> doesn't
>> > > >>>>>>>>> support
>> > > >>>>>>>>>>>>>>>>>> setting
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> > you
>> > > >>> can
>> > > >>>>> always sum
>> > > >>>>>> both
>> > > >>>>>>>>> up
>> > > >>>>>>>>>>>>>>>>>> and set
>> > > >>>>>>>>>>>>>>>>>>>>>> that
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> > you
>> > > >>>>> explain why you
>> > > >>>>>>>>>>>>> implemented
>> > > >>>>>>>>>>>>>>>>>> it
>> > > >>>>>>>>>>>>>>>>>>>> this
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> > > >>> feature
>> > > >>>>> includes:
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> > the
>> > > >>>>> maximum rate of
>> > > >>>>>>>>> VMs
>> > > >>>>>>>>>>>>> (in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> configuration)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> > change
>> > > >>> the
>> > > >>>>> maximum rate
>> > > >>>>>> of
>> > > >>>>>>>>>>> VMs
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> > limit
>> > > >>> the
>> > > >>>>> disk rate
>> > > >>>>>> (total
>> > > >>>>>>>>>>> bps
>> > > >>>>>>>>>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>> iops)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
>> > ticket:
>> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> issues.apache.org/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> jira/browse/CLOUDSTACK-1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1192>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>> 1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 1192
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1192<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 1192>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
>> > will
>> > > >>>>> update later) :
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>> VM+Disk+IO+Throttling<https://
>> > > >>>>>>>>>>>>> ****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cwiki.apache.org/confluence/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>
>> > > >>>
>> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>> > > >>>>> rottling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Merge
>> > > >>> check
>> > > >>>>> list :-
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
>> > you
>> > > >>>>> check the branch's
>> > > >>>>>>>>> RAT
>> > > >>>>>>>>>>>>>>>>>> execution
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
>> > > >>> there
>> > > >>>>> new dependencies
>> > > >>>>>>>>>>>>>>>>>> introduced?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> > What
>> > > >>>>> automated testing
>> > > >>>>>> (unit
>> > > >>>>>>>>>>> and
>> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > feature?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
>> > tests
>> > > >>> are
>> > > >>>>> added.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
>> > What
>> > > >>>>> testing has been done
>> > > >>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>>> check for
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> > the
>> > > >>>>> bytes rate and
>> > > >>>>>>>>> IOPS
>> > > >>>>>>>>>>>>>>>>>> rate on
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> > VM
>> > > >>>>> operations, including
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > deploy,
>> > > >>> stop,
>> > > >>>>> start, reboot,
>> > > >>>>>>>>>>>>>>>>>> destroy,
>> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> > > >>> Volume
>> > > >>>>> operations,
>> > > >>>>>>>>> including
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
>> > > >>>>> Detach
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
>> > review
>> > > >>> the
>> > > >>>>> code, you can
>> > > >>>>>>>>> try
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
>> > > >>> regards,
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>
>> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>> VM+Disk+IO+Throttling<https://
>> > > >>>>>>>>>>>>> ****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cwiki.apache.org/confluence/****
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>
>> > > >>>
>> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>> > > >>>>> rottling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>> > > >>>>>>>>> refs/heads/disk_io_throttling
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>>>> 1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1301>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 1301
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 1301<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 1301>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <ht**tps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > issues.apache.org/****jira/**browse/CLOUDSTACK-
>> > > >>> 2071
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
>> > 2071
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>> > > >>>>> 2071
>> > > >>>>>>>>>>>>>>>>>>>> <
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>> > > >>>>> 2071>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>> <h**ttps://
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
>> > 2071>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> (**CLOUDSTACK-1301
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
>> > VM
>> > > >>> Disk
>> > > >>>>> I/O
>> > > >>>>>> Throttling)
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > > >>>>> Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>>>> CloudStack Developer,
>> > > >>>>>>>>>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>>>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > > >>> 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > Advancing
>> > > >>> the
>> > > >>>>> way the world
>> > > >>>>>>>>> uses
>> > > >>>>>>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > > >>> Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>> CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> > the
>> > > >>> way
>> > > >>>>> the world uses
>> > > >>>>>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > > >>> CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> > 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > > >>> way
>> > > >>>>> the world uses
>> > > >>>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> > Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > CloudStack
>> > > >>>>> Developer,
>> > > >>>>>>>>> SolidFire
>> > > >>>>>>>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > way
>> > > >>> the
>> > > >>>>> world uses the
>> > > >>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> > CloudStack
>> > > >>>>> Developer,
>> > > >>>>>> SolidFire
>> > > >>>>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> > way
>> > > >>> the
>> > > >>>>> world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>>>> Developer, SolidFire
>> > > >>>>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> > the
>> > > >>>>> world uses the
>> > > >>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>> Developer,
>> > > >>>>> SolidFire
>> > > >>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> > the
>> > > >>> world
>> > > >>>>> uses the
>> > > >>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > > >>> Developer,
>> > > >>>>> SolidFire
>> > > >>>>>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>>>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > > >>> world
>> > > >>>>> uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > Developer,
>> > > >>>>> SolidFire
>> > > >>>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > > >>> mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > world
>> > > >>> uses
>> > > >>>>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> > Developer,
>> > > >>>>> SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> > world
>> > > >>> uses
>> > > >>>>> the cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>>>> SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> > uses
>> > > >>> the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>
>> > http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> > mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> > uses
>> > > >>> the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > > >>> SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > SolidFire
>> > > >>>>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> > the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> > SolidFire
>> > > >>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>> cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > > >>> Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > cloud<
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> > Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
>> > > >>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>>>>>>>
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>>>>>
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>>>>> *(tm)*
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> --
>> > > >>>>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>>>> o: 303.746.7302
>> > > >>>>>>>>>> Advancing the way the world uses the
>> > > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>>>> *(tm)*
>> > > >>>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> --
>> > > >>>>>>>> *Mike Tutkowski*
>> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>>> o: 303.746.7302
>> > > >>>>>>>> Advancing the way the world uses the cloud<
>> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>>> *(tm)*
>> > > >>>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>>
>> > > >>>>>>> --
>> > > >>>>>>> *Mike Tutkowski*
>> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>>>> o: 303.746.7302
>> > > >>>>>>> Advancing the way the world uses the
>> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>>>> *(tm)*
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> --
>> > > >>>>> *Mike Tutkowski*
>> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>>>> e: mike.tutkowski@solidfire.com
>> > > >>>>> o: 303.746.7302
>> > > >>>>> Advancing the way the world uses the
>> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>>>> *(tm)*
>> > > >>>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> --
>> > > >>> *Mike Tutkowski*
>> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >>> e: mike.tutkowski@solidfire.com
>> > > >>> o: 303.746.7302
>> > > >>> Advancing the way the world uses the
>> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > > >>> *(tm)*
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Mike Tutkowski*
>> > > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > > e: mike.tutkowski@solidfire.com
>> > > > o: 303.746.7302
>> > > > Advancing the way the world uses the
>> > > > cloud<http://solidfire.com/solution/overview/?video=play>
>> > > > *(tm)*
>> > >
>> > >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *(tm)*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi Edison and John,
I wanted to point out something I thought was relevant to this conversation.
My plug-in supports both XenServer and ESX(i).
These two hypervisors handle their respective hypervisor data structures
(XenServer SR or ESX(i) datastore) quite differently.
In XenServer, the SR that resides on the SAN LUN is simply "forgotten" when
you issue a remove-SR command to XenServer.
When you later want to access the contents of that data, it is still safely
stored on the LUN and can be accessed from XenServer (from the same or
different cluster (what XenServer calls a resource pool)) by creating a new
SR that is based on the same IQN and LUN.
XenServer SRs exist at the cluster level.
In ESX(i), the datastore that resides on the SAN LUN is actually destroyed
when you issue a remove-datastore command to ESX(i). In other words, the
contents of the SAN LUN are destroyed.
ESX(i) datastores exist at the datacenter level.
The XenServer behavior (not destroying the contents of the LUN when the SR
is removed) is desirable; the ESX(i) behavior (destroying the contents of
the LUN when the datastore is removed) is not desirable.
I can get around the ESX(i) behavior by removing the referenced IQN from
each host in the cluster when the CS storage framework notifies my plug-in
of a detach-volume event. When the CS volume is later attached again
(either to the same or a different ESX(i) cluster), I can have each host in
the relevant ESX(i) cluster reference the IQN and the datastore will be
back again.
This sequence of events is critical for my plug-in to support both
environments:
*XenServer flow (events for Driver class):*
createAsync (called right before first attach) = creates SAN volume
preAttachVolume = creates SR and creates VDI or just introduces SR
(CS volume gets attached and later detached)
postDetachVolume = removes SR (this does not delete data on SAN volume)
deleteAsync (can be called when volume is not attached) = deletes the SAN
volume
*VMware flow (events for Driver class):*
createAsync (called right before first attach) = creates SAN volume,
creates datastore, and creates VMDK file
preAttachVolume = adds iSCSI target to each host in cluster
(CS volume gets attached and later detached)
postDetachVolume = removes iSCSI target from each host in cluster
deleteAsync (can be called when volume is not attached) = deletes datastore
and deletes the SAN volume
********************
So, the way I have it written up here, the driver is talking to the
hypervisor. As John has pointed out, this may not be ideal.
Whatever we decide on, it needs to be able to support this kind of flow. :)
We could go John's route, where the driver says if it's managed or not.
If it is, then the storage framework could send the necessary messages to
the hypervisor in question at the right times (just migrate the hypervisor
logic above from the driver to the right parts of the storage framework).
This is fine, except that it does not resolve Edison's use case of desiring
to have the storage LUN potentially connected directly to a VM.
I'm not sure we should design around that use case, though. Is it common
enough? Has anyone asked for it? Is it even desirable? How are usage
statistics gathered for such volumes? It's nice to offer the flexibility,
but is it a good idea in this case?
Thanks!
On Fri, Jun 7, 2013 at 6:19 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
> > -----Original Message-----
> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > Sent: Friday, June 07, 2013 4:08 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > Hi John,
> >
> > Yes, you are correct that Xen must layer an SR on top of the iSCSI volume
> > to use it. Same story for VMware: ESX(i) must layer a datastore on top of
> > the iSCSI volume to use it. I look at it like they are layering a
> clustered
> > file system on the SAN volume so the hypervisors can share access to the
> > contents of the volume.
> >
> > In 4.1, by the time the hypervisor-attach-volume logic is called, the SR
> or
> > datastore has already been created (usually manually by an admin).
> >
> > This pre-setup of the, say, SR is, of course, not acceptable in an
> > environment where each CloudStack volume that a user creates is mapped
> > to a
> > single SAN volume (via the SR).
> >
> > The question comes down to who should allocate the SR dynamically.
> >
> > We could have the storage framework ask the storage plug-in if it is
> > managed. If it is, then the storage framework could send a message to the
> > hypervisor in question to create (let's talk Xen here) the SR ahead of
> > time. Then, when the storage framework next sends the attach-volume
> > command
> > to the hypervisor, it should work without changes to that attach-volume
> > logic (because - from the point of view of the attach logic - the SR is
> > already existent, as expected).
> >
> > Now, as Edison has pointed out, this limits the power of the storage
> > plug-in. A storage plug-in in this model cannot directly attach storage
> to
> > a VM (it must go through the hypervisor). Perhaps that is OK. We need to
> > make a call on that.
>
>
> Yes, I think we need to address this usage case, and won't limit the power
> of the storage plug-in.
> John, what's your idea to fit this usage case into your abstraction?
>
>
> >
> > If we want to give more power to the storage vendor, we would have to
> > have
> > the storage framework call into the storage plug-in to send the
> appropriate
> > attach commands to the hypervisor.
> >
> > Let's discuss and come to a consensus (or at least agree on a path) as
> soon
> > as we can. :)
> >
> > Thanks!
> >
> >
> >
> > On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com> wrote:
> >
> > > Mike,
> > >
> > > My understanding is the Xen expects an SR structure on all ISCSI
> devices
> > > -- at least that is how I read the code in your patch. Is my
> understanding
> > > correct? If so, the Xen plugin should be able to query the storage
> device
> > > to determine the presence of the SR structure and create it if it does
> not
> > > exist. Am I missing something in the implementation that makes that
> type
> > > of implementation impossible?
> > >
> > > Thanks,
> > > -John
> > >
> > > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> > > wrote:
> > >
> > > > Yeah, if a storage vendor wanted to attach a volume directly to a VM
> in
> > > 4.2
> > > > today, it would probably fail because the attach-volume logic assumes
> > the
> > > > existence of the necessary hypervisor data structure (ex. SR on Xen).
> > > >
> > > > If we wanted to enable such an attach, we could do it the way Edison
> > > > suggests.
> > > >
> > > >
> > > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com>
> wrote:
> > > >
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > >>> Sent: Friday, June 07, 2013 2:37 PM
> > > >>> To: dev@cloudstack.apache.org
> > > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > > >>>
> > > >>> As we only have three weeks until feature freeze, we should come to
> > a
> > > >>> consensus on this design point as soon as possible.
> > > >>>
> > > >>> Right now, if the storage framework asks my driver it is is
> managed, it
> > > >>> will say 'yes.' This means the framework will tell the driver to
> > > perform
> > > >>> its management activities. This then means the driver will call
> into
> > > the
> > > >>> host (it doesn't know which hypervisor, by the way) to perform the
> > > >> activity
> > > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> > > >>>
> > > >>> The driver doesn't know which hypervisor it's talking to, it just
> > > sends a
> > > >>> message to the host to perform the necessary pre-attach work.
> > > >>
> > > >> Could we just expose a method like "attachVolume/dettachVolume" on
> > the
> > > >> PrimaryDataStoreDriver driver?
> > > >> For most of cases, the implementation of each driver would just send
> > > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
> > the
> > > >> implementation in a base class, so that can be shared by all of this
> > > >> drivers), and in each hypervisor resource code, which may just call
> > > >> hypervisor's api to attach the volume to VM, while for certain
> storage,
> > > >> like SolidFire, may need to create a SR first, and create volume on
> it,
> > > >> then call hypervisor's API to attach volume to VM.
> > > >> While for some other storage vendor, may want to bypass hypervisor
> > > during
> > > >> attaching volume, so inside driver's attachvolume implementation,
> the
> > > >> driver can do some magic, such as, directly talk to an agent inside
> VM
> > > >> instance, then create a disk inside VM.
> > > >>
> > > >> How do you guys think?
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> > > wrote:
> > > >>>
> > > >>>>
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> > > >>>>> To: dev@cloudstack.apache.org
> > > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > > >>>>>
> > > >>>>> Hi John,
> > > >>>>>
> > > >>>>> How's about this:
> > > >>>>>
> > > >>>>> The driver can implement an isManaged() method. The
> > > >>> VolumeManagerImpl
> > > >>>>> can
> > > >>>>> call into the driver to see if its managed. If it is, the
> > > >>>> VolumeManagerImpl
> > > >>>>> (which is responsible for calling into the hypervisor to attach
> the
> > > >> disk)
> > > >>>>> can call into the hypervisor to create the necessary hypervisor
> data
> > > >>>>> structure (ex. for XenServer, a storage repository).
> > > >>>>
> > > >>>> The problem here is that storage vendor may work differently with
> > > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> > > >> other
> > > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> > directly
> > > >> to
> > > >>>> VM instance, see the discuss(
> > > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> > > >>>
> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> > > >>> %3E
> > > >>>> ).
> > > >>>> So I would let storage provider to implement attach disk to VM,
> > > >> instead of
> > > >>>> implemented by cloudstack itself.
> > > >>>>
> > > >>>>
> > > >>>>>
> > > >>>>> If that's what you're going for, that works for me. By the way,
> > > >> Edison's
> > > >>>>> default storage plug-in (which handles the default storage
> behavior
> > > >> in
> > > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks
> to
> > > >>>>> hypervisors. You might want to contact him and inform him of your
> > > >>>> concerns
> > > >>>>> or that logic (as is) will make it to production.
> > > >>>>>
> > > >>>>> Please let me know if what I wrote in above (for my solution) is
> OK
> > > >> with
> > > >>>>> you. :)
> > > >>>>>
> > > >>>>> Thanks!
> > > >>>>>
> > > >>>>>
> > > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jburwell@basho.com
> >
> > > >>> wrote:
> > > >>>>>
> > > >>>>>> Mike,
> > > >>>>>>
> > > >>>>>> Please see my responses in-line below.
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> -John
> > > >>>>>>
> > > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > > >>>> mike.tutkowski@solidfire.com>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hey John,
> > > >>>>>>>
> > > >>>>>>> I still have a bit more testing I'd like to do before I build
> up
> > > >> a
> > > >>>> patch
> > > >>>>>>> file, but this is the gist of what I've done:
> > > >>>>>>>
> > > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
> > tells
> > > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
> > > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if
> it
> > > >>>>> managed.
> > > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> > > >> whatever
> > > >>>>>>> activity is required. In my case, this includes sending a
> > > >> message to
> > > >>>> the
> > > >>>>>>> host where the VM is running to have, say XenServer, add a
> > > >> storage
> > > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
> > > >> SAN
> > > >>>>>> volume,
> > > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it
> can
> > > >> on
> > > >>>> the
> > > >>>>>>> storage repository). After this, the normal attach-volume
> > > >> message is
> > > >>>> sent
> > > >>>>>>> to the host by VolumeManagerImpl.
> > > >>>>>>
> > > >>>>>> There should be **no** code from a storage driver to a
> hypervisor.
> > > >> I
> > > >>>>>> apologize for the repetition, but we simply can not have
> hypervisor
> > > >>>>>> specific code in the storage layer. The circular dependencies
> > > >> between
> > > >>>> the
> > > >>>>>> two layers are not sustainable in the long term. Either the
> > > >>>> VirtualManager
> > > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> > > >>> coordinate
> > > >>>>>> volume creation and then populating the SR. Ideally, we can
> > > >>>> generalize the
> > > >>>>>> process flow for attaching volumes such that the Xen hypervisor
> > > >> plugin
> > > >>>>>> would only implement callbacks to perform the attach action and
> > > >> create
> > > >>>>> the
> > > >>>>>> structure and SR. To my mind, the SolidFire driver should only
> be
> > > >>>>>> allocating space and providing information about contents (e.g.
> > > >> space
> > > >>>>>> available, space consumed, streams to a URI, file handle for a
> URI,
> > > >>>> etc)
> > > >>>>>> and capabilities.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> * The reverse is performed for a detach-volume command.
> > > >>>>>>>
> > > >>>>>>> * Right now I simply "return true;" for isManaged() in my
> driver.
> > > >>>>>> Edison's
> > > >>>>>>> default driver simply does a "return false;". We could add a
> new
> > > >>>>>> parameter
> > > >>>>>>> to the createStoragePool API command, if we want, to remove
> > the
> > > >>>>>> hard-coded
> > > >>>>>>> return values in the drivers (although my driver will probably
> > > >> just
> > > >>>>>> ignore
> > > >>>>>>> this parameter and always return true since it wouldn't make
> > > >> sense
> > > >>>> for it
> > > >>>>>>> to ever return false). We'd need another column in the
> > > >> storage_pool
> > > >>>>> table
> > > >>>>>>> to store this value.
> > > >>>>>>
> > > >>>>>> Yes, I think we should have a parameter added to the
> > > >>> createStoragePool
> > > >>>>>> surfaced to the HTTP API that allows DataStores to be configured
> > > >> for
> > > >>>>>> management when their underlying drivers support it. To
> simplify
> > > >>>> things,
> > > >>>>>> this flag should only be mutable when the DataStore is created.
> It
> > > >>>> would be
> > > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
> > > >>>> creation.
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> Sound like I'm in sync with what you were thinking?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > > >>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> I agree, John. Just wanted to point out that I have a working
> > > >> GUI
> > > >>>> for
> > > >>>>>> you
> > > >>>>>>>> to review (in that document), if you'd like to check it out.
> > > >>>>>>>>
> > > >>>>>>>> Thanks!
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> > > >> jburwell@basho.com>
> > > >>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Mike,
> > > >>>>>>>>>
> > > >>>>>>>>> I would like the UIs of two features reviewed together to
> > > >> ensure
> > > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
> > > >> and
> > > >>>>>>>>> storage device provisioned IOPs. I see the potential for
> > > >>>> confusion,
> > > >>>>>>>>> and I think a side-by-side Ui review of these features will
> > > >> help
> > > >>>>>>>>> minimize any potential confusion.
> > > >>>>>>>>>
> > > >>>>>>>>> As I mentioned, the term reconciliation issue will work
> itself
> > > >> if
> > > >>>> it
> > > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> > > >>>> throttled
> > > >>>>>>>>> IOPs or storage provisioned IOPs.
> > > >>>>>>>>>
> > > >>>>>>>>> Thanks,
> > > >>>>>>>>> -John
> > > >>>>>>>>>
> > > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > > >>>>>>>>> <mi...@solidfire.com> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hi John,
> > > >>>>>>>>>>
> > > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
> > to
> > > >>> you
> > > >>>>> the
> > > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
> > > >> IOPS.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> > > >> and we
> > > >>>>>>>>> decided
> > > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> > Offering
> > > >>>> dialog.
> > > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> > should
> > > >>> be
> > > >>>>> able
> > > >>>>>> to
> > > >>>>>>>>>> leverage this GUI going forward (even if they, say, only use
> > > >> Max
> > > >>>>>> IOPS).
> > > >>>>>>>>>> These fields are optional and can be ignored for storage
> that
> > > >>>> does not
> > > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> > > >>>> admin can
> > > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
> > > >> IOPS.
> > > >>>>>>>>>>
> > > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
> > > >> mine,
> > > >>>> as
> > > >>>>>> is
> > > >>>>>>>>>> decided by the admin).
> > > >>>>>>>>>>
> > > >>>>>>>>>> I'm not sure what we can do about these two features
> > > >> expressing
> > > >>>> the
> > > >>>>>>>>> speed
> > > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS
> for
> > > >>>> QoS in
> > > >>>>>>>>> any
> > > >>>>>>>>>> way other than total IOPS (i.e. not broken in into
> read/write
> > > >>>> IOPS).
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thanks
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > > >>> <jb...@basho.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Wei,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> > > >>> Storage/Hypervisor
> > > >>>>>> layer
> > > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
> > > >>>> hypervisor
> > > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> > > >> following
> > > >>>>>> issues
> > > >>>>>>>>>>> outstanding:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > > >>> read/write
> > > >>>>>>>>> values
> > > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
> > > >> IOPS.
> > > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> > > >>> storage
> > > >>>>>> volumes
> > > >>>>>>>>>>> with provisioned IOPS?
> > > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> > > >>>> provisioned
> > > >>>>>> IOPS
> > > >>>>>>>>>>> that will permit providers to discriminate these guaranteed
> > > >>>>>> operations
> > > >>>>>>>>> in
> > > >>>>>>>>>>> the event they want to bill for it differently?
> > > >>>>>>>>>>> 4. What is the user experience for throttled and
> provisioned
> > > >>> IOPS
> > > >>>>>> that
> > > >>>>>>>>>>> minimizes confusion of these concepts?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> > > >>>> throttled
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly
> solve
> > > >>>> items 1
> > > >>>>>>>>> and 2.
> > > >>>>>>>>>>> Since the two facilities would not be permitted to operate
> > > >>>> together,
> > > >>>>>>>>> they
> > > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
> > > >> can be
> > > >>>>>>>>> resolved
> > > >>>>>>>>>>> with an additional flag or two on the usage records. As
> for
> > > >>>> Item 4,
> > > >>>>>> I
> > > >>>>>>>>> am
> > > >>>>>>>>>>> not familiar with how these two enhancements are
> > depicted in
> > > >>> the
> > > >>>>> user
> > > >>>>>>>>>>> interface. I think we need to review the UI enhancements
> > for
> > > >>>> both
> > > >>>>>>>>>>> enhancements and ensure they are consistent.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Do these solutions make sense?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks,
> > > >>>>>>>>>>> -John
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> > <us...@gmail.com>
> > > >>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> John and Mike,
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> > 2780/2729,
> > > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > > >>> CLOUDSTACK-
> > > >>>>> 2875 ) this
> > > >>>>>>>>> week.
> > > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
> > > >> ask
> > > >>>> for
> > > >>>>>>>>> second
> > > >>>>>>>>>>>> merge request after testing.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> I believe I understand where you're going with this,
> John,
> > > >> and
> > > >>>>> have
> > > >>>>>>>>> been
> > > >>>>>>>>>>>>> re-working this section of code today.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks for the comments,
> > > >>>>>>>>>>>>> Mike
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > > >>>>> <jb...@basho.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> See my responses in-line below.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks for the response.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> > whether or
> > > >>>>> not a
> > > >>>>>>>>> Volume
> > > >>>>>>>>>>> is
> > > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> > > >>> actual
> > > >>>>>>>>> isManaged
> > > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> > > >> resources
> > > >>>> for
> > > >>>>>> the
> > > >>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>> if it is managed."
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> > method
> > > >>>>> added to
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> > > >> then the
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
> > > >>> method
> > > >>>>> (which
> > > >>>>>>>>>>> would
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
> > and
> > > >>> this
> > > >>>>>> would
> > > >>>>>>>>>>> call
> > > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
> > on
> > > >>>>>>>>> XenServer,
> > > >>>>>>>>>>> an
> > > >>>>>>>>>>>>> SR?
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> > > >>>>> DataStoreDriver.
> > > >>>>>>>>> Since
> > > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
> > I
> > > >>>> would
> > > >>>>>> add
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> > straw
> > > >>> man
> > > >>>>> term
> > > >>>>>>>>> --
> > > >>>>>>>>>>>>>> something other than volume) methods to
> > > >>>>>>>>>>> allocate/create/deallocate/delete
> > > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> > > >>> property
> > > >>>>> of
> > > >>>>>>>>>>>>> DataStore
> > > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> > > >>> DataStoreDriver
> > > >>>>> can be
> > > >>>>>>>>>>>>> managed.
> > > >>>>>>>>>>>>>> This approach allows operators to override
> > manageability
> > > >> of
> > > >>>>>> devices.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> > > >> plugin
> > > >>>>>> would
> > > >>>>>>>>> be
> > > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
> > to
> > > >>>>> create any
> > > >>>>>>>>>>>>>> directories or files necessary for the SR. There should
> > > >> be no
> > > >>>>>>>>>>>>> dependencies
> > > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> > > >> earlier,
> > > >>>> such
> > > >>>>>>>>>>>>> circular
> > > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
> > mess.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks again, John
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > > >>>>> <jb...@basho.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
> > that
> > > >>>>>> supports n
> > > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> > > >> hypervisors.
> > > >>>>>> Such
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> > > >>> beast.
> > > >>>>>>>>>>>>> Therefore, my
> > > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
> > the
> > > >>>>> Storage
> > > >>>>>> layer
> > > >>>>>>>>>>>>> towards
> > > >>>>>>>>>>>>>>> this separation.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> > whether
> > > >>> or
> > > >>>>> not a
> > > >>>>>>>>> Volume
> > > >>>>>>>>>>> is
> > > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
> > an
> > > >>> actual
> > > >>>>>>>>>>> isManaged
> > > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> > > >> resources
> > > >>>> for
> > > >>>>>> the
> > > >>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
> > > >> to
> > > >>>>>> indicate
> > > >>>>>>>>>>>>> whether
> > > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
> > > >> think
> > > >>>> we
> > > >>>>>>>>> need to
> > > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to
> have
> > > >>>>>> something
> > > >>>>>>>>>>>>> that is
> > > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> > > >>> managing
> > > >>>>> the
> > > >>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> > unmanaged).
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
> > in
> > > >>> the
> > > >>>>> Xen
> > > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> > Storage
> > > >>>>> layer to
> > > >>>>>>>>>>> perform
> > > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
> > > >> file,
> > > >>>> etc).
> > > >>>>>>>>>>> This
> > > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> > > >>>> hypervisor, as
> > > >>>>>>>>> such,
> > > >>>>>>>>>>>>>>> should be confined to its implementation. From my
> > > >>>>> perspective,
> > > >>>>>>>>>>> nothing
> > > >>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
> > From
> > > >>> its
> > > >>>>>>>>>>>>> perspective,
> > > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
> > to
> > > >>>> query
> > > >>>>>> the
> > > >>>>>>>>>>> data
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
> > level
> > > >>>>> layers
> > > >>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> > composition
> > > >>> of
> > > >>>>>>>>>>> operations
> > > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> > Volume
> > > >>>>> storage
> > > >>>>>>>>>>> protocol
> > > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> > implemented
> > > >> in
> > > >>>> my
> > > >>>>>>>>> patch,
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>> could do the following:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> > > >>>>>>>>> VolumeManagerImpl
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> > > >>>>> hypervisor.
> > > >>>>>> Before
> > > >>>>>>>>>>> this
> > > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> > dealing
> > > >>> with
> > > >>>>>>>>> Dynamic
> > > >>>>>>>>>>> (or
> > > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so -
> send a
> > > >>>> "Create
> > > >>>>>>>>> SR"
> > > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> > would
> > > >>> then
> > > >>>>>>>>> proceed
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
> > > >> add
> > > >>>>>> another
> > > >>>>>>>>>>>>>>> command to the agent code that is called for this
> > > >> particular
> > > >>>>> type
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>> storage.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> What do you think?
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Hey John,
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> I created a document for a customer today that
> > outlines
> > > >>> how
> > > >>>>> the
> > > >>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> > > >> use
> > > >>>> to
> > > >>>>>>>>> you, as
> > > >>>>>>>>>>>>> well,
> > > >>>>>>>>>>>>>>>> as you perform the code review.
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> I have shared this document with you (you should
> > have
> > > >>>>> received
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>>>>> information in a separate e-mail).
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Talk to you later!
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > > >>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> > thoughts,
> > > >>> but
> > > >>>>>> want
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > > >>>>>>>>>>>>> disk_io_throttle,
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> > > >>>>> coherent
> > > >>>>>>>>> form
> > > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
> > the
> > > >>>> notion
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
> > > >> the
> > > >>>>> notion
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
> > was
> > > >>>>>>>>> incomplete,
> > > >>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>> I had
> > > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> > > >> device.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> > > >> this.
> > > >>>> :)
> > > >>>>> Do
> > > >>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>> have a
> > > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> > > >>> what
> > > >>>>> you're
> > > >>>>>>>>> looking
> > > >>>>>>>>>>>>>>>>>> for?
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > > >>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
> > the
> > > >>>> notion
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> > > >>>>> capabilities
> > > >>>>>>>>> (i.e.
> > > >>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> > > >>> managed
> > > >>>>> by
> > > >>>>>>>>> CloudStack)
> > > >>>>>>>>>>>>>>>>>> from the
> > > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
> > a
> > > >>>>> semantic
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
> > I
> > > >>> also
> > > >>>>> want
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
> > the
> > > >>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> definitions.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > > >>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> > questions
> > > >>> for
> > > >>>>> me
> > > >>>>>> with
> > > >>>>>>>>>>>>>>>>>> regards to
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
> > and
> > > >>> SAN
> > > >>>>>> volumes?
> > > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
> > (LUN
> > > >>> 0).
> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
> > and
> > > >>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> Volumes?
> > > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> > > >>> server or
> > > >>>>>>>>> externally
> > > >>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> > > >>> volume
> > > >>>>> (same
> > > >>>>>> as a
> > > >>>>>>>>>>> SAN
> > > >>>>>>>>>>>>>>>>>> LUN) is
> > > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
> > plug-in)
> > > >>>>> the
> > > >>>>>> first
> > > >>>>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
> > but
> > > >>>> still
> > > >>>>>>>>> want
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> use a
> > > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> > > >>> today
> > > >>>>>> (prior
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> 4.2). The
> > > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
> > in
> > > >>> this
> > > >>>>> case -
> > > >>>>>>>>>>>>>>>>>> multiple VMs
> > > >>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
> > you
> > > >>> can do
> > > >>>>> this
> > > >>>>>>>>>>>>> today,
> > > >>>>>>>>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
> > server
> > > >>> or
> > > >>>>> externally
> > > >>>>>>>>> by
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> > > >>> volumes
> > > >>>>> are
> > > >>>>>>>>>>>>> completely
> > > >>>>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
> > There
> > > >>> is
> > > >>>>> no
> > > >>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>> interaction.
> > > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> > > >>> volume
> > > >>>>> and a
> > > >>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > > >>>>> supports
> > > >>>>>> true
> > > >>>>>>>>>>> QoS.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> > > >> related
> > > >>>>> and
> > > >>>>>>>>>>>>> expected
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> operate.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> > multiple
> > > >>> VMs
> > > >>>>> with
> > > >>>>>>>>>>>>>>>>>> provisioned
> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> > > >>> single
> > > >>>>> SAN
> > > >>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>> (LUN) to a
> > > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> > > >>> multiple
> > > >>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> > guarantee
> > > >>> QoS.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> > volume
> > > >>> house
> > > >>>>> more
> > > >>>>>>>>> than
> > > >>>>>>>>>>>>> one
> > > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> > > >>> without
> > > >>>>> any of
> > > >>>>>> my
> > > >>>>>>>>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>>>> code.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> > volume
> > > >>> for
> > > >>>>>>>>>>> CloudStack's
> > > >>>>>>>>>>>>>>>>>> use
> > > >>>>>>>>>>>>>>>>>>>> onto
> > > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> > Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> > concerns me
> > > >>> --
> > > >>>>>> concepts
> > > >>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
> > today
> > > >>>> only
> > > >>>>>> used
> > > >>>>>>>>>>> (as
> > > >>>>>>>>>>>>>>>>>> far as
> > > >>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
> > not
> > > >>>>> PreSetup an
> > > >>>>>>>>> SR).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> > from
> > > >>>>> VMware, it
> > > >>>>>>>>> uses
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> vmfs
> > > >>>>>>>>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> > from
> > > >>> KVM,
> > > >>>>> it uses
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> > > >>> here
> > > >>>>> was to
> > > >>>>>>>>> make a
> > > >>>>>>>>>>>>>>>>>> new type
> > > >>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> > > >>> Edison
> > > >>>>> added
> > > >>>>>>>>> into
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> > > >>> specificy,
> > > >>>>> though:
> > > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> > > >>> Tutkowski
> > > >>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
> > know
> > > >>> or
> > > >>>>> care
> > > >>>>>> that
> > > >>>>>>>>> it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>> used
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > > >>>>> questions
> > > >>>>>>>>> about
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > > >>>>> SolidFire
> > > >>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>> needs to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> call
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> > > >>> hypervisor
> > > >>>>> logic
> > > >>>>>>>>>>> knows
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> treat
> > > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> > constraint
> > > >>>>> and just
> > > >>>>>>>>>>>>>>>>>> calling it
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
> > for
> > > >>> the
> > > >>>>> hypervisor
> > > >>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> > perform
> > > >>> the
> > > >>>>> necessary
> > > >>>>>>>>>>>>>>>>>> activities.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> > > >>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > > >>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
> > able to
> > > >>>>> manage a
> > > >>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > > >>> CloudStack
> > > >>>>> (e.g. the
> > > >>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>> shared
> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> > > >>> chosen
> > > >>>>> to
> > > >>>>>>>>> dedicate
> > > >>>>>>>>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>>>>>>>> portion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> > > >>> make
> > > >>>>> sense?"
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
> > SAN
> > > >>>>> volume via
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
> > would
> > > >>>>> work unless
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
> > an
> > > >>> SR.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > > >>>>> CloudStack, an
> > > >>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>> allocates
> > > >>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> > > >>> exposes it
> > > >>>>>> through
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> LUN. They
> > > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
> > to
> > > >>>>> CloudStack,
> > > >>>>>>>>> but
> > > >>>>>>>>>>>>>>>>>> not allow
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > > >>>>> changes, we
> > > >>>>>>>>> would
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>> passing
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > > >>>>> necessary
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>> (like
> > > >>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
> > have
> > > >>> the
> > > >>>>> storage
> > > >>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>> itself
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> > support
> > > >>> from
> > > >>>>> the
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> side and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
> > to
> > > >>> create
> > > >>>>> the
> > > >>>>>>>>> SR
> > > >>>>>>>>>>>>>>>>>> before the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
> > plugin
> > > >>>>> should
> > > >>>>>> have a
> > > >>>>>>>>>>>>> rich
> > > >>>>>>>>>>>>>>>>>> enough
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> > > >>> available
> > > >>>>> for
> > > >>>>>> volume
> > > >>>>>>>>>>>>>>>>>> storage.
> > > >>>>>>>>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > > >>> interrogation of
> > > >>>>> the
> > > >>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
> > itself
> > > >>>>> shouldn't
> > > >>>>>>>>> know
> > > >>>>>>>>>>>>>>>>>> or care
> > > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
> > able
> > > >>> to
> > > >>>>>> answer
> > > >>>>>>>>>>>>>>>>>> questions
> > > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> > > >>> driver
> > > >>>>> (which
> > > >>>>>>>>> is a
> > > >>>>>>>>>>>>>>>>>> type of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> > done, a
> > > >>> new
> > > >>>>> Primary
> > > >>>>>>>>>>>>>>>>>> Storage shows
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> > > >>> that
> > > >>>>>> references
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> tag
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> > CloudStack
> > > >>>>> volume. This
> > > >>>>>>>>> leads
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> a new
> > > >>>>>>>>>>>>>>>>>>>>>> row
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> > CloudStack
> > > >>>>> volume to a
> > > >>>>>>>>> VM
> > > >>>>>>>>>>>>>>>>>> (attach
> > > >>>>>>>>>>>>>>>>>>>>>> disk).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
> > calling
> > > >>> the
> > > >>>>> plug-in
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> > > >>> SAN).
> > > >>>>> The
> > > >>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>> also
> > > >>>>>>>>>>>>>>>>>>>>>> updates
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
> > data
> > > >>>>> (like the
> > > >>>>>>>>> IQN of
> > > >>>>>>>>>>>>>>>>>> the SAN
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
> > the
> > > >>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> volume is in
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
> > the
> > > >>>>> volume
> > > >>>>>>>>> will be
> > > >>>>>>>>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
> > the
> > > >>>>> plug-in
> > > >>>>>>>>> code
> > > >>>>>>>>>>>>>>>>>> will not
> > > >>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
> > and
> > > >>>>> detects
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
> > form
> > > >>> of a
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > > >>>>> terminology,
> > > >>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>> in these
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > > >>>>> allocation on
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> > > >>> mean
> > > >>>>> 'SAN
> > > >>>>>>>>> volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
> > user
> > > >>> can
> > > >>>>> make in
> > > >>>>>>>>>>>>>>>>>> CloudStack, I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
> > well
> > > >>> make
> > > >>>>> sense, but
> > > >>>>>>>>> I'm
> > > >>>>>>>>>>>>>>>>>> having a
> > > >>>>>>>>>>>>>>>>>>>>>> hard
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
> > in on
> > > >>>>> this
> > > >>>>>>>>> conversation
> > > >>>>>>>>>>>>>>>>>> as he
> > > >>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > > >>> approach I
> > > >>>>> took.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> > > >>> John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
> > two
> > > >>>>> distinct
> > > >>>>>>>>> concepts
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> > protocols.
> > > >>> In
> > > >>>>> both
> > > >>>>>>>>> cases, we
> > > >>>>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> > system to
> > > >>>>>> create/delete
> > > >>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
> > the
> > > >>>>> volume to
> > > >>>>>> be
> > > >>>>>>>>>>>>>>>>>> volume to be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> > context.
> > > >>> To
> > > >>>>> ensure
> > > >>>>>>>>> that
> > > >>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>> are in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > > >>>>> definitions,
> > > >>>>>>>>> refers to
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> > correct?
> > > >>>>> Minimally,
> > > >>>>>> we
> > > >>>>>>>>>>>>> must
> > > >>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>> able
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
> > device
> > > >>> to
> > > >>>>> move bits
> > > >>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>> one place
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > > >>> Optionally, a
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> > Therefore,
> > > >>>>> we can have
> > > >>>>>> a
> > > >>>>>>>>>>>>>>>>>> unmanaged
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
> > and
> > > >>> we
> > > >>>>> can have a
> > > >>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> > > >>> capable
> > > >>>>> of
> > > >>>>>>>>> allocating
> > > >>>>>>>>>>>>>>>>>> LUNs and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> > CloudStack
> > > >>> may
> > > >>>>> be able
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> manage a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > > >>>>> unmanaged by
> > > >>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
> > the
> > > >>>>> operator has
> > > >>>>>>>>> chosen
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> dedicate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
> > Does
> > > >>> my
> > > >>>>>> reasoning
> > > >>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>> sense?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
> > are
> > > >>>>> reasonable, it
> > > >>>>>>>>> seems
> > > >>>>>>>>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> > concerns
> > > >>> from
> > > >>>>>>>>> StoragePoolType,
> > > >>>>>>>>>>>>>>>>>> add the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> > attached
> > > >>> driver
> > > >>>>> that
> > > >>>>>>>>>>> indicates
> > > >>>>>>>>>>>>>>>>>> whether
> > > >>>>>>>>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> > > >>> establish
> > > >>>>> a
> > > >>>>>>>>> abstraction
> > > >>>>>>>>>>>>>>>>>>>>>> representing a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
> > that
> > > >>> is
> > > >>>>>>>>> associated
> > > >>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>> volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> > hypervisor
> > > >>>>> drivers can
> > > >>>>>>>>>>>>> declare
> > > >>>>>>>>>>>>>>>>>> which
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> > encounter
> > > >>> a
> > > >>>>> device
> > > >>>>>> managed
> > > >>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>> CloudStack,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> > exposed
> > > >>> by
> > > >>>>> the driver to
> > > >>>>>>>>>>>>> automate
> > > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> > > >>> sense,
> > > >>>>> then we can
> > > >>>>>>>>> sit
> > > >>>>>>>>>>>>>>>>>> down and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> > Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> > between
> > > >>> the
> > > >>>>> current
> > > >>>>>> iSCSI
> > > >>>>>>>>>>>>> type
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
> > go in
> > > >>> and
> > > >>>>> create
> > > >>>>>> a
> > > >>>>>>>>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
> > time,
> > > >>>>> the
> > > >>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>> volume must
> > > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> > allocated).
> > > >>>>> Future
> > > >>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
> > created
> > > >>>>> behind the
> > > >>>>>>>>>>>>> scenes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
> > to
> > > >>> go in
> > > >>>>> and
> > > >>>>>> create
> > > >>>>>>>>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
> > delete
> > > >>>>> volumes on
> > > >>>>>>>>> its
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
> > the
> > > >>>>> storage
> > > >>>>>>>>>>>>>>>>>> framework). When
> > > >>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> user
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
> > that
> > > >>> was
> > > >>>>> created,
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>>>> tells
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> > volume.
> > > >>> After
> > > >>>>> this is
> > > >>>>>>>>> done,
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
> > called.
> > > >>>>> No
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> > volume
> > > >>> was
> > > >>>>> just
> > > >>>>>>>>> created.
> > > >>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
> > PM,
> > > >>> Mike
> > > >>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
> > should
> > > >>>>> probably
> > > >>>>>> have
> > > >>>>>>>>>>>>>>>>>> defined them
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> > > >>> pre-
> > > >>>>> allocated
> > > >>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>> an admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> > > >>> hypervisor
> > > >>>>> data
> > > >>>>>>>>> structure
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
> > SR),
> > > >>> then
> > > >>>>> that
> > > >>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
> > Disks
> > > >>> (VDI)
> > > >>>>> are later
> > > >>>>>>>>>>>>> placed
> > > >>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> > > >>> these
> > > >>>>> cases, the
> > > >>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> > > >>> already
> > > >>>>> in place
> > > >>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> > structure
> > > >>> to
> > > >>>>> the VM in
> > > >>>>>>>>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
> > that is
> > > >>> not
> > > >>>>>>>>>>> pre-allocated.
> > > >>>>>>>>>>>>>>>>>> Instead
> > > >>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
> > could
> > > >>> be a
> > > >>>>> SAN
> > > >>>>>> (not
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> volume on
> > > >>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
> > data
> > > >>>>> structure
> > > >>>>>>>>> must be
> > > >>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> > because
> > > >>> these
> > > >>>>> types of
> > > >>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>> have not
> > > >>>>>>>>>>>>>>>>>>>>>>>>> been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > > >>>>> structure by an
> > > >>>>>>>>>>> admin.
> > > >>>>>>>>>>>>>>>>>> Once
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> > XenServer
> > > >>> for
> > > >>>>> this
> > > >>>>>>>>> volume,
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
> > VM
> > > >>> in
> > > >>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
> > PM,
> > > >>>>> John Burwell
> > > >>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> > implementation of
> > > >>> the
> > > >>>>> Dynamic
> > > >>>>>> type
> > > >>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>> behavior
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
> > why I
> > > >>> ask
> > > >>>>> about the
> > > >>>>>>>>>>>>>>>>>> difference.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
> > the
> > > >>>>> definition
> > > >>>>>> of
> > > >>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> > > >>> Mike
> > > >>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
> > type
> > > >>> is
> > > >>>>>> uniquely
> > > >>>>>>>>>>> used
> > > >>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> > > >>> Storage
> > > >>>>> that is
> > > >>>>>>>>> directly
> > > >>>>>>>>>>>>>>>>>> based on
> > > >>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> > > >>> step of
> > > >>>>> going
> > > >>>>>> to
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
> > on
> > > >>> that
> > > >>>>> iSCSI
> > > >>>>>>>>> target as
> > > >>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>> does
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > > >>>>> supported for
> > > >>>>>>>>>>>>>>>>>> XenServer.
> > > >>>>>>>>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
> > first go
> > > >>> to
> > > >>>>> the
> > > >>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>> perform
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
> > RBD is.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 2:13
> > > >>> PM,
> > > >>>>> John
> > > >>>>>> Burwell
> > > >>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
> > code,
> > > >>> what
> > > >>>>> is the
> > > >>>>>>>>>>> difference
> > > >>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
> > isn't
> > > >>> RBD
> > > >>>>> considered
> > > >>>>>>>>>>>>> Dynamic?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
> > PM,
> > > >>> Mike
> > > >>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> > storage is
> > > >>>>> defined in
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > > >>>>> StoragePoolType {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
> > local
> > > >>>>> directory
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > NetworkFilesystem(true),
> > > >>> //
> > > >>>>> NFS or CIFS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> > shared
> > > >>> LUN,
> > > >>>>> with a
> > > >>>>>>>>>>> clusterfs
> > > >>>>>>>>>>>>>>>>>>>> overlay
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
> > ZFS
> > > >>>>> Comstar
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
> > image
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> > XenServer
> > > >>> local
> > > >>>>> LVM SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > SharedMountPoint(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> > VMware
> > > >>> VMFS
> > > >>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > > >>>>> XenServer, Storage
> > > >>>>>>>>> Pool
> > > >>>>>>>>>>>>>>>>>> is set
> > > >>>>>>>>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> > XenServer
> > > >>> local
> > > >>>>> EXT SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> > dynamic,
> > > >>>>> zone-wide
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > StoragePoolType(boolean
> > > >>>>> shared) {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> > isShared() {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 1:41
> > > >>> PM,
> > > >>>>> Mike
> > > >>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> > > >>> another
> > > >>>>> storage
> > > >>>>>>>>>>> company
> > > >>>>>>>>>>>>>>>>>> wants
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> > > >>> Quality
> > > >>>>> of
> > > >>>>>> Service
> > > >>>>>>>>>>>>>>>>>> feature. It
> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
> > well.
> > > >>>>> They would
> > > >>>>>>>>> need
> > > >>>>>>>>>>>>> only
> > > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
> > the
> > > >>>>> necessary
> > > >>>>>>>>> changes to
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
> > in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > > >>> 1:39
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
> > the
> > > >>>>> XenServer code
> > > >>>>>> is
> > > >>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> > attach
> > > >>> logic is
> > > >>>>> now aware
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > > >>>>> implementation of
> > > >>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> > > >>> with
> > > >>>>> Edison's
> > > >>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
> > new
> > > >>>>> framework that
> > > >>>>>>>>>>> supported
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> > dynamically.
> > > >>>>> However, when I
> > > >>>>>>>>>>>>>>>>>> visited with
> > > >>>>>>>>>>>>>>>>>>>>>> him
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
> > that
> > > >>> it
> > > >>>>> was not
> > > >>>>>>>>>>>>>>>>>> complete. We
> > > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
> > could
> > > >>> do
> > > >>>>> with these
> > > >>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>> unless
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> > recognize
> > > >>> this
> > > >>>>> new type of
> > > >>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> > structure.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > > >>> 1:28
> > > >>>>> PM, John
> > > >>>>>>>>> Burwell
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
> > to me
> > > >>>>> that any
> > > >>>>>>>>>>> operation
> > > >>>>>>>>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> > > >>> about
> > > >>>>> details. I
> > > >>>>>>>>> expect
> > > >>>>>>>>>>>>>>>>>> to see
> > > >>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> > > >>> operations
> > > >>>>> that can be
> > > >>>>>>>>>>>>>>>>>>>> composed/driven
> > > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > > >>>>> allocate
> > > >>>>>> space/create
> > > >>>>>>>>>>>>>>>>>> structures
> > > >>>>>>>>>>>>>>>>>>>>>> per
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > > >>>>> dependency, we are
> > > >>>>>>>>> going
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> end
> > > >>>>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
> > make
> > > >>> the
> > > >>>>> system
> > > >>>>>>>>>>>>> increasingly
> > > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > > >>>>> understanding that the
> > > >>>>>>>>> Xen
> > > >>>>>>>>>>>>>>>>>> specific
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > > >>>>> CitrixResourceBase
> > > >>>>>> class?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > > >>> 3:12
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
> > in a
> > > >>> bit
> > > >>>>> more
> > > >>>>>>>>> detail:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
> > aside
> > > >>> from
> > > >>>>> one setup
> > > >>>>>>>>>>>>> method
> > > >>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> > > >>> volume on
> > > >>>>> the
> > > >>>>>> storage
> > > >>>>>>>>>>>>>>>>>> system,
> > > >>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> > > >>> structure
> > > >>>>> to make
> > > >>>>>> use
> > > >>>>>>>>> of
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > > >>> XenServer
> > > >>>>> or a
> > > >>>>>> datastore
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> ESX(i)).
> > > >>>>>>>>>>>>>>>>>>>>>> VMs
> > > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> > storage
> > > >>>>> system's volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> > > >>> storage
> > > >>>>> framework,
> > > >>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> > > >>> create
> > > >>>>> a 1:1
> > > >>>>>>>>>>>>> relationship
> > > >>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
> > VM's
> > > >>> data
> > > >>>>> disk
> > > >>>>>>>>> (necessary
> > > >>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
> > a
> > > >>> plug-
> > > >>>>> in that is
> > > >>>>>>>>> called
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> > needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
> > the
> > > >>>>> storage
> > > >>>>>> framework
> > > >>>>>>>>> did
> > > >>>>>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>>>> address
> > > >>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > > >>> hypervisor-
> > > >>>>> specific data
> > > >>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>> when
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
> > case,
> > > >>> I've
> > > >>>>> been
> > > >>>>>>>>> enhancing
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>>>>>>>> so.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > > >>>>> submitted. I've got
> > > >>>>>>>>> ESX(i)
> > > >>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>>>>>>> sandbox
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
> > 4.2
> > > >>>>> freeze date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
> > bit? :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013 at
> > > >>>>> 1:03 PM, Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> > plug-in -
> > > >>> by
> > > >>>>> itself - is
> > > >>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
> > the
> > > >>>>> volume-attach
> > > >>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>> (in the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
> > into
> > > >>> the
> > > >>>>> plug-in to
> > > >>>>>>>>> have
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> > > >>> comes to
> > > >>>>> attach the
> > > >>>>>>>>> volume
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> > > >>> enough
> > > >>>>> to recognize
> > > >>>>>>>>> it's
> > > >>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> > volume
> > > >>> has
> > > >>>>> just been
> > > >>>>>>>>> created)
> > > >>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> create,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > > >>>>> XenServer) or a
> > > >>>>>>>>> datastore
> > > >>>>>>>>>>>>>>>>>> (for
> > > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
> > just
> > > >>>>> created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> > spending
> > > >>> most
> > > >>>>> of my time
> > > >>>>>>>>>>>>> recently
> > > >>>>>>>>>>>>>>>>>> making
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> > code.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
> > up?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > > >>> at
> > > >>>>> 12:48 PM,
> > > >>>>>> John
> > > >>>>>>>>>>>>>>>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> > > >>> why
> > > >>>>> the the storage
> > > >>>>>>>>>>>>> driver
> > > >>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> > at
> > > >>> 1:21
> > > >>>>> PM, Mike
> > > >>>>>>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> > ultimately I
> > > >>>>> would like to
> > > >>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
> > think
> > > >>> I'm
> > > >>>>> just out of
> > > >>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>> for 4.2
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> > > >>> plug-in
> > > >>>>> supports
> > > >>>>>>>>>>>>> XenServer.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> > > >>> feature
> > > >>>>> freeze, I
> > > >>>>>>>>> have
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> working
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
> > be
> > > >>> clear,
> > > >>>>> this is
> > > >>>>>> all
> > > >>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> regards
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > Compute
> > > >>>>> Offerings post
> > > >>>>>> 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > > >>> 2013
> > > >>>>> at 11:14 AM,
> > > >>>>>>>>> Kelcey
> > > >>>>>>>>>>>>>>>>>> Jamison
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
> > plan
> > > >>> on
> > > >>>>> supporting
> > > >>>>>>>>> KVM in
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > > >>>>> Message -----
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > > >>>>> Tutkowski" <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > > >>>>> dev@cloudstack.apache.org
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> > Monday,
> > > >>> June
> > > >>>>> 3, 2013
> > > >>>>>>>>> 10:12:32 AM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> > > >>> [MERGE]
> > > >>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>> MASTER
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> > > >>> merging
> > > >>>>> Wei's feature
> > > >>>>>>>>>>> first,
> > > >>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>> mine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
> > is
> > > >>> for
> > > >>>>> KVM only,
> > > >>>>>>>>> then
> > > >>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> is a
> > > >>>>>>>>>>>>>>>>>>>> non
> > > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> > 3,
> > > >>> 2013
> > > >>>>> at 8:55 AM,
> > > >>>>>> Wei
> > > >>>>>>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> > billing,
> > > >>> as
> > > >>>>> no one works
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> billing
> > > >>>>>>>>>>>>>>>>>>>> now,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
> > by
> > > >>>>> themselves. They
> > > >>>>>>>>> can
> > > >>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_offering of
> > > >>> a
> > > >>>>> VMs and
> > > >>>>>> volumes
> > > >>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
> > the
> > > >>>>> exact
> > > >>>>>> limitation
> > > >>>>>>>>>>>>> value
> > > >>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > limitation
> > > >>> for
> > > >>>>> nics as
> > > >>>>>> well.
> > > >>>>>>>>> I
> > > >>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
> > I/O
> > > >>>>> throttling?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > > >>>>> misunstand the second
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> > > >>> what
> > > >>>>> you said about
> > > >>>>>> the
> > > >>>>>>>>>>> two
> > > >>>>>>>>>>>>>>>>>>>> features.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> > 2013,
> > > >>> at
> > > >>>>> 2:13 AM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> > Mike's
> > > >>>>> aswer helps you.
> > > >>>>>>>>> I am
> > > >>>>>>>>>>>>>>>>>> trying
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> > think
> > > >>> billing
> > > >>>>> should
> > > >>>>>>>>> depend
> > > >>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>> IO
> > > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > limitation.
> > > >>> Please
> > > >>>>> review
> > > >>>>>>>>>>>>>>>>>> disk_io_stat if
> > > >>>>>>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
> > IO
> > > >>>>> statistics
> > > >>>>>> including
> > > >>>>>>>>>>>>>>>>>> bytes/iops
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> > > >>> machine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
> > the
> > > >>> AWS
> > > >>>>> model,
> > > >>>>>> customers
> > > >>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>> billed
> > > >>>>>>>>>>>>>>>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > provisioned
> > > >>> IOPS,
> > > >>>>> as well as,
> > > >>>>>> for
> > > >>>>>>>>>>>>> those
> > > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://aws.amazon.com/ebs/).
> > > >>>>>> I
> > > >>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>> imagine
> > > >>>>>>>>>>>>>>>>>>>>>>>>> our
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> > > >>> employ
> > > >>>>> similar cost
> > > >>>>>>>>>>> models.
> > > >>>>>>>>>>>>>>>>>> Could
> > > >>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
> > model
> > > >>> in
> > > >>>>> the current
> > > >>>>>>>>> patch?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
> > you
> > > >>> mean
> > > >>>>> IOPS runtime
> > > >>>>>>>>>>> change?
> > > >>>>>>>>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > limitation for
> > > >>> a
> > > >>>>> running
> > > >>>>>> virtual
> > > >>>>>>>>>>>>>>>>>> machine
> > > >>>>>>>>>>>>>>>>>>>>>> through
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > CloudStack
> > > >>> does
> > > >>>>> not support
> > > >>>>>>>>>>> changing
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (computer
> > > >>>>> offering or disk
> > > >>>>>>>>>>>>> offering).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
> > the
> > > >>>>> Java interface
> > > >>>>>>>>> level.
> > > >>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>> apologize
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > > >>>>> generalize allocation
> > > >>>>>>>>>>>>>>>>>> algorithms
> > > >>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
> > the
> > > >>>>> service guarantees
> > > >>>>>>>>>>>>>>>>>> provided by a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> > > >>> good
> > > >>>>> question.
> > > >>>>>>>>> Maybe it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> disk_io_throttling as Mike
> > > >>>>>>>>> needs to
> > > >>>>>>>>>>>>>>>>>> consider
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
> > think.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> > expand
> > > >>> on
> > > >>>>> my thoughts
> > > >>>>>> in a
> > > >>>>>>>>>>>>> later
> > > >>>>>>>>>>>>>>>>>>>>>> response
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
> > points
> > > >>>>> between these two
> > > >>>>>>>>>>>>>>>>>> features. I
> > > >>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
> > to be
> > > >>>>> merged before
> > > >>>>>>>>>>>>>>>>>> SolidFire, but
> > > >>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > coordination
> > > >>>>> between the
> > > >>>>>> branches
> > > >>>>>>>>>>>>>>>>>> (possibly
> > > >>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> disk_io_throttling)
> > > >>>>> to
> > > >>>>>>>>> coordinate on
> > > >>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>> issue.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > > >>> John
> > > >>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > things I
> > > >>>>> want to
> > > >>>>>> understand
> > > >>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
> > there
> > > >>>>> value in
> > > >>>>>> capturing
> > > >>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>> policies
> > > >>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > model
> > > >>> (e.g.
> > > >>>>> for
> > > >>>>>>>>> billing/usage
> > > >>>>>>>>>>>>>>>>>>>> purposes,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> > Should
> > > >>> there
> > > >>>>> be a common
> > > >>>>>>>>>>>>>>>>>> interface model
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > provisioning
> > > >>> at
> > > >>>>> runtime?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
> > are
> > > >>>>> conflicting
> > > >>>>>>>>> provisioned
> > > >>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > hypervisor
> > > >>> and
> > > >>>>> storage device
> > > >>>>>>>>>>>>>>>>>> reconciled?
> > > >>>>>>>>>>>>>>>>>>>> In
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
> > to
> > > >>>>> believe (and
> > > >>>>>> billed)
> > > >>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>> device
> > > >>>>> has been
> > > >>>>>>>>> configured
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> deliver.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > consistent
> > > >>>>> configuration
> > > >>>>>>>>> between a
> > > >>>>>>>>>>>>>>>>>> VM and a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
> > but a
> > > >>>>> later
> > > >>>>>> modification
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> inconsistency.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
> > 2,
> > > >>> 2013,
> > > >>>>> at 8:38 PM,
> > > >>>>>>>>> Mike
> > > >>>>>>>>>>>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > believe
> > > >>> Wei's
> > > >>>>> feature deals
> > > >>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>> controlling
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>> hypervisor
> > > >>>>> side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > feature
> > > >>> is
> > > >>>>> focused on
> > > >>>>>>>>>>>>> controlling
> > > >>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> > that
> > > >>>>> helps. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
> > Jun
> > > >>> 2,
> > > >>>>> 2013 at 6:35
> > > >>>>>> PM,
> > > >>>>>>>>>>>>> John
> > > >>>>>>>>>>>>>>>>>>>> Burwell
> > > >>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > > >>> opinion is
> > > >>>>> that no
> > > >>>>>> features
> > > >>>>>>>>>>>>>>>>>> should be
> > > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> > > >>> have
> > > >>>>> been resolved
> > > >>>>>> and
> > > >>>>>>>>> it
> > > >>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>> ready
> > > >>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > Ops
> > > >>> vs
> > > >>>>> discrete
> > > >>>>>>>>> read/write
> > > >>>>>>>>>>>>>>>>>> ops issue
> > > >>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> > reviewed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > I
> > > >>> don't
> > > >>>>> think this
> > > >>>>>>>>> criteria
> > > >>>>>>>>>>>>>>>>>> has been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> > how
> > > >>>>> does this work
> > > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> > > >>>>>>>>>>>>>>>>>> )?
> > > >>>>>>>>>>>>>>>>>>>> As I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> > > >>> involves
> > > >>>>> provisioned
> > > >>>>>>>>> IOPS. I
> > > >>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > scenario
> > > >>>>> where provisioned
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> KVM and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > unnecessarily
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> incompatible.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > Jun 1,
> > > >>> 2013,
> > > >>>>> at 6:47 AM,
> > > >>>>>>>>> Wei
> > > >>>>>>>>>>>>>>>>>> ZHOU <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
> > will
> > > >>>>> change it next
> > > >>>>>>>>> week.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/6/1
> > > >>>>> Wido den Hollander
> > > >>>>>> <
> > > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 06/01/2013 08:24 AM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Exactly. I
> > > >>> have
> > > >>>>> pushed the
> > > >>>>>>>>>>>>> features
> > > >>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > anyone
> > > >>>>> object thems for
> > > >>>>>>>>>>>>> technical
> > > >>>>>>>>>>>>>>>>>>>> reason
> > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> > the
> > > >>> sake
> > > >>>>> of clarity I
> > > >>>>>> just
> > > >>>>>>>>>>>>> want
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> mention
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > total
> > > >>> IOps
> > > >>>>> to R/W IOps
> > > >>>>>>>>> asap
> > > >>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>>>>>>>> never
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > total
> > > >>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
> > laid
> > > >>> the
> > > >>>>> groundwork for
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > however
> > > >>>>> prevent that we
> > > >>>>>> create
> > > >>>>>>>>>>>>>>>>>> legacy from
> > > >>>>>>>>>>>>>>>>>>>>>> day
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/5/31
> > > >>>>> Wido den
> > > >>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/31/2013 03:59 PM, John
> > > >>>>>>>>>>>>> Burwell
> > > >>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
> > this
> > > >>>>> enhancement must
> > > >>>>>> to
> > > >>>>>>>>>>>>>>>>>> discretely
> > > >>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > see
> > > >>> how
> > > >>>>> it could be
> > > >>>>>>>>> fixed
> > > >>>>>>>>>>>>>>>>>> later
> > > >>>>>>>>>>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > correctly
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
> > total
> > > >>> IOPS
> > > >>>>> into read
> > > >>>>>> and
> > > >>>>>>>>>>>>>>>>>> write.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>> unless/until we
> > > >>>>>> decided
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> break
> > > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
> > Wei
> > > >>>>> meant was merging
> > > >>>>>> it
> > > >>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>>>>>>>> now
> > > >>>>>>>>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > branch
> > > >>>>> and add Read /
> > > >>>>>>>>> Write
> > > >>>>>>>>>>>>> IOps
> > > >>>>>>>>>>>>>>>>>>>> before
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > > >>>>> released with Read
> > > >>>>>> and
> > > >>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>> instead
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
> > to
> > > >>>>> make the May 31st
> > > >>>>>>>>>>>>> feature
> > > >>>>>>>>>>>>>>>>>>>> freeze
> > > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
> > other
> > > >>>>> threads) then it
> > > >>>>>>>>> won't
> > > >>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>> necessary
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > > >>>>> completely agree that
> > > >>>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>>> is no
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > I/O.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > May 31,
> > > >>>>> 2013, at 9:51 AM,
> > > >>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>> den
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/31/2013 03:13 PM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> > Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Thanks.
> > > >>> Good
> > > >>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > thought
> > > >>>>> about at the
> > > >>>>>>>>>>> beginning.
> > > >>>>>>>>>>>>>>>>>>>> Finally I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > difference
> > > >>> of
> > > >>>>> read and write
> > > >>>>>>>>>>>>> mainly
> > > >>>>>>>>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
> > the
> > > >>>>> difference of sent
> > > >>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>> received
> > > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > > >>>>> reasoning seems odd.
> > > >>>>>>>>>>>>>>>>>> Networking and
> > > >>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > I/O is
> > > >>>>> much more
> > > >>>>>>>>> expensive
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>> most
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > bandwith.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> Implementing
> > > >>>>> it will be some
> > > >>>>>>>>>>>>>>>>>> copy-paste
> > > >>>>>>>>>>>>>>>>>>>>>> work.
> > > >>>>>>>>>>>>>>>>>>>>>>>>> It
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> implemented
> > > >>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
> > days.
> > > >>> For
> > > >>>>> the deadline
> > > >>>>>> of
> > > >>>>>>>>>>>>>>>>>> feature
> > > >>>>>>>>>>>>>>>>>>>>>> freeze,
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
> > if
> > > >>>>> needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> > think it's
> > > >>> a
> > > >>>>> feature we
> > > >>>>>>>>> can't
> > > >>>>>>>>>>>>>>>>>> miss. But
> > > >>>>>>>>>>>>>>>>>>>>>> if
> > > >>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > window
> > > >>> we
> > > >>>>> have to make sure
> > > >>>>>> we
> > > >>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
> > 4.3,
> > > >>> that
> > > >>>>> will confuse
> > > >>>>>>>>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > 2013/5/31
> > > >>>>> Wido den
> > > >>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > >>>>> 05/30/2013 06:03 PM, Wei
> > > >>>>>>>>> ZHOU
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > would
> > > >>> like to
> > > >>>>> merge
> > > >>>>>>>>>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > nobody
> > > >>>>> object, I will
> > > >>>>>> merge
> > > >>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>> purpose
> > > >>>>> is :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > > >>>>> machines are running
> > > >>>>>>>>> on
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>> same
> > > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > storage or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> > > >>> strage).
> > > >>>>> Because of
> > > >>>>>> the
> > > >>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>>>> limitation
> > > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
> > if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
> > VM
> > > >>> has
> > > >>>>> large disk
> > > >>>>>>>>> operation,
> > > >>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>> affect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > VMs
> > > >>>>> running on the
> > > >>>>>> same
> > > >>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> device.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > >>> neccesary
> > > >>>>> to set the
> > > >>>>>>>>>>> maximum
> > > >>>>>>>>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Looking at
> > > >>> the
> > > >>>>> code I see
> > > >>>>>> you
> > > >>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
> > and
> > > >>>>> libvirt support
> > > >>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>> both a
> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > >>> which
> > > >>>>> could benefit a
> > > >>>>>>>>> lot of
> > > >>>>>>>>>>>>>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
> > also
> > > >>>>> strange, in the
> > > >>>>>>>>> polling
> > > >>>>>>>>>>>>>>>>>> side you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > but
> > > >>> on
> > > >>>>> the throttling
> > > >>>>>>>>> side
> > > >>>>>>>>>>>>>>>>>> you only
> > > >>>>>>>>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > IOps
> > > >>> are
> > > >>>>> usually much
> > > >>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>> expensive
> > > >>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
> > valid
> > > >>>>> use-case where
> > > >>>>>>>>> that
> > > >>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > vs
> > > >>> Read
> > > >>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
> > this
> > > >>> only
> > > >>>>> supports KVM
> > > >>>>>>>>> at
> > > >>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>> point I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > to at
> > > >>>>> least have the
> > > >>>>>>>>>>>>>>>>>> mechanism in
> > > >>>>>>>>>>>>>>>>>>>>>> place
> > > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> implementing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > later
> > > >>>>> would be a lot of
> > > >>>>>>>>>>> work.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> > > >>> hypervisor
> > > >>>>> doesn't
> > > >>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > you
> > > >>> can
> > > >>>>> always sum
> > > >>>>>> both
> > > >>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>> and set
> > > >>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > you
> > > >>>>> explain why you
> > > >>>>>>>>>>>>> implemented
> > > >>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>> feature
> > > >>>>> includes:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> > the
> > > >>>>> maximum rate of
> > > >>>>>>>>> VMs
> > > >>>>>>>>>>>>> (in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> configuration)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > change
> > > >>> the
> > > >>>>> maximum rate
> > > >>>>>> of
> > > >>>>>>>>>>> VMs
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > limit
> > > >>> the
> > > >>>>> disk rate
> > > >>>>>> (total
> > > >>>>>>>>>>> bps
> > > >>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>> iops)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
> > ticket:
> > > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> jira/browse/CLOUDSTACK-1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>> 1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 1192
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
> > will
> > > >>>>> update later) :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>> VM+Disk+IO+Throttling<https://
> > > >>>>>>>>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>
> > > >>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > >>>>> rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Merge
> > > >>> check
> > > >>>>> list :-
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
> > you
> > > >>>>> check the branch's
> > > >>>>>>>>> RAT
> > > >>>>>>>>>>>>>>>>>> execution
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> > > >>> there
> > > >>>>> new dependencies
> > > >>>>>>>>>>>>>>>>>> introduced?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > What
> > > >>>>> automated testing
> > > >>>>>> (unit
> > > >>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > feature?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
> > tests
> > > >>> are
> > > >>>>> added.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> > What
> > > >>>>> testing has been done
> > > >>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>> check for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> > the
> > > >>>>> bytes rate and
> > > >>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>> rate on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> > VM
> > > >>>>> operations, including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > deploy,
> > > >>> stop,
> > > >>>>> start, reboot,
> > > >>>>>>>>>>>>>>>>>> destroy,
> > > >>>>>>>>>>>>>>>>>>>>>> expunge.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > > >>> Volume
> > > >>>>> operations,
> > > >>>>>>>>> including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > > >>>>> Detach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> > review
> > > >>> the
> > > >>>>> code, you can
> > > >>>>>>>>> try
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> > > >>> regards,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>> VM+Disk+IO+Throttling<https://
> > > >>>>>>>>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>
> > > >>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > >>>>> rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > > >>>>>>>>> refs/heads/disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>>>> 1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 1301
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > issues.apache.org/****jira/**browse/CLOUDSTACK-
> > > >>> 2071
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> > 2071
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > >>>>> 2071
> > > >>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > >>>>> 2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> <h**ttps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> > 2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> (**CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> > VM
> > > >>> Disk
> > > >>>>> I/O
> > > >>>>>> Throttling)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > >>>>> Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>>>> CloudStack Developer,
> > > >>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > > >>> 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Advancing
> > > >>> the
> > > >>>>> way the world
> > > >>>>>>>>> uses
> > > >>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > >>> Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>> CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > > >>> way
> > > >>>>> the world uses
> > > >>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > >>> CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > > >>> way
> > > >>>>> the world uses
> > > >>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > >>>>> Developer,
> > > >>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > >>> the
> > > >>>>> world uses the
> > > >>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > >>>>> Developer,
> > > >>>>>> SolidFire
> > > >>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > >>> the
> > > >>>>> world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>>>> Developer, SolidFire
> > > >>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > >>>>> world uses the
> > > >>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>> Developer,
> > > >>>>> SolidFire
> > > >>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > >>> world
> > > >>>>> uses the
> > > >>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > >>> Developer,
> > > >>>>> SolidFire
> > > >>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > > >>> world
> > > >>>>> uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>>>> SolidFire
> > > >>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > >>> uses
> > > >>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>>>> SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > >>> uses
> > > >>>>> the cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>>>> SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > >>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > >>> the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > >>> SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>> *(tm)*
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> *Mike Tutkowski*
> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>> o: 303.746.7302
> > > >>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>> *(tm)*
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> *Mike Tutkowski*
> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>> o: 303.746.7302
> > > >>>>>>> Advancing the way the world uses the
> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>> *(tm)*
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> *Mike Tutkowski*
> > > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>> e: mike.tutkowski@solidfire.com
> > > >>>>> o: 303.746.7302
> > > >>>>> Advancing the way the world uses the
> > > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>> *(tm)*
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> *Mike Tutkowski*
> > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> e: mike.tutkowski@solidfire.com
> > > >>> o: 303.746.7302
> > > >>> Advancing the way the world uses the
> > > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>> *(tm)*
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkowski@solidfire.com
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > *(tm)*
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Friday, June 07, 2013 4:08 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Hi John,
>
> Yes, you are correct that Xen must layer an SR on top of the iSCSI volume
> to use it. Same story for VMware: ESX(i) must layer a datastore on top of
> the iSCSI volume to use it. I look at it like they are layering a clustered
> file system on the SAN volume so the hypervisors can share access to the
> contents of the volume.
>
> In 4.1, by the time the hypervisor-attach-volume logic is called, the SR or
> datastore has already been created (usually manually by an admin).
>
> This pre-setup of the, say, SR is, of course, not acceptable in an
> environment where each CloudStack volume that a user creates is mapped
> to a
> single SAN volume (via the SR).
>
> The question comes down to who should allocate the SR dynamically.
>
> We could have the storage framework ask the storage plug-in if it is
> managed. If it is, then the storage framework could send a message to the
> hypervisor in question to create (let's talk Xen here) the SR ahead of
> time. Then, when the storage framework next sends the attach-volume
> command
> to the hypervisor, it should work without changes to that attach-volume
> logic (because - from the point of view of the attach logic - the SR is
> already existent, as expected).
>
> Now, as Edison has pointed out, this limits the power of the storage
> plug-in. A storage plug-in in this model cannot directly attach storage to
> a VM (it must go through the hypervisor). Perhaps that is OK. We need to
> make a call on that.
Yes, I think we need to address this usage case, and won't limit the power of the storage plug-in.
John, what's your idea to fit this usage case into your abstraction?
>
> If we want to give more power to the storage vendor, we would have to
> have
> the storage framework call into the storage plug-in to send the appropriate
> attach commands to the hypervisor.
>
> Let's discuss and come to a consensus (or at least agree on a path) as soon
> as we can. :)
>
> Thanks!
>
>
>
> On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com> wrote:
>
> > Mike,
> >
> > My understanding is the Xen expects an SR structure on all ISCSI devices
> > -- at least that is how I read the code in your patch. Is my understanding
> > correct? If so, the Xen plugin should be able to query the storage device
> > to determine the presence of the SR structure and create it if it does not
> > exist. Am I missing something in the implementation that makes that type
> > of implementation impossible?
> >
> > Thanks,
> > -John
> >
> > On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <mi...@solidfire.com>
> > wrote:
> >
> > > Yeah, if a storage vendor wanted to attach a volume directly to a VM in
> > 4.2
> > > today, it would probably fail because the attach-volume logic assumes
> the
> > > existence of the necessary hypervisor data structure (ex. SR on Xen).
> > >
> > > If we wanted to enable such an attach, we could do it the way Edison
> > > suggests.
> > >
> > >
> > > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com> wrote:
> > >
> > >>
> > >>
> > >>> -----Original Message-----
> > >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >>> Sent: Friday, June 07, 2013 2:37 PM
> > >>> To: dev@cloudstack.apache.org
> > >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >>>
> > >>> As we only have three weeks until feature freeze, we should come to
> a
> > >>> consensus on this design point as soon as possible.
> > >>>
> > >>> Right now, if the storage framework asks my driver it is is managed, it
> > >>> will say 'yes.' This means the framework will tell the driver to
> > perform
> > >>> its management activities. This then means the driver will call into
> > the
> > >>> host (it doesn't know which hypervisor, by the way) to perform the
> > >> activity
> > >>> of, say, creating an SR on XenServer or a datastore on ESX.
> > >>>
> > >>> The driver doesn't know which hypervisor it's talking to, it just
> > sends a
> > >>> message to the host to perform the necessary pre-attach work.
> > >>
> > >> Could we just expose a method like "attachVolume/dettachVolume" on
> the
> > >> PrimaryDataStoreDriver driver?
> > >> For most of cases, the implementation of each driver would just send
> > >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put
> the
> > >> implementation in a base class, so that can be shared by all of this
> > >> drivers), and in each hypervisor resource code, which may just call
> > >> hypervisor's api to attach the volume to VM, while for certain storage,
> > >> like SolidFire, may need to create a SR first, and create volume on it,
> > >> then call hypervisor's API to attach volume to VM.
> > >> While for some other storage vendor, may want to bypass hypervisor
> > during
> > >> attaching volume, so inside driver's attachvolume implementation, the
> > >> driver can do some magic, such as, directly talk to an agent inside VM
> > >> instance, then create a disk inside VM.
> > >>
> > >> How do you guys think?
> > >>
> > >>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> > wrote:
> > >>>
> > >>>>
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > >>>>> Sent: Friday, June 07, 2013 1:14 PM
> > >>>>> To: dev@cloudstack.apache.org
> > >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >>>>>
> > >>>>> Hi John,
> > >>>>>
> > >>>>> How's about this:
> > >>>>>
> > >>>>> The driver can implement an isManaged() method. The
> > >>> VolumeManagerImpl
> > >>>>> can
> > >>>>> call into the driver to see if its managed. If it is, the
> > >>>> VolumeManagerImpl
> > >>>>> (which is responsible for calling into the hypervisor to attach the
> > >> disk)
> > >>>>> can call into the hypervisor to create the necessary hypervisor data
> > >>>>> structure (ex. for XenServer, a storage repository).
> > >>>>
> > >>>> The problem here is that storage vendor may work differently with
> > >>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> > >> other
> > >>>> vendor wants to totally bypass hypervisor, and assign the LUN
> directly
> > >> to
> > >>>> VM instance, see the discuss(
> > >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> > >>>
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> > >>> %3E
> > >>>> ).
> > >>>> So I would let storage provider to implement attach disk to VM,
> > >> instead of
> > >>>> implemented by cloudstack itself.
> > >>>>
> > >>>>
> > >>>>>
> > >>>>> If that's what you're going for, that works for me. By the way,
> > >> Edison's
> > >>>>> default storage plug-in (which handles the default storage behavior
> > >> in
> > >>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
> > >>>>> hypervisors. You might want to contact him and inform him of your
> > >>>> concerns
> > >>>>> or that logic (as is) will make it to production.
> > >>>>>
> > >>>>> Please let me know if what I wrote in above (for my solution) is OK
> > >> with
> > >>>>> you. :)
> > >>>>>
> > >>>>> Thanks!
> > >>>>>
> > >>>>>
> > >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Mike,
> > >>>>>>
> > >>>>>> Please see my responses in-line below.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> -John
> > >>>>>>
> > >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > >>>> mike.tutkowski@solidfire.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hey John,
> > >>>>>>>
> > >>>>>>> I still have a bit more testing I'd like to do before I build up
> > >> a
> > >>>> patch
> > >>>>>>> file, but this is the gist of what I've done:
> > >>>>>>>
> > >>>>>>> * During a volume-attach operation, after VolumeManagerImpl
> tells
> > >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
> > >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> > >>>>> managed.
> > >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> > >> whatever
> > >>>>>>> activity is required. In my case, this includes sending a
> > >> message to
> > >>>> the
> > >>>>>>> host where the VM is running to have, say XenServer, add a
> > >> storage
> > >>>>>>> repository (based on the IP address of the SAN, the IQN of the
> > >> SAN
> > >>>>>> volume,
> > >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can
> > >> on
> > >>>> the
> > >>>>>>> storage repository). After this, the normal attach-volume
> > >> message is
> > >>>> sent
> > >>>>>>> to the host by VolumeManagerImpl.
> > >>>>>>
> > >>>>>> There should be **no** code from a storage driver to a hypervisor.
> > >> I
> > >>>>>> apologize for the repetition, but we simply can not have hypervisor
> > >>>>>> specific code in the storage layer. The circular dependencies
> > >> between
> > >>>> the
> > >>>>>> two layers are not sustainable in the long term. Either the
> > >>>> VirtualManager
> > >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> > >>> coordinate
> > >>>>>> volume creation and then populating the SR. Ideally, we can
> > >>>> generalize the
> > >>>>>> process flow for attaching volumes such that the Xen hypervisor
> > >> plugin
> > >>>>>> would only implement callbacks to perform the attach action and
> > >> create
> > >>>>> the
> > >>>>>> structure and SR. To my mind, the SolidFire driver should only be
> > >>>>>> allocating space and providing information about contents (e.g.
> > >> space
> > >>>>>> available, space consumed, streams to a URI, file handle for a URI,
> > >>>> etc)
> > >>>>>> and capabilities.
> > >>>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>> * The reverse is performed for a detach-volume command.
> > >>>>>>>
> > >>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
> > >>>>>> Edison's
> > >>>>>>> default driver simply does a "return false;". We could add a new
> > >>>>>> parameter
> > >>>>>>> to the createStoragePool API command, if we want, to remove
> the
> > >>>>>> hard-coded
> > >>>>>>> return values in the drivers (although my driver will probably
> > >> just
> > >>>>>> ignore
> > >>>>>>> this parameter and always return true since it wouldn't make
> > >> sense
> > >>>> for it
> > >>>>>>> to ever return false). We'd need another column in the
> > >> storage_pool
> > >>>>> table
> > >>>>>>> to store this value.
> > >>>>>>
> > >>>>>> Yes, I think we should have a parameter added to the
> > >>> createStoragePool
> > >>>>>> surfaced to the HTTP API that allows DataStores to be configured
> > >> for
> > >>>>>> management when their underlying drivers support it. To simplify
> > >>>> things,
> > >>>>>> this flag should only be mutable when the DataStore is created. It
> > >>>> would be
> > >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
> > >>>> creation.
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Sound like I'm in sync with what you were thinking?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > >>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> I agree, John. Just wanted to point out that I have a working
> > >> GUI
> > >>>> for
> > >>>>>> you
> > >>>>>>>> to review (in that document), if you'd like to check it out.
> > >>>>>>>>
> > >>>>>>>> Thanks!
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> > >> jburwell@basho.com>
> > >>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Mike,
> > >>>>>>>>>
> > >>>>>>>>> I would like the UIs of two features reviewed together to
> > >> ensure
> > >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
> > >> and
> > >>>>>>>>> storage device provisioned IOPs. I see the potential for
> > >>>> confusion,
> > >>>>>>>>> and I think a side-by-side Ui review of these features will
> > >> help
> > >>>>>>>>> minimize any potential confusion.
> > >>>>>>>>>
> > >>>>>>>>> As I mentioned, the term reconciliation issue will work itself
> > >> if
> > >>>> it
> > >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> > >>>> throttled
> > >>>>>>>>> IOPs or storage provisioned IOPs.
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> -John
> > >>>>>>>>>
> > >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > >>>>>>>>> <mi...@solidfire.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi John,
> > >>>>>>>>>>
> > >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent
> to
> > >>> you
> > >>>>> the
> > >>>>>>>>>> other day to see how the GUI looks for provisioned storage
> > >> IOPS.
> > >>>>>>>>>>
> > >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> > >> and we
> > >>>>>>>>> decided
> > >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk
> Offering
> > >>>> dialog.
> > >>>>>>>>>> Other storage vendors are coming out with QoS, so they
> should
> > >>> be
> > >>>>> able
> > >>>>>> to
> > >>>>>>>>>> leverage this GUI going forward (even if they, say, only use
> > >> Max
> > >>>>>> IOPS).
> > >>>>>>>>>> These fields are optional and can be ignored for storage that
> > >>>> does not
> > >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> > >>>> admin can
> > >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
> > >> IOPS.
> > >>>>>>>>>>
> > >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
> > >> mine,
> > >>>> as
> > >>>>>> is
> > >>>>>>>>>> decided by the admin).
> > >>>>>>>>>>
> > >>>>>>>>>> I'm not sure what we can do about these two features
> > >> expressing
> > >>>> the
> > >>>>>>>>> speed
> > >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
> > >>>> QoS in
> > >>>>>>>>> any
> > >>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
> > >>>> IOPS).
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > >>> <jb...@basho.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Wei,
> > >>>>>>>>>>>
> > >>>>>>>>>>> We have been down the rabbit hole a bit on the
> > >>> Storage/Hypervisor
> > >>>>>> layer
> > >>>>>>>>>>> separation, but we still need to reconcile the behavior of
> > >>>> hypervisor
> > >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> > >> following
> > >>>>>> issues
> > >>>>>>>>>>> outstanding:
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > >>> read/write
> > >>>>>>>>> values
> > >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
> > >> IOPS.
> > >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> > >>> storage
> > >>>>>> volumes
> > >>>>>>>>>>> with provisioned IOPS?
> > >>>>>>>>>>> 3. How should usage data be captured for throttled and
> > >>>> provisioned
> > >>>>>> IOPS
> > >>>>>>>>>>> that will permit providers to discriminate these guaranteed
> > >>>>>> operations
> > >>>>>>>>> in
> > >>>>>>>>>>> the event they want to bill for it differently?
> > >>>>>>>>>>> 4. What is the user experience for throttled and provisioned
> > >>> IOPS
> > >>>>>> that
> > >>>>>>>>>>> minimizes confusion of these concepts?
> > >>>>>>>>>>>
> > >>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> > >>>> throttled
> > >>>>>>>>> IOPS
> > >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
> > >>>> items 1
> > >>>>>>>>> and 2.
> > >>>>>>>>>>> Since the two facilities would not be permitted to operate
> > >>>> together,
> > >>>>>>>>> they
> > >>>>>>>>>>> do not need to be semantically compatible. I think item 3
> > >> can be
> > >>>>>>>>> resolved
> > >>>>>>>>>>> with an additional flag or two on the usage records. As for
> > >>>> Item 4,
> > >>>>>> I
> > >>>>>>>>> am
> > >>>>>>>>>>> not familiar with how these two enhancements are
> depicted in
> > >>> the
> > >>>>> user
> > >>>>>>>>>>> interface. I think we need to review the UI enhancements
> for
> > >>>> both
> > >>>>>>>>>>> enhancements and ensure they are consistent.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Do these solutions make sense?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> -John
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> <us...@gmail.com>
> > >>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> John and Mike,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-
> 2780/2729,
> > >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > >>> CLOUDSTACK-
> > >>>>> 2875 ) this
> > >>>>>>>>> week.
> > >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
> > >> ask
> > >>>> for
> > >>>>>>>>> second
> > >>>>>>>>>>>> merge request after testing.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> I believe I understand where you're going with this, John,
> > >> and
> > >>>>> have
> > >>>>>>>>> been
> > >>>>>>>>>>>>> re-working this section of code today.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for the comments,
> > >>>>>>>>>>>>> Mike
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > >>>>> <jb...@basho.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> See my responses in-line below.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks for the response.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> "I am fine with the VolumeManager determining
> whether or
> > >>>>> not a
> > >>>>>>>>> Volume
> > >>>>>>>>>>> is
> > >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> > >>> actual
> > >>>>>>>>> isManaged
> > >>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> resources
> > >>>> for
> > >>>>>> the
> > >>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>> if it is managed."
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged()
> method
> > >>>>> added to
> > >>>>>>>>> the
> > >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> > >> then the
> > >>>>>>>>> storage
> > >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
> > >>> method
> > >>>>> (which
> > >>>>>>>>>>> would
> > >>>>>>>>>>>>> be
> > >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well)
> and
> > >>> this
> > >>>>>> would
> > >>>>>>>>>>> call
> > >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say
> on
> > >>>>>>>>> XenServer,
> > >>>>>>>>>>> an
> > >>>>>>>>>>>>> SR?
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> > >>>>> DataStoreDriver.
> > >>>>>>>>> Since
> > >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage,
> I
> > >>>> would
> > >>>>>> add
> > >>>>>>>>> a
> > >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a
> straw
> > >>> man
> > >>>>> term
> > >>>>>>>>> --
> > >>>>>>>>>>>>>> something other than volume) methods to
> > >>>>>>>>>>> allocate/create/deallocate/delete
> > >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> > >>> property
> > >>>>> of
> > >>>>>>>>>>>>> DataStore
> > >>>>>>>>>>>>>> which can be enabled if/when the underlying
> > >>> DataStoreDriver
> > >>>>> can be
> > >>>>>>>>>>>>> managed.
> > >>>>>>>>>>>>>> This approach allows operators to override
> manageability
> > >> of
> > >>>>>> devices.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> > >> plugin
> > >>>>>> would
> > >>>>>>>>> be
> > >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods
> to
> > >>>>> create any
> > >>>>>>>>>>>>>> directories or files necessary for the SR. There should
> > >> be no
> > >>>>>>>>>>>>> dependencies
> > >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> > >> earlier,
> > >>>> such
> > >>>>>>>>>>>>> circular
> > >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable
> mess.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks again, John
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > >>>>> <jb...@basho.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer
> that
> > >>>>>> supports n
> > >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> > >> hypervisors.
> > >>>>>> Such
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> > >>> beast.
> > >>>>>>>>>>>>> Therefore, my
> > >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve
> the
> > >>>>> Storage
> > >>>>>> layer
> > >>>>>>>>>>>>> towards
> > >>>>>>>>>>>>>>> this separation.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I am fine with the VolumeManager determining
> whether
> > >>> or
> > >>>>> not a
> > >>>>>>>>> Volume
> > >>>>>>>>>>> is
> > >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but
> an
> > >>> actual
> > >>>>>>>>>>> isManaged
> > >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> > >> resources
> > >>>> for
> > >>>>>> the
> > >>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
> > >> to
> > >>>>>> indicate
> > >>>>>>>>>>>>> whether
> > >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
> > >> think
> > >>>> we
> > >>>>>>>>> need to
> > >>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
> > >>>>>> something
> > >>>>>>>>>>>>> that is
> > >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> > >>> managing
> > >>>>> the
> > >>>>>>>>>>> device,
> > >>>>>>>>>>>>>>> but the administrator has elected to leave it
> unmanaged).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Creation of a structure on the volume should be done
> in
> > >>> the
> > >>>>> Xen
> > >>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> Storage
> > >>>>> layer to
> > >>>>>>>>>>> perform
> > >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
> > >> file,
> > >>>> etc).
> > >>>>>>>>>>> This
> > >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> > >>>> hypervisor, as
> > >>>>>>>>> such,
> > >>>>>>>>>>>>>>> should be confined to its implementation. From my
> > >>>>> perspective,
> > >>>>>>>>>>> nothing
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>> the Storage layer should be concerned with content.
> From
> > >>> its
> > >>>>>>>>>>>>> perspective,
> > >>>>>>>>>>>>>>> structure and data are opaque. It provides the means
> to
> > >>>> query
> > >>>>>> the
> > >>>>>>>>>>> data
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>> support the interpretation of the content by higher-
> level
> > >>>>> layers
> > >>>>>>>>> (e.g.
> > >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a
> composition
> > >>> of
> > >>>>>>>>>>> operations
> > >>>>>>>>>>>>>>> from the Storage layer that varies based on the
> Volume
> > >>>>> storage
> > >>>>>>>>>>> protocol
> > >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Alternatively to the way the attach logic is
> implemented
> > >> in
> > >>>> my
> > >>>>>>>>> patch,
> > >>>>>>>>>>> we
> > >>>>>>>>>>>>>>> could do the following:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> > >>>>>>>>> VolumeManagerImpl
> > >>>>>>>>>>> we
> > >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> > >>>>> hypervisor.
> > >>>>>> Before
> > >>>>>>>>>>> this
> > >>>>>>>>>>>>>>> command is sent, we could check to see if we're
> dealing
> > >>> with
> > >>>>>>>>> Dynamic
> > >>>>>>>>>>> (or
> > >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
> > >>>> "Create
> > >>>>>>>>> SR"
> > >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we
> would
> > >>> then
> > >>>>>>>>> proceed
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
> > >> add
> > >>>>>> another
> > >>>>>>>>>>>>>>> command to the agent code that is called for this
> > >> particular
> > >>>>> type
> > >>>>>>>>> of
> > >>>>>>>>>>>>>>> storage.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hey John,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I created a document for a customer today that
> outlines
> > >>> how
> > >>>>> the
> > >>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> > >> use
> > >>>> to
> > >>>>>>>>> you, as
> > >>>>>>>>>>>>> well,
> > >>>>>>>>>>>>>>>> as you perform the code review.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I have shared this document with you (you should
> have
> > >>>>> received
> > >>>>>>>>> that
> > >>>>>>>>>>>>>>>> information in a separate e-mail).
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Talk to you later!
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > >>>>>> jburwell@basho.com
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some
> thoughts,
> > >>> but
> > >>>>>> want
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > >>>>>>>>>>>>> disk_io_throttle,
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> > >>>>> coherent
> > >>>>>>>>> form
> > >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces
> the
> > >>>> notion
> > >>>>>> of
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
> > >> the
> > >>>>> notion
> > >>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
> was
> > >>>>>>>>> incomplete,
> > >>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>> I had
> > >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> > >> device.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> > >> this.
> > >>>> :)
> > >>>>> Do
> > >>>>>>>>> you
> > >>>>>>>>>>>>>>>>>> have a
> > >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> > >>> what
> > >>>>> you're
> > >>>>>>>>> looking
> > >>>>>>>>>>>>>>>>>> for?
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > >>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces
> the
> > >>>> notion
> > >>>>>> of
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> > >>>>> capabilities
> > >>>>>>>>> (i.e.
> > >>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> > >>> managed
> > >>>>> by
> > >>>>>>>>> CloudStack)
> > >>>>>>>>>>>>>>>>>> from the
> > >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with
> a
> > >>>>> semantic
> > >>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed.
> I
> > >>> also
> > >>>>> want
> > >>>>>>>>> to
> > >>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from
> the
> > >>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> definitions.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > >>>>>>>>>>> jburwell@basho.com
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following
> questions
> > >>> for
> > >>>>> me
> > >>>>>> with
> > >>>>>>>>>>>>>>>>>> regards to
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs
> and
> > >>> SAN
> > >>>>>> volumes?
> > >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
> (LUN
> > >>> 0).
> > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
> and
> > >>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> Volumes?
> > >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> > >>> server or
> > >>>>>>>>> externally
> > >>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> operator?
> > >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> > >>> volume
> > >>>>> (same
> > >>>>>> as a
> > >>>>>>>>>>> SAN
> > >>>>>>>>>>>>>>>>>> LUN) is
> > >>>>>>>>>>>>>>>>>>>>> created by the management server (via the
> plug-in)
> > >>>>> the
> > >>>>>> first
> > >>>>>>>>>>> time
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in,
> but
> > >>>> still
> > >>>>>>>>> want
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> use a
> > >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> > >>> today
> > >>>>>> (prior
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> 4.2). The
> > >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and -
> in
> > >>> this
> > >>>>> case -
> > >>>>>>>>>>>>>>>>>> multiple VMs
> > >>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While
> you
> > >>> can do
> > >>>>> this
> > >>>>>>>>>>>>> today,
> > >>>>>>>>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management
> server
> > >>> or
> > >>>>> externally
> > >>>>>>>>> by
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> operator?
> > >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> > >>> volumes
> > >>>>> are
> > >>>>>>>>>>>>> completely
> > >>>>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
> There
> > >>> is
> > >>>>> no
> > >>>>>> admin
> > >>>>>>>>>>>>>>>>>>>> interaction.
> > >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> > >>> volume
> > >>>>> and a
> > >>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>> volume,
> > >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > >>>>> supports
> > >>>>>> true
> > >>>>>>>>>>> QoS.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> > >> related
> > >>>>> and
> > >>>>>>>>>>>>> expected
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> operate.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that
> multiple
> > >>> VMs
> > >>>>> with
> > >>>>>>>>>>>>>>>>>> provisioned
> > >>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> > >>> single
> > >>>>> SAN
> > >>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>> (LUN) to a
> > >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> > >>> multiple
> > >>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still
> guarantee
> > >>> QoS.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN
> volume
> > >>> house
> > >>>>> more
> > >>>>>>>>> than
> > >>>>>>>>>>>>> one
> > >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> > >>> without
> > >>>>> any of
> > >>>>>> my
> > >>>>>>>>>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>>>> code.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN
> volume
> > >>> for
> > >>>>>>>>>>> CloudStack's
> > >>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>> onto
> > >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John
> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which
> concerns me
> > >>> --
> > >>>>>> concepts
> > >>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is
> today
> > >>>> only
> > >>>>>> used
> > >>>>>>>>>>> (as
> > >>>>>>>>>>>>>>>>>> far as
> > >>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>> know)
> > >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
> not
> > >>>>> PreSetup an
> > >>>>>>>>> SR).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> from
> > >>>>> VMware, it
> > >>>>>>>>> uses
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> vmfs
> > >>>>>>>>>>>>>>>>>>>>>> type.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume
> from
> > >>> KVM,
> > >>>>> it uses
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> > >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> > >>> here
> > >>>>> was to
> > >>>>>>>>> make a
> > >>>>>>>>>>>>>>>>>> new type
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> > >>> Edison
> > >>>>> added
> > >>>>>>>>> into
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> > >>> specificy,
> > >>>>> though:
> > >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> > >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> > >>> Tutkowski
> > >>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't
> know
> > >>> or
> > >>>>> care
> > >>>>>> that
> > >>>>>>>>> it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>>>>> used
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > >>>>> questions
> > >>>>>>>>> about
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > >>>>> SolidFire
> > >>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>> needs to
> > >>>>>>>>>>>>>>>>>>>>>>>>> call
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> > >>> hypervisor
> > >>>>> logic
> > >>>>>>>>>>> knows
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>> treat
> > >>>>>>>>>>>>>>>>>>>>>>>>> it as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> constraint
> > >>>>> and just
> > >>>>>>>>>>>>>>>>>> calling it
> > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way
> for
> > >>> the
> > >>>>> hypervisor
> > >>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and
> perform
> > >>> the
> > >>>>> necessary
> > >>>>>>>>>>>>>>>>>> activities.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> > >>> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > >>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be
> able to
> > >>>>> manage a
> > >>>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > >>> CloudStack
> > >>>>> (e.g. the
> > >>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>> shared
> > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> > >>> chosen
> > >>>>> to
> > >>>>>>>>> dedicate
> > >>>>>>>>>>>>>>>>>> only a
> > >>>>>>>>>>>>>>>>>>>>>> portion
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> > >>> make
> > >>>>> sense?"
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a
> SAN
> > >>>>> volume via
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>>>>>>>>> (before
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor)
> would
> > >>>>> work unless
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say,
> an
> > >>> SR.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > >>>>> CloudStack, an
> > >>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>> allocates
> > >>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> > >>> exposes it
> > >>>>>> through
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> LUN. They
> > >>>>>>>>>>>>>>>>>>>>>>>>> simply
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over
> to
> > >>>>> CloudStack,
> > >>>>>>>>> but
> > >>>>>>>>>>>>>>>>>> not allow
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > >>>>> changes, we
> > >>>>>>>>> would
> > >>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>> passing
> > >>>>>>>>>>>>>>>>>>>>>>>>> in a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > >>>>> necessary
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>> (like
> > >>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
> have
> > >>> the
> > >>>>> storage
> > >>>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>>>>>> itself
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs
> support
> > >>> from
> > >>>>> the
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> side and
> > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically
> to
> > >>> create
> > >>>>> the
> > >>>>>>>>> SR
> > >>>>>>>>>>>>>>>>>> before the
> > >>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
> plugin
> > >>>>> should
> > >>>>>> have a
> > >>>>>>>>>>>>> rich
> > >>>>>>>>>>>>>>>>>> enough
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> > >>> available
> > >>>>> for
> > >>>>>> volume
> > >>>>>>>>>>>>>>>>>> storage.
> > >>>>>>>>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > >>> interrogation of
> > >>>>> the
> > >>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device
> itself
> > >>>>> shouldn't
> > >>>>>>>>> know
> > >>>>>>>>>>>>>>>>>> or care
> > >>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
> able
> > >>> to
> > >>>>>> answer
> > >>>>>>>>>>>>>>>>>> questions
> > >>>>>>>>>>>>>>>>>>>>>>>>> about it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> > >>> driver
> > >>>>> (which
> > >>>>>>>>> is a
> > >>>>>>>>>>>>>>>>>> type of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is
> done, a
> > >>> new
> > >>>>> Primary
> > >>>>>>>>>>>>>>>>>> Storage shows
> > >>>>>>>>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> > >>> that
> > >>>>>> references
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>> tag
> > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a
> CloudStack
> > >>>>> volume. This
> > >>>>>>>>> leads
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> a new
> > >>>>>>>>>>>>>>>>>>>>>> row
> > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> CloudStack
> > >>>>> volume to a
> > >>>>>>>>> VM
> > >>>>>>>>>>>>>>>>>> (attach
> > >>>>>>>>>>>>>>>>>>>>>> disk).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework
> calling
> > >>> the
> > >>>>> plug-in
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> create a
> > >>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> > >>> SAN).
> > >>>>> The
> > >>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>> also
> > >>>>>>>>>>>>>>>>>>>>>> updates
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
> data
> > >>>>> (like the
> > >>>>>>>>> IQN of
> > >>>>>>>>>>>>>>>>>> the SAN
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if
> the
> > >>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> volume is in
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach,
> the
> > >>>>> volume
> > >>>>>>>>> will be
> > >>>>>>>>>>>>>>>>>> in the
> > >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
> the
> > >>>>> plug-in
> > >>>>>>>>> code
> > >>>>>>>>>>>>>>>>>> will not
> > >>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
> and
> > >>>>> detects
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the
> form
> > >>> of a
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM,
> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > >>>>> terminology,
> > >>>>>>>>> volume,
> > >>>>>>>>>>>>>>>>>> in these
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > >>>>> allocation on
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> > >>> mean
> > >>>>> 'SAN
> > >>>>>>>>> volume'.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end
> user
> > >>> can
> > >>>>> make in
> > >>>>>>>>>>>>>>>>>> CloudStack, I
> > >>>>>>>>>>>>>>>>>>>>>>>>> try to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very
> well
> > >>> make
> > >>>>> sense, but
> > >>>>>>>>> I'm
> > >>>>>>>>>>>>>>>>>> having a
> > >>>>>>>>>>>>>>>>>>>>>> hard
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison
> in on
> > >>>>> this
> > >>>>>>>>> conversation
> > >>>>>>>>>>>>>>>>>> as he
> > >>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > >>> approach I
> > >>>>> took.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> > >>> John
> > >>>>> Burwell <
> > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining
> two
> > >>>>> distinct
> > >>>>>>>>> concepts
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> protocols.
> > >>> In
> > >>>>> both
> > >>>>>>>>> cases, we
> > >>>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the
> system to
> > >>>>>> create/delete
> > >>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
> the
> > >>>>> volume to
> > >>>>>> be
> > >>>>>>>>>>>>>>>>>> volume to be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack
> context.
> > >>> To
> > >>>>> ensure
> > >>>>>>>>> that
> > >>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>> are in
> > >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > >>>>> definitions,
> > >>>>>>>>> refers to
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> physical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device,
> correct?
> > >>>>> Minimally,
> > >>>>>> we
> > >>>>>>>>>>>>> must
> > >>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>> able
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
> device
> > >>> to
> > >>>>> move bits
> > >>>>>>>>> from
> > >>>>>>>>>>>>>>>>>> one place
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > >>> Optionally, a
> > >>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> Therefore,
> > >>>>> we can have
> > >>>>>> a
> > >>>>>>>>>>>>>>>>>> unmanaged
> > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR,
> and
> > >>> we
> > >>>>> can have a
> > >>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> > >>> capable
> > >>>>> of
> > >>>>>>>>> allocating
> > >>>>>>>>>>>>>>>>>> LUNs and
> > >>>>>>>>>>>>>>>>>>>>>>>>> storing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while
> CloudStack
> > >>> may
> > >>>>> be able
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> manage a
> > >>>>>>>>>>>>>>>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > >>>>> unmanaged by
> > >>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>> (e.g.
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and
> the
> > >>>>> operator has
> > >>>>>>>>> chosen
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> dedicate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack).
> Does
> > >>> my
> > >>>>>> reasoning
> > >>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>> sense?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above
> are
> > >>>>> reasonable, it
> > >>>>>>>>> seems
> > >>>>>>>>>>>>>>>>>>>> appropriate
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management
> concerns
> > >>> from
> > >>>>>>>>> StoragePoolType,
> > >>>>>>>>>>>>>>>>>> add the
> > >>>>>>>>>>>>>>>>>>>>>>>>> notion
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an
> attached
> > >>> driver
> > >>>>> that
> > >>>>>>>>>>> indicates
> > >>>>>>>>>>>>>>>>>> whether
> > >>>>>>>>>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> > >>> establish
> > >>>>> a
> > >>>>>>>>> abstraction
> > >>>>>>>>>>>>>>>>>>>>>> representing a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
> that
> > >>> is
> > >>>>>>>>> associated
> > >>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>> volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place,
> hypervisor
> > >>>>> drivers can
> > >>>>>>>>>>>>> declare
> > >>>>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they
> encounter
> > >>> a
> > >>>>> device
> > >>>>>> managed
> > >>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>> CloudStack,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations
> exposed
> > >>> by
> > >>>>> the driver to
> > >>>>>>>>>>>>> automate
> > >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> > >>> sense,
> > >>>>> then we can
> > >>>>>>>>> sit
> > >>>>>>>>>>>>>>>>>> down and
> > >>>>>>>>>>>>>>>>>>>>>>>>> drill
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM,
> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference
> between
> > >>> the
> > >>>>> current
> > >>>>>> iSCSI
> > >>>>>>>>>>>>> type
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to
> go in
> > >>> and
> > >>>>> create
> > >>>>>> a
> > >>>>>>>>>>>>>>>>>> Primary
> > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
> time,
> > >>>>> the
> > >>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>> volume must
> > >>>>>>>>>>>>>>>>>>>>>>>>> exist
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> allocated).
> > >>>>> Future
> > >>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
> created
> > >>>>> behind the
> > >>>>>>>>>>>>> scenes.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has
> to
> > >>> go in
> > >>>>> and
> > >>>>>> create
> > >>>>>>>>>>>>>>>>>> Primary
> > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
> delete
> > >>>>> volumes on
> > >>>>>>>>> its
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>> system
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via
> the
> > >>>>> storage
> > >>>>>>>>>>>>>>>>>> framework). When
> > >>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>> user
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume
> that
> > >>> was
> > >>>>> created,
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>>>>>>>> tells
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new
> volume.
> > >>> After
> > >>>>> this is
> > >>>>>>>>> done,
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
> called.
> > >>>>> No
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the
> volume
> > >>> was
> > >>>>> just
> > >>>>>>>>> created.
> > >>>>>>>>>>>>> The
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21
> PM,
> > >>> Mike
> > >>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
> should
> > >>>>> probably
> > >>>>>> have
> > >>>>>>>>>>>>>>>>>> defined them
> > >>>>>>>>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> > >>> pre-
> > >>>>> allocated
> > >>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>> an admin
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> > >>> hypervisor
> > >>>>> data
> > >>>>>>>>> structure
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer
> SR),
> > >>> then
> > >>>>> that
> > >>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack.
> Disks
> > >>> (VDI)
> > >>>>> are later
> > >>>>>>>>>>>>> placed
> > >>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> > >>> these
> > >>>>> cases, the
> > >>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> > >>> already
> > >>>>> in place
> > >>>>>>>>> and
> > >>>>>>>>>>>>>>>>>> simply
> > >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> structure
> > >>> to
> > >>>>> the VM in
> > >>>>>>>>>>>>>>>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage
> that is
> > >>> not
> > >>>>>>>>>>> pre-allocated.
> > >>>>>>>>>>>>>>>>>> Instead
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this
> could
> > >>> be a
> > >>>>> SAN
> > >>>>>> (not
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> volume on
> > >>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor
> data
> > >>>>> structure
> > >>>>>>>>> must be
> > >>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>> when an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed
> because
> > >>> these
> > >>>>> types of
> > >>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>> have not
> > >>>>>>>>>>>>>>>>>>>>>>>>> been
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > >>>>> structure by an
> > >>>>>>>>>>> admin.
> > >>>>>>>>>>>>>>>>>> Once
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> XenServer
> > >>> for
> > >>>>> this
> > >>>>>>>>> volume,
> > >>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>> attaches
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
> VM
> > >>> in
> > >>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13
> PM,
> > >>>>> John Burwell
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current
> implementation of
> > >>> the
> > >>>>> Dynamic
> > >>>>>> type
> > >>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>> behavior
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which
> why I
> > >>> ask
> > >>>>> about the
> > >>>>>>>>>>>>>>>>>> difference.
> > >>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is
> the
> > >>>>> definition
> > >>>>>> of
> > >>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> Dynamic
> > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> > >>> Mike
> > >>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
> type
> > >>> is
> > >>>>>> uniquely
> > >>>>>>>>>>> used
> > >>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> > >>> Storage
> > >>>>> that is
> > >>>>>>>>> directly
> > >>>>>>>>>>>>>>>>>> based on
> > >>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> > >>> step of
> > >>>>> going
> > >>>>>> to
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based
> on
> > >>> that
> > >>>>> iSCSI
> > >>>>>>>>> target as
> > >>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>> does
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > >>>>> supported for
> > >>>>>>>>>>>>>>>>>> XenServer.
> > >>>>>>>>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>>>>>>>> all
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must
> first go
> > >>> to
> > >>>>> the
> > >>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> perform
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what
> RBD is.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 2:13
> > >>> PM,
> > >>>>> John
> > >>>>>> Burwell
> > >>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the
> code,
> > >>> what
> > >>>>> is the
> > >>>>>>>>>>> difference
> > >>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why
> isn't
> > >>> RBD
> > >>>>> considered
> > >>>>>>>>>>>>> Dynamic?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46
> PM,
> > >>> Mike
> > >>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of
> storage is
> > >>>>> defined in
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > >>>>> StoragePoolType {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), //
> local
> > >>>>> directory
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> NetworkFilesystem(true),
> > >>> //
> > >>>>> NFS or CIFS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), //
> shared
> > >>> LUN,
> > >>>>> with a
> > >>>>>>>>>>> clusterfs
> > >>>>>>>>>>>>>>>>>>>> overlay
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g.,
> ZFS
> > >>>>> Comstar
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso
> image
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), //
> XenServer
> > >>> local
> > >>>>> LVM SR
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> SharedMountPoint(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), //
> VMware
> > >>> VMFS
> > >>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > >>>>> XenServer, Storage
> > >>>>>>>>> Pool
> > >>>>>>>>>>>>>>>>>> is set
> > >>>>>>>>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), //
> XenServer
> > >>> local
> > >>>>> EXT SR
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); //
> dynamic,
> > >>>>> zone-wide
> > >>>>>>>>> storage
> > >>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> StoragePoolType(boolean
> > >>>>> shared) {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean
> isShared() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 1:41
> > >>> PM,
> > >>>>> Mike
> > >>>>>>>>> Tutkowski
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> > >>> another
> > >>>>> storage
> > >>>>>>>>>>> company
> > >>>>>>>>>>>>>>>>>> wants
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> > >>> Quality
> > >>>>> of
> > >>>>>> Service
> > >>>>>>>>>>>>>>>>>> feature. It
> > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
> well.
> > >>>>> They would
> > >>>>>>>>> need
> > >>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made
> the
> > >>>>> necessary
> > >>>>>>>>> changes to
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-
> in.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> > >>> 1:39
> > >>>>> PM, Mike
> > >>>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply
> the
> > >>>>> XenServer code
> > >>>>>> is
> > >>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer
> attach
> > >>> logic is
> > >>>>> now aware
> > >>>>>>>>> of
> > >>>>>>>>>>>>>>>>>> dynamic,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > >>>>> implementation of
> > >>>>>>>>> this
> > >>>>>>>>>>>>>>>>>> kind of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> > >>> with
> > >>>>> Edison's
> > >>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a
> new
> > >>>>> framework that
> > >>>>>>>>>>> supported
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes
> dynamically.
> > >>>>> However, when I
> > >>>>>>>>>>>>>>>>>> visited with
> > >>>>>>>>>>>>>>>>>>>>>> him
> > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized
> that
> > >>> it
> > >>>>> was not
> > >>>>>>>>>>>>>>>>>> complete. We
> > >>>>>>>>>>>>>>>>>>>>>>>>> realized
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
> could
> > >>> do
> > >>>>> with these
> > >>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>> unless
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to
> recognize
> > >>> this
> > >>>>> new type of
> > >>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>> create
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> structure.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> > >>> 1:28
> > >>>>> PM, John
> > >>>>>>>>> Burwell
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd
> to me
> > >>>>> that any
> > >>>>>>>>>>> operation
> > >>>>>>>>>>>>>>>>>> in the
> > >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> > >>> about
> > >>>>> details. I
> > >>>>>>>>> expect
> > >>>>>>>>>>>>>>>>>> to see
> > >>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> > >>> operations
> > >>>>> that can be
> > >>>>>>>>>>>>>>>>>>>> composed/driven
> > >>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > >>>>> allocate
> > >>>>>> space/create
> > >>>>>>>>>>>>>>>>>> structures
> > >>>>>>>>>>>>>>>>>>>>>> per
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > >>>>> dependency, we are
> > >>>>>>>>> going
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> end
> > >>>>>>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
> make
> > >>> the
> > >>>>> system
> > >>>>>>>>>>>>> increasingly
> > >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > >>>>> understanding that the
> > >>>>>>>>> Xen
> > >>>>>>>>>>>>>>>>>> specific
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > >>>>> CitrixResourceBase
> > >>>>>> class?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013 at
> > >>> 3:12
> > >>>>> PM, Mike
> > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this
> in a
> > >>> bit
> > >>>>> more
> > >>>>>>>>> detail:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and
> aside
> > >>> from
> > >>>>> one setup
> > >>>>>>>>>>>>> method
> > >>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> > >>> volume on
> > >>>>> the
> > >>>>>> storage
> > >>>>>>>>>>>>>>>>>> system,
> > >>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> > >>> structure
> > >>>>> to make
> > >>>>>> use
> > >>>>>>>>> of
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > >>> XenServer
> > >>>>> or a
> > >>>>>> datastore
> > >>>>>>>>> on
> > >>>>>>>>>>>>>>>>>> ESX(i)).
> > >>>>>>>>>>>>>>>>>>>>>> VMs
> > >>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this
> storage
> > >>>>> system's volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> > >>> storage
> > >>>>> framework,
> > >>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> > >>> create
> > >>>>> a 1:1
> > >>>>>>>>>>>>> relationship
> > >>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the
> VM's
> > >>> data
> > >>>>> disk
> > >>>>>>>>> (necessary
> > >>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write
> a
> > >>> plug-
> > >>>>> in that is
> > >>>>>>>>> called
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as
> needed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that
> the
> > >>>>> storage
> > >>>>>> framework
> > >>>>>>>>> did
> > >>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>> address
> > >>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > >>> hypervisor-
> > >>>>> specific data
> > >>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>> when
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the
> case,
> > >>> I've
> > >>>>> been
> > >>>>>>>>> enhancing
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> to do
> > >>>>>>>>>>>>>>>>>>>> so.
> > >>>>>>>>>>>>>>>>>>>>>>>>> I've
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > >>>>> submitted. I've got
> > >>>>>>>>> ESX(i)
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>>>>>>>> sandbox
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the
> 4.2
> > >>>>> freeze date.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
> bit? :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013 at
> > >>>>> 1:03 PM, Mike
> > >>>>>>>>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage
> plug-in -
> > >>> by
> > >>>>> itself - is
> > >>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with
> the
> > >>>>> volume-attach
> > >>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>> (in the
> > >>>>>>>>>>>>>>>>>>>>>>>>> agent
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls
> into
> > >>> the
> > >>>>> plug-in to
> > >>>>>>>>> have
> > >>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>> create a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> > >>> comes to
> > >>>>> attach the
> > >>>>>>>>> volume
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> > >>> enough
> > >>>>> to recognize
> > >>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the
> volume
> > >>> has
> > >>>>> just been
> > >>>>>>>>> created)
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>> create,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > >>>>> XenServer) or a
> > >>>>>>>>> datastore
> > >>>>>>>>>>>>>>>>>> (for
> > >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
> just
> > >>>>> created.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been
> spending
> > >>> most
> > >>>>> of my time
> > >>>>>>>>>>>>> recently
> > >>>>>>>>>>>>>>>>>> making
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent
> code.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it
> up?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> > >>> at
> > >>>>> 12:48 PM,
> > >>>>>> John
> > >>>>>>>>>>>>>>>>>> Burwell <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> > >>> why
> > >>>>> the the storage
> > >>>>>>>>>>>>> driver
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> at
> > >>> 1:21
> > >>>>> PM, Mike
> > >>>>>>>>>>>>> Tutkowski
> > >>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes,
> ultimately I
> > >>>>> would like to
> > >>>>>>>>> support
> > >>>>>>>>>>>>>>>>>> all
> > >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I
> think
> > >>> I'm
> > >>>>> just out of
> > >>>>>>>>> time
> > >>>>>>>>>>>>>>>>>> for 4.2
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> get
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> > >>> plug-in
> > >>>>> supports
> > >>>>>>>>>>>>> XenServer.
> > >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> > >>> feature
> > >>>>> freeze, I
> > >>>>>>>>> have
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> working
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to
> be
> > >>> clear,
> > >>>>> this is
> > >>>>>> all
> > >>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> regards
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> Compute
> > >>>>> Offerings post
> > >>>>>> 4.2.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > >>> 2013
> > >>>>> at 11:14 AM,
> > >>>>>>>>> Kelcey
> > >>>>>>>>>>>>>>>>>> Jamison
> > >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
> plan
> > >>> on
> > >>>>> supporting
> > >>>>>>>>> KVM in
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > >>>>> Message -----
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > >>>>> Tutkowski" <
> > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > >>>>> dev@cloudstack.apache.org
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent:
> Monday,
> > >>> June
> > >>>>> 3, 2013
> > >>>>>>>>> 10:12:32 AM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> > >>> [MERGE]
> > >>>>>>>>>>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>> MASTER
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> > >>> merging
> > >>>>> Wei's feature
> > >>>>>>>>>>> first,
> > >>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>> mine.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature
> is
> > >>> for
> > >>>>> KVM only,
> > >>>>>>>>> then
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> is a
> > >>>>>>>>>>>>>>>>>>>> non
> > >>>>>>>>>>>>>>>>>>>>>>>>> issue
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun
> 3,
> > >>> 2013
> > >>>>> at 8:55 AM,
> > >>>>>> Wei
> > >>>>>>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> billing,
> > >>> as
> > >>>>> no one works
> > >>>>>>>>> on
> > >>>>>>>>>>>>>>>>>> billing
> > >>>>>>>>>>>>>>>>>>>> now,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing
> by
> > >>>>> themselves. They
> > >>>>>>>>> can
> > >>>>>>>>>>>>> get
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk_offering of
> > >>> a
> > >>>>> VMs and
> > >>>>>> volumes
> > >>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user
> the
> > >>>>> exact
> > >>>>>> limitation
> > >>>>>>>>>>>>> value
> > >>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>> individual
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> limitation
> > >>> for
> > >>>>> nics as
> > >>>>>> well.
> > >>>>>>>>> I
> > >>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of
> I/O
> > >>>>> throttling?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > >>>>> misunstand the second
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> > >>> what
> > >>>>> you said about
> > >>>>>> the
> > >>>>>>>>>>> two
> > >>>>>>>>>>>>>>>>>>>> features.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> John
> > >>>>> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> 2013,
> > >>> at
> > >>>>> 2:13 AM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> Mike
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> Mike's
> > >>>>> aswer helps you.
> > >>>>>>>>> I am
> > >>>>>>>>>>>>>>>>>> trying
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I
> think
> > >>> billing
> > >>>>> should
> > >>>>>>>>> depend
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>> IO
> > >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> limitation.
> > >>> Please
> > >>>>> review
> > >>>>>>>>>>>>>>>>>> disk_io_stat if
> > >>>>>>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the
> IO
> > >>>>> statistics
> > >>>>>> including
> > >>>>>>>>>>>>>>>>>> bytes/iops
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> > >>> machine.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by
> the
> > >>> AWS
> > >>>>> model,
> > >>>>>> customers
> > >>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>> billed
> > >>>>>>>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> provisioned
> > >>> IOPS,
> > >>>>> as well as,
> > >>>>>> for
> > >>>>>>>>>>>>> those
> > >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://aws.amazon.com/ebs/).
> > >>>>>> I
> > >>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>> imagine
> > >>>>>>>>>>>>>>>>>>>>>>>>> our
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> > >>> employ
> > >>>>> similar cost
> > >>>>>>>>>>> models.
> > >>>>>>>>>>>>>>>>>> Could
> > >>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing
> model
> > >>> in
> > >>>>> the current
> > >>>>>>>>> patch?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do
> you
> > >>> mean
> > >>>>> IOPS runtime
> > >>>>>>>>>>> change?
> > >>>>>>>>>>>>>>>>>> KVM
> > >>>>>>>>>>>>>>>>>>>>>>>>> supports
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> limitation for
> > >>> a
> > >>>>> running
> > >>>>>> virtual
> > >>>>>>>>>>>>>>>>>> machine
> > >>>>>>>>>>>>>>>>>>>>>> through
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> CloudStack
> > >>> does
> > >>>>> not support
> > >>>>>>>>>>> changing
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (computer
> > >>>>> offering or disk
> > >>>>>>>>>>>>> offering).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
> the
> > >>>>> Java interface
> > >>>>>>>>> level.
> > >>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>> apologize
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > >>>>> generalize allocation
> > >>>>>>>>>>>>>>>>>> algorithms
> > >>>>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>> set
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe
> the
> > >>>>> service guarantees
> > >>>>>>>>>>>>>>>>>> provided by a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> > >>> good
> > >>>>> question.
> > >>>>>>>>> Maybe it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>> better
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> disk_io_throttling as Mike
> > >>>>>>>>> needs to
> > >>>>>>>>>>>>>>>>>> consider
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I
> think.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will
> expand
> > >>> on
> > >>>>> my thoughts
> > >>>>>> in a
> > >>>>>>>>>>>>> later
> > >>>>>>>>>>>>>>>>>>>>>> response
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch
> points
> > >>>>> between these two
> > >>>>>>>>>>>>>>>>>> features. I
> > >>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need
> to be
> > >>>>> merged before
> > >>>>>>>>>>>>>>>>>> SolidFire, but
> > >>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> coordination
> > >>>>> between the
> > >>>>>> branches
> > >>>>>>>>>>>>>>>>>> (possibly
> > >>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> disk_io_throttling)
> > >>>>> to
> > >>>>>>>>> coordinate on
> > >>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>> issue.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > >>> John
> > >>>>> Burwell <
> > >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> things I
> > >>>>> want to
> > >>>>>> understand
> > >>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>> following:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is
> there
> > >>>>> value in
> > >>>>>> capturing
> > >>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>> policies
> > >>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> model
> > >>> (e.g.
> > >>>>> for
> > >>>>>>>>> billing/usage
> > >>>>>>>>>>>>>>>>>>>> purposes,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2)
> Should
> > >>> there
> > >>>>> be a common
> > >>>>>>>>>>>>>>>>>> interface model
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> provisioning
> > >>> at
> > >>>>> runtime?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How
> are
> > >>>>> conflicting
> > >>>>>>>>> provisioned
> > >>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> hypervisor
> > >>> and
> > >>>>> storage device
> > >>>>>>>>>>>>>>>>>> reconciled?
> > >>>>>>>>>>>>>>>>>>>> In
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead
> to
> > >>>>> believe (and
> > >>>>>> billed)
> > >>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>> device
> > >>>>> has been
> > >>>>>>>>> configured
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> deliver.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> consistent
> > >>>>> configuration
> > >>>>>>>>> between a
> > >>>>>>>>>>>>>>>>>> VM and a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time,
> but a
> > >>>>> later
> > >>>>>> modification
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> inconsistency.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun
> 2,
> > >>> 2013,
> > >>>>> at 8:38 PM,
> > >>>>>>>>> Mike
> > >>>>>>>>>>>>>>>>>> Tutkowski
> > >>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> believe
> > >>> Wei's
> > >>>>> feature deals
> > >>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>> controlling
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>> hypervisor
> > >>>>> side.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> feature
> > >>> is
> > >>>>> focused on
> > >>>>>>>>>>>>> controlling
> > >>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> that
> > >>>>> helps. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
> Jun
> > >>> 2,
> > >>>>> 2013 at 6:35
> > >>>>>> PM,
> > >>>>>>>>>>>>> John
> > >>>>>>>>>>>>>>>>>>>> Burwell
> > >>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> jburwell@basho.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > >>> opinion is
> > >>>>> that no
> > >>>>>> features
> > >>>>>>>>>>>>>>>>>> should be
> > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> > >>> have
> > >>>>> been resolved
> > >>>>>> and
> > >>>>>>>>> it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>> ready
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>> turn
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> Ops
> > >>> vs
> > >>>>> discrete
> > >>>>>>>>> read/write
> > >>>>>>>>>>>>>>>>>> ops issue
> > >>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-
> reviewed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> I
> > >>> don't
> > >>>>> think this
> > >>>>>>>>> criteria
> > >>>>>>>>>>>>>>>>>> has been
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> how
> > >>>>> does this work
> > >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> > >>>>>>>>>>>>>>>>>> )?
> > >>>>>>>>>>>>>>>>>>>> As I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> > >>> involves
> > >>>>> provisioned
> > >>>>>>>>> IOPS. I
> > >>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> scenario
> > >>>>> where provisioned
> > >>>>>>>>> IOPS
> > >>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> KVM and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> unnecessarily
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> incompatible.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> Jun 1,
> > >>> 2013,
> > >>>>> at 6:47 AM,
> > >>>>>>>>> Wei
> > >>>>>>>>>>>>>>>>>> ZHOU <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
> will
> > >>>>> change it next
> > >>>>>>>>> week.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/6/1
> > >>>>> Wido den Hollander
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 06/01/2013 08:24 AM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Exactly. I
> > >>> have
> > >>>>> pushed the
> > >>>>>>>>>>>>> features
> > >>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> anyone
> > >>>>> object thems for
> > >>>>>>>>>>>>> technical
> > >>>>>>>>>>>>>>>>>>>> reason
> > >>>>>>>>>>>>>>>>>>>>>>>>> till
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For
> the
> > >>> sake
> > >>>>> of clarity I
> > >>>>>> just
> > >>>>>>>>>>>>> want
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> mention
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> total
> > >>> IOps
> > >>>>> to R/W IOps
> > >>>>>>>>> asap
> > >>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>>>>>>>>>> never
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> total
> > >>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You
> laid
> > >>> the
> > >>>>> groundwork for
> > >>>>>>>>> the
> > >>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>> throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> however
> > >>>>> prevent that we
> > >>>>>> create
> > >>>>>>>>>>>>>>>>>> legacy from
> > >>>>>>>>>>>>>>>>>>>>>> day
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/5/31
> > >>>>> Wido den
> > >>>>>> Hollander <
> > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 05/31/2013 03:59 PM, John
> > >>>>>>>>>>>>> Burwell
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 --
> this
> > >>>>> enhancement must
> > >>>>>> to
> > >>>>>>>>>>>>>>>>>> discretely
> > >>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> see
> > >>> how
> > >>>>> it could be
> > >>>>>>>>> fixed
> > >>>>>>>>>>>>>>>>>> later
> > >>>>>>>>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> correctly
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split
> total
> > >>> IOPS
> > >>>>> into read
> > >>>>>> and
> > >>>>>>>>>>>>>>>>>> write.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>> unless/until we
> > >>>>>> decided
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> break
> > >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
> Wei
> > >>>>> meant was merging
> > >>>>>> it
> > >>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>> master
> > >>>>>>>>>>>>>>>>>>>>>> now
> > >>>>>>>>>>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> branch
> > >>>>> and add Read /
> > >>>>>>>>> Write
> > >>>>>>>>>>>>> IOps
> > >>>>>>>>>>>>>>>>>>>> before
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > >>>>> released with Read
> > >>>>>> and
> > >>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>> instead
> > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is
> to
> > >>>>> make the May 31st
> > >>>>>>>>>>>>> feature
> > >>>>>>>>>>>>>>>>>>>> freeze
> > >>>>>>>>>>>>>>>>>>>>>>>>> date.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
> other
> > >>>>> threads) then it
> > >>>>>>>>> won't
> > >>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>> necessary
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > >>>>> completely agree that
> > >>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>>> is no
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> I/O.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> May 31,
> > >>>>> 2013, at 9:51 AM,
> > >>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>> den
> > >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 05/31/2013 03:13 PM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Thanks.
> > >>> Good
> > >>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> thought
> > >>>>> about at the
> > >>>>>>>>>>> beginning.
> > >>>>>>>>>>>>>>>>>>>> Finally I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> difference
> > >>> of
> > >>>>> read and write
> > >>>>>>>>>>>>> mainly
> > >>>>>>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care
> the
> > >>>>> difference of sent
> > >>>>>>>>> and
> > >>>>>>>>>>>>>>>>>> received
> > >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > >>>>> reasoning seems odd.
> > >>>>>>>>>>>>>>>>>> Networking and
> > >>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> I/O is
> > >>>>> much more
> > >>>>>>>>> expensive
> > >>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> most
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> bandwith.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> Implementing
> > >>>>> it will be some
> > >>>>>>>>>>>>>>>>>> copy-paste
> > >>>>>>>>>>>>>>>>>>>>>> work.
> > >>>>>>>>>>>>>>>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> implemented
> > >>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few
> days.
> > >>> For
> > >>>>> the deadline
> > >>>>>> of
> > >>>>>>>>>>>>>>>>>> feature
> > >>>>>>>>>>>>>>>>>>>>>> freeze,
> > >>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that ,
> if
> > >>>>> needed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It
> think it's
> > >>> a
> > >>>>> feature we
> > >>>>>>>>> can't
> > >>>>>>>>>>>>>>>>>> miss. But
> > >>>>>>>>>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> window
> > >>> we
> > >>>>> have to make sure
> > >>>>>> we
> > >>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in
> 4.3,
> > >>> that
> > >>>>> will confuse
> > >>>>>>>>>>>>> users.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/5/31
> > >>>>> Wido den
> > >>>>>> Hollander <
> > >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > >>>>> 05/30/2013 06:03 PM, Wei
> > >>>>>>>>> ZHOU
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> would
> > >>> like to
> > >>>>> merge
> > >>>>>>>>>>>>>>>>>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>> branch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> nobody
> > >>>>> object, I will
> > >>>>>> merge
> > >>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>> master
> > >>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>> 48
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>> purpose
> > >>>>> is :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > >>>>> machines are running
> > >>>>>>>>> on
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> same
> > >>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> storage or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> > >>> strage).
> > >>>>> Because of
> > >>>>>> the
> > >>>>>>>>>>> rate
> > >>>>>>>>>>>>>>>>>>>>>> limitation
> > >>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops),
> if
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one
> VM
> > >>> has
> > >>>>> large disk
> > >>>>>>>>> operation,
> > >>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>>>>>>>> affect
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> VMs
> > >>>>> running on the
> > >>>>>> same
> > >>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>> device.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > >>> neccesary
> > >>>>> to set the
> > >>>>>>>>>>> maximum
> > >>>>>>>>>>>>>>>>>> rate
> > >>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>> limit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Looking at
> > >>> the
> > >>>>> code I see
> > >>>>>> you
> > >>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>>>>>>>>> difference
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
> and
> > >>>>> libvirt support
> > >>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>> both a
> > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > >>> which
> > >>>>> could benefit a
> > >>>>>>>>> lot of
> > >>>>>>>>>>>>>>>>>> users.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's
> also
> > >>>>> strange, in the
> > >>>>>>>>> polling
> > >>>>>>>>>>>>>>>>>> side you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> but
> > >>> on
> > >>>>> the throttling
> > >>>>>>>>> side
> > >>>>>>>>>>>>>>>>>> you only
> > >>>>>>>>>>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> IOps
> > >>> are
> > >>>>> usually much
> > >>>>>>>>> more
> > >>>>>>>>>>>>>>>>>> expensive
> > >>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
> valid
> > >>>>> use-case where
> > >>>>>>>>> that
> > >>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>> admin
> > >>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> vs
> > >>> Read
> > >>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since
> this
> > >>> only
> > >>>>> supports KVM
> > >>>>>>>>> at
> > >>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>> point I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> to at
> > >>>>> least have the
> > >>>>>>>>>>>>>>>>>> mechanism in
> > >>>>>>>>>>>>>>>>>>>>>> place
> > >>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> implementing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> later
> > >>>>> would be a lot of
> > >>>>>>>>>>> work.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> > >>> hypervisor
> > >>>>> doesn't
> > >>>>>>>>> support
> > >>>>>>>>>>>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>>>>>>>>> different
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> you
> > >>> can
> > >>>>> always sum
> > >>>>>> both
> > >>>>>>>>> up
> > >>>>>>>>>>>>>>>>>> and set
> > >>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> you
> > >>>>> explain why you
> > >>>>>>>>>>>>> implemented
> > >>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>> way?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>> feature
> > >>>>> includes:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> the
> > >>>>> maximum rate of
> > >>>>>>>>> VMs
> > >>>>>>>>>>>>> (in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> configuration)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> change
> > >>> the
> > >>>>> maximum rate
> > >>>>>> of
> > >>>>>>>>>>> VMs
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> limit
> > >>> the
> > >>>>> disk rate
> > >>>>>> (total
> > >>>>>>>>>>> bps
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>> iops)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
> ticket:
> > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> issues.apache.org/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> jira/browse/CLOUDSTACK-1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1192>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >>> 1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> 1192
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> 1192>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I
> will
> > >>>>> update later) :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>> VM+Disk+IO+Throttling<https://
> > >>>>>>>>>>>>> ****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> cwiki.apache.org/confluence/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>
> > >>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >>>>> rottling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Merge
> > >>> check
> > >>>>> list :-
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did
> you
> > >>>>> check the branch's
> > >>>>>>>>> RAT
> > >>>>>>>>>>>>>>>>>> execution
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> > >>> there
> > >>>>> new dependencies
> > >>>>>>>>>>>>>>>>>> introduced?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> What
> > >>>>> automated testing
> > >>>>>> (unit
> > >>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> feature?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit
> tests
> > >>> are
> > >>>>> added.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *
> What
> > >>>>> testing has been done
> > >>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> check for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> the
> > >>>>> bytes rate and
> > >>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>> rate on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> VM
> > >>>>> operations, including
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> deploy,
> > >>> stop,
> > >>>>> start, reboot,
> > >>>>>>>>>>>>>>>>>> destroy,
> > >>>>>>>>>>>>>>>>>>>>>> expunge.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > >>> Volume
> > >>>>> operations,
> > >>>>>>>>> including
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > >>>>> Detach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> review
> > >>> the
> > >>>>> code, you can
> > >>>>>>>>> try
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> > >>> regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>> VM+Disk+IO+Throttling<https://
> > >>>>>>>>>>>>> ****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> cwiki.apache.org/confluence/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>
> > >>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > >>>>> rottling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > >>>>>>>>> refs/heads/disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >>>>> 1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> > >>> 1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1301>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >>> 1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> 1301
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> 1301>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> issues.apache.org/****jira/**browse/CLOUDSTACK-
> > >>> 2071
> > >>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >>>>> 2071
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > >>>>> 2071
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-
> 2071
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > >>>>> 2071
> > >>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > >>>>> 2071>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> <h**ttps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-
> 2071>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> (**CLOUDSTACK-1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -
> VM
> > >>> Disk
> > >>>>> I/O
> > >>>>>> Throttling)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >>>>> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >>>>> CloudStack Developer,
> > >>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > >>> 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Advancing
> > >>> the
> > >>>>> way the world
> > >>>>>>>>> uses
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > >>> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >>> CloudStack
> > >>>>> Developer,
> > >>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> > >>> way
> > >>>>> the world uses
> > >>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > >>> CloudStack
> > >>>>> Developer,
> > >>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > >>> way
> > >>>>> the world uses
> > >>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> > >>>>> Developer,
> > >>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> > >>> the
> > >>>>> world uses the
> > >>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> > >>>>> Developer,
> > >>>>>> SolidFire
> > >>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> > >>> the
> > >>>>> world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >>>>> Developer, SolidFire
> > >>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> > >>>>> world uses the
> > >>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >>> Developer,
> > >>>>> SolidFire
> > >>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> > >>> world
> > >>>>> uses the
> > >>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > >>> Developer,
> > >>>>> SolidFire
> > >>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > >>> world
> > >>>>> uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > >>>>> SolidFire
> > >>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> > >>> uses
> > >>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > >>>>> SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> > >>> uses
> > >>>>> the cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>>>> SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> > >>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> > >>> the
> > >>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > >>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > >>> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > >>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > >>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > >>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> > >>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>> *(tm)*
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> *Mike Tutkowski*
> > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>> o: 303.746.7302
> > >>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>> *(tm)*
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> *Mike Tutkowski*
> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>> o: 303.746.7302
> > >>>>>>> Advancing the way the world uses the
> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>> *(tm)*
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> *Mike Tutkowski*
> > >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>> e: mike.tutkowski@solidfire.com
> > >>>>> o: 303.746.7302
> > >>>>> Advancing the way the world uses the
> > >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>> *(tm)*
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> *Mike Tutkowski*
> > >>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>> e: mike.tutkowski@solidfire.com
> > >>> o: 303.746.7302
> > >>> Advancing the way the world uses the
> > >>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>> *(tm)*
> > >>
> > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *(tm)*
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
Yes, you are correct that Xen must layer an SR on top of the iSCSI volume
to use it. Same story for VMware: ESX(i) must layer a datastore on top of
the iSCSI volume to use it. I look at it like they are layering a clustered
file system on the SAN volume so the hypervisors can share access to the
contents of the volume.
In 4.1, by the time the hypervisor-attach-volume logic is called, the SR or
datastore has already been created (usually manually by an admin).
This pre-setup of the, say, SR is, of course, not acceptable in an
environment where each CloudStack volume that a user creates is mapped to a
single SAN volume (via the SR).
The question comes down to who should allocate the SR dynamically.
We could have the storage framework ask the storage plug-in if it is
managed. If it is, then the storage framework could send a message to the
hypervisor in question to create (let's talk Xen here) the SR ahead of
time. Then, when the storage framework next sends the attach-volume command
to the hypervisor, it should work without changes to that attach-volume
logic (because - from the point of view of the attach logic - the SR is
already existent, as expected).
Now, as Edison has pointed out, this limits the power of the storage
plug-in. A storage plug-in in this model cannot directly attach storage to
a VM (it must go through the hypervisor). Perhaps that is OK. We need to
make a call on that.
If we want to give more power to the storage vendor, we would have to have
the storage framework call into the storage plug-in to send the appropriate
attach commands to the hypervisor.
Let's discuss and come to a consensus (or at least agree on a path) as soon
as we can. :)
Thanks!
On Fri, Jun 7, 2013 at 4:28 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> My understanding is the Xen expects an SR structure on all ISCSI devices
> -- at least that is how I read the code in your patch. Is my understanding
> correct? If so, the Xen plugin should be able to query the storage device
> to determine the presence of the SR structure and create it if it does not
> exist. Am I missing something in the implementation that makes that type
> of implementation impossible?
>
> Thanks,
> -John
>
> On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Yeah, if a storage vendor wanted to attach a volume directly to a VM in
> 4.2
> > today, it would probably fail because the attach-volume logic assumes the
> > existence of the necessary hypervisor data structure (ex. SR on Xen).
> >
> > If we wanted to enable such an attach, we could do it the way Edison
> > suggests.
> >
> >
> > On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com> wrote:
> >
> >>
> >>
> >>> -----Original Message-----
> >>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>> Sent: Friday, June 07, 2013 2:37 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>
> >>> As we only have three weeks until feature freeze, we should come to a
> >>> consensus on this design point as soon as possible.
> >>>
> >>> Right now, if the storage framework asks my driver it is is managed, it
> >>> will say 'yes.' This means the framework will tell the driver to
> perform
> >>> its management activities. This then means the driver will call into
> the
> >>> host (it doesn't know which hypervisor, by the way) to perform the
> >> activity
> >>> of, say, creating an SR on XenServer or a datastore on ESX.
> >>>
> >>> The driver doesn't know which hypervisor it's talking to, it just
> sends a
> >>> message to the host to perform the necessary pre-attach work.
> >>
> >> Could we just expose a method like "attachVolume/dettachVolume" on the
> >> PrimaryDataStoreDriver driver?
> >> For most of cases, the implementation of each driver would just send
> >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put the
> >> implementation in a base class, so that can be shared by all of this
> >> drivers), and in each hypervisor resource code, which may just call
> >> hypervisor's api to attach the volume to VM, while for certain storage,
> >> like SolidFire, may need to create a SR first, and create volume on it,
> >> then call hypervisor's API to attach volume to VM.
> >> While for some other storage vendor, may want to bypass hypervisor
> during
> >> attaching volume, so inside driver's attachvolume implementation, the
> >> driver can do some magic, such as, directly talk to an agent inside VM
> >> instance, then create a disk inside VM.
> >>
> >> How do you guys think?
> >>
> >>
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> wrote:
> >>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>> To: dev@cloudstack.apache.org
> >>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>
> >>>>> Hi John,
> >>>>>
> >>>>> How's about this:
> >>>>>
> >>>>> The driver can implement an isManaged() method. The
> >>> VolumeManagerImpl
> >>>>> can
> >>>>> call into the driver to see if its managed. If it is, the
> >>>> VolumeManagerImpl
> >>>>> (which is responsible for calling into the hypervisor to attach the
> >> disk)
> >>>>> can call into the hypervisor to create the necessary hypervisor data
> >>>>> structure (ex. for XenServer, a storage repository).
> >>>>
> >>>> The problem here is that storage vendor may work differently with
> >>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> >> other
> >>>> vendor wants to totally bypass hypervisor, and assign the LUN directly
> >> to
> >>>> VM instance, see the discuss(
> >>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>> %3E
> >>>> ).
> >>>> So I would let storage provider to implement attach disk to VM,
> >> instead of
> >>>> implemented by cloudstack itself.
> >>>>
> >>>>
> >>>>>
> >>>>> If that's what you're going for, that works for me. By the way,
> >> Edison's
> >>>>> default storage plug-in (which handles the default storage behavior
> >> in
> >>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
> >>>>> hypervisors. You might want to contact him and inform him of your
> >>>> concerns
> >>>>> or that logic (as is) will make it to production.
> >>>>>
> >>>>> Please let me know if what I wrote in above (for my solution) is OK
> >> with
> >>>>> you. :)
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
> >>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Please see my responses in-line below.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hey John,
> >>>>>>>
> >>>>>>> I still have a bit more testing I'd like to do before I build up
> >> a
> >>>> patch
> >>>>>>> file, but this is the gist of what I've done:
> >>>>>>>
> >>>>>>> * During a volume-attach operation, after VolumeManagerImpl tells
> >>>>>>> VolumeServiceImpl to have the driver create a volume, I have
> >>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> >>>>> managed.
> >>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> >> whatever
> >>>>>>> activity is required. In my case, this includes sending a
> >> message to
> >>>> the
> >>>>>>> host where the VM is running to have, say XenServer, add a
> >> storage
> >>>>>>> repository (based on the IP address of the SAN, the IQN of the
> >> SAN
> >>>>>> volume,
> >>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can
> >> on
> >>>> the
> >>>>>>> storage repository). After this, the normal attach-volume
> >> message is
> >>>> sent
> >>>>>>> to the host by VolumeManagerImpl.
> >>>>>>
> >>>>>> There should be **no** code from a storage driver to a hypervisor.
> >> I
> >>>>>> apologize for the repetition, but we simply can not have hypervisor
> >>>>>> specific code in the storage layer. The circular dependencies
> >> between
> >>>> the
> >>>>>> two layers are not sustainable in the long term. Either the
> >>>> VirtualManager
> >>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >>> coordinate
> >>>>>> volume creation and then populating the SR. Ideally, we can
> >>>> generalize the
> >>>>>> process flow for attaching volumes such that the Xen hypervisor
> >> plugin
> >>>>>> would only implement callbacks to perform the attach action and
> >> create
> >>>>> the
> >>>>>> structure and SR. To my mind, the SolidFire driver should only be
> >>>>>> allocating space and providing information about contents (e.g.
> >> space
> >>>>>> available, space consumed, streams to a URI, file handle for a URI,
> >>>> etc)
> >>>>>> and capabilities.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> * The reverse is performed for a detach-volume command.
> >>>>>>>
> >>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
> >>>>>> Edison's
> >>>>>>> default driver simply does a "return false;". We could add a new
> >>>>>> parameter
> >>>>>>> to the createStoragePool API command, if we want, to remove the
> >>>>>> hard-coded
> >>>>>>> return values in the drivers (although my driver will probably
> >> just
> >>>>>> ignore
> >>>>>>> this parameter and always return true since it wouldn't make
> >> sense
> >>>> for it
> >>>>>>> to ever return false). We'd need another column in the
> >> storage_pool
> >>>>> table
> >>>>>>> to store this value.
> >>>>>>
> >>>>>> Yes, I think we should have a parameter added to the
> >>> createStoragePool
> >>>>>> surfaced to the HTTP API that allows DataStores to be configured
> >> for
> >>>>>> management when their underlying drivers support it. To simplify
> >>>> things,
> >>>>>> this flag should only be mutable when the DataStore is created. It
> >>>> would be
> >>>>>> a bit crazy to take a DataStore from managed to unmanaged after
> >>>> creation.
> >>>>>>
> >>>>>>>
> >>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I agree, John. Just wanted to point out that I have a working
> >> GUI
> >>>> for
> >>>>>> you
> >>>>>>>> to review (in that document), if you'd like to check it out.
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> >> jburwell@basho.com>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> I would like the UIs of two features reviewed together to
> >> ensure
> >>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
> >> and
> >>>>>>>>> storage device provisioned IOPs. I see the potential for
> >>>> confusion,
> >>>>>>>>> and I think a side-by-side Ui review of these features will
> >> help
> >>>>>>>>> minimize any potential confusion.
> >>>>>>>>>
> >>>>>>>>> As I mentioned, the term reconciliation issue will work itself
> >> if
> >>>> it
> >>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> >>>> throttled
> >>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi John,
> >>>>>>>>>>
> >>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent to
> >>> you
> >>>>> the
> >>>>>>>>>> other day to see how the GUI looks for provisioned storage
> >> IOPS.
> >>>>>>>>>>
> >>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> >> and we
> >>>>>>>>> decided
> >>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
> >>>> dialog.
> >>>>>>>>>> Other storage vendors are coming out with QoS, so they should
> >>> be
> >>>>> able
> >>>>>> to
> >>>>>>>>>> leverage this GUI going forward (even if they, say, only use
> >> Max
> >>>>>> IOPS).
> >>>>>>>>>> These fields are optional and can be ignored for storage that
> >>>> does not
> >>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> >>>> admin can
> >>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
> >> IOPS.
> >>>>>>>>>>
> >>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
> >> mine,
> >>>> as
> >>>>>> is
> >>>>>>>>>> decided by the admin).
> >>>>>>>>>>
> >>>>>>>>>> I'm not sure what we can do about these two features
> >> expressing
> >>>> the
> >>>>>>>>> speed
> >>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
> >>>> QoS in
> >>>>>>>>> any
> >>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
> >>>> IOPS).
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>> <jb...@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Wei,
> >>>>>>>>>>>
> >>>>>>>>>>> We have been down the rabbit hole a bit on the
> >>> Storage/Hypervisor
> >>>>>> layer
> >>>>>>>>>>> separation, but we still need to reconcile the behavior of
> >>>> hypervisor
> >>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> >> following
> >>>>>> issues
> >>>>>>>>>>> outstanding:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >>> read/write
> >>>>>>>>> values
> >>>>>>>>>>> where as storage provisioned IOPS are expressed as total
> >> IOPS.
> >>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> >>> storage
> >>>>>> volumes
> >>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>> 3. How should usage data be captured for throttled and
> >>>> provisioned
> >>>>>> IOPS
> >>>>>>>>>>> that will permit providers to discriminate these guaranteed
> >>>>>> operations
> >>>>>>>>> in
> >>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>> 4. What is the user experience for throttled and provisioned
> >>> IOPS
> >>>>>> that
> >>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>
> >>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> >>>> throttled
> >>>>>>>>> IOPS
> >>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
> >>>> items 1
> >>>>>>>>> and 2.
> >>>>>>>>>>> Since the two facilities would not be permitted to operate
> >>>> together,
> >>>>>>>>> they
> >>>>>>>>>>> do not need to be semantically compatible. I think item 3
> >> can be
> >>>>>>>>> resolved
> >>>>>>>>>>> with an additional flag or two on the usage records. As for
> >>>> Item 4,
> >>>>>> I
> >>>>>>>>> am
> >>>>>>>>>>> not familiar with how these two enhancements are depicted in
> >>> the
> >>>>> user
> >>>>>>>>>>> interface. I think we need to review the UI enhancements for
> >>>> both
> >>>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>>
> >>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> >>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >>> CLOUDSTACK-
> >>>>> 2875 ) this
> >>>>>>>>> week.
> >>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
> >> ask
> >>>> for
> >>>>>>>>> second
> >>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Wei
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I believe I understand where you're going with this, John,
> >> and
> >>>>> have
> >>>>>>>>> been
> >>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>>> <jb...@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> "I am fine with the VolumeManager determining whether or
> >>>>> not a
> >>>>>>>>> Volume
> >>>>>>>>>>> is
> >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> >>> actual
> >>>>>>>>> isManaged
> >>>>>>>>>>>>>> method), and asking the device driver to allocate
> >> resources
> >>>> for
> >>>>>> the
> >>>>>>>>>>>>> volume
> >>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged() method
> >>>>> added to
> >>>>>>>>> the
> >>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> >> then the
> >>>>>>>>> storage
> >>>>>>>>>>>>>> framework could call the manage() (or whatever name)
> >>> method
> >>>>> (which
> >>>>>>>>>>> would
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
> >>> this
> >>>>>> would
> >>>>>>>>>>> call
> >>>>>>>>>>>>>> into a new method in the hypervisor code to create, say on
> >>>>>>>>> XenServer,
> >>>>>>>>>>> an
> >>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >>>>> DataStoreDriver.
> >>>>>>>>> Since
> >>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage, I
> >>>> would
> >>>>>> add
> >>>>>>>>> a
> >>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
> >>> man
> >>>>> term
> >>>>>>>>> --
> >>>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> >>> property
> >>>>> of
> >>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>> which can be enabled if/when the underlying
> >>> DataStoreDriver
> >>>>> can be
> >>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>> This approach allows operators to override manageability
> >> of
> >>>>>> devices.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> >> plugin
> >>>>>> would
> >>>>>>>>> be
> >>>>>>>>>>>>>> responsible for composing DataStore/Volume methods to
> >>>>> create any
> >>>>>>>>>>>>>> directories or files necessary for the SR. There should
> >> be no
> >>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> >> earlier,
> >>>> such
> >>>>>>>>>>>>> circular
> >>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>>> <jb...@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> >>>>>> supports n
> >>>>>>>>>>>>>>> devices types with each specific behaviors of m
> >> hypervisors.
> >>>>>> Such
> >>>>>>>>> a
> >>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> >>> beast.
> >>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve the
> >>>>> Storage
> >>>>>> layer
> >>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I am fine with the VolumeManager determining whether
> >>> or
> >>>>> not a
> >>>>>>>>> Volume
> >>>>>>>>>>> is
> >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> >>> actual
> >>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >> resources
> >>>> for
> >>>>>> the
> >>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
> >> to
> >>>>>> indicate
> >>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>> or not it supports management operations. Finally, I
> >> think
> >>>> we
> >>>>>>>>> need to
> >>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
> >>>>>> something
> >>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> >>> managing
> >>>>> the
> >>>>>>>>>>> device,
> >>>>>>>>>>>>>>> but the administrator has elected to leave it unmanaged).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Creation of a structure on the volume should be done in
> >>> the
> >>>>> Xen
> >>>>>>>>>>>>>>> hypervisor module using methods exposed by the Storage
> >>>>> layer to
> >>>>>>>>>>> perform
> >>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
> >> file,
> >>>> etc).
> >>>>>>>>>>> This
> >>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >>>> hypervisor, as
> >>>>>>>>> such,
> >>>>>>>>>>>>>>> should be confined to its implementation. From my
> >>>>> perspective,
> >>>>>>>>>>> nothing
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the Storage layer should be concerned with content. From
> >>> its
> >>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>> structure and data are opaque. It provides the means to
> >>>> query
> >>>>>> the
> >>>>>>>>>>> data
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> support the interpretation of the content by higher-level
> >>>>> layers
> >>>>>>>>> (e.g.
> >>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a composition
> >>> of
> >>>>>>>>>>> operations
> >>>>>>>>>>>>>>> from the Storage layer that varies based on the Volume
> >>>>> storage
> >>>>>>>>>>> protocol
> >>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Alternatively to the way the attach logic is implemented
> >> in
> >>>> my
> >>>>>>>>> patch,
> >>>>>>>>>>> we
> >>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>> we
> >>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> >>>>> hypervisor.
> >>>>>> Before
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>> command is sent, we could check to see if we're dealing
> >>> with
> >>>>>>>>> Dynamic
> >>>>>>>>>>> (or
> >>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
> >>>> "Create
> >>>>>>>>> SR"
> >>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we would
> >>> then
> >>>>>>>>> proceed
> >>>>>>>>>>> to
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> This way the attach logic remains the same and we just
> >> add
> >>>>>> another
> >>>>>>>>>>>>>>> command to the agent code that is called for this
> >> particular
> >>>>> type
> >>>>>>>>> of
> >>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I created a document for a customer today that outlines
> >>> how
> >>>>> the
> >>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> >> use
> >>>> to
> >>>>>>>>> you, as
> >>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I have shared this document with you (you should have
> >>>>> received
> >>>>>>>>> that
> >>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >>>>>> jburwell@basho.com
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
> >>> but
> >>>>>> want
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> >>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> >>>>> coherent
> >>>>>>>>> form
> >>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
> >>>> notion
> >>>>>> of
> >>>>>>>>> a
> >>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
> >> the
> >>>>> notion
> >>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in was
> >>>>>>>>> incomplete,
> >>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> >> device.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> >> this.
> >>>> :)
> >>>>> Do
> >>>>>>>>> you
> >>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> >>> what
> >>>>> you're
> >>>>>>>>> looking
> >>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> >>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
> >>>> notion
> >>>>>> of
> >>>>>>>>> a
> >>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> >>>>> capabilities
> >>>>>>>>> (i.e.
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> >>> managed
> >>>>> by
> >>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
> >>>>> semantic
> >>>>>>>>> that
> >>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed. I
> >>> also
> >>>>> want
> >>>>>>>>> to
> >>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from the
> >>>>> hypervisor
> >>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> >>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Could you please answer the following questions
> >>> for
> >>>>> me
> >>>>>> with
> >>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
> >>> SAN
> >>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN
> >>> 0).
> >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and
> >>>>> CloudStack
> >>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> >>> server or
> >>>>>>>>> externally
> >>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> >>> volume
> >>>>> (same
> >>>>>> as a
> >>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>> created by the management server (via the plug-in)
> >>>>> the
> >>>>>> first
> >>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
> >>>> still
> >>>>>>>>> want
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> >>> today
> >>>>>> (prior
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
> >>> this
> >>>>> case -
> >>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While you
> >>> can do
> >>>>> this
> >>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management server
> >>> or
> >>>>> externally
> >>>>>>>>> by
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> >>> volumes
> >>>>> are
> >>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in). There
> >>> is
> >>>>> no
> >>>>>> admin
> >>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> >>> volume
> >>>>> and a
> >>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> >>>>> supports
> >>>>>> true
> >>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> >> related
> >>>>> and
> >>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
> >>> VMs
> >>>>> with
> >>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> >>> single
> >>>>> SAN
> >>>>>>>>> volume
> >>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> >>> multiple
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
> >>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
> >>> house
> >>>>> more
> >>>>>>>>> than
> >>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> >>> without
> >>>>> any of
> >>>>>> my
> >>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
> >>> for
> >>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me
> >>> --
> >>>>>> concepts
> >>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
> >>>> only
> >>>>>> used
> >>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not
> >>>>> PreSetup an
> >>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> >>>>> VMware, it
> >>>>>>>>> uses
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> >>> KVM,
> >>>>> it uses
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> >>> here
> >>>>> was to
> >>>>>>>>> make a
> >>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> >>> Edison
> >>>>> added
> >>>>>>>>> into
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> >>> specificy,
> >>>>> though:
> >>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> >>> Tutkowski
> >>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
> >>> or
> >>>>> care
> >>>>>> that
> >>>>>>>>> it
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> >>>>> questions
> >>>>>>>>> about
> >>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> >>>>> SolidFire
> >>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> >>> hypervisor
> >>>>> logic
> >>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint
> >>>>> and just
> >>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
> >>> the
> >>>>> hypervisor
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
> >>> the
> >>>>> necessary
> >>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> >>> Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> >>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to
> >>>>> manage a
> >>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> >>> CloudStack
> >>>>> (e.g. the
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> >>> chosen
> >>>>> to
> >>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> >>> make
> >>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
> >>>>> volume via
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
> >>>>> work unless
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
> >>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> >>>>> CloudStack, an
> >>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> >>> exposes it
> >>>>>> through
> >>>>>>>>> a
> >>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
> >>>>> CloudStack,
> >>>>>>>>> but
> >>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> >>>>> changes, we
> >>>>>>>>> would
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> >>>>> necessary
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have
> >>> the
> >>>>> storage
> >>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
> >>> from
> >>>>> the
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
> >>> create
> >>>>> the
> >>>>>>>>> SR
> >>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin
> >>>>> should
> >>>>>> have a
> >>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> >>> available
> >>>>> for
> >>>>>> volume
> >>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >>> interrogation of
> >>>>> the
> >>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
> >>>>> shouldn't
> >>>>>>>>> know
> >>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able
> >>> to
> >>>>>> answer
> >>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> >>> driver
> >>>>> (which
> >>>>>>>>> is a
> >>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
> >>> new
> >>>>> Primary
> >>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> >>> that
> >>>>>> references
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
> >>>>> volume. This
> >>>>>>>>> leads
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack
> >>>>> volume to a
> >>>>>>>>> VM
> >>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
> >>> the
> >>>>> plug-in
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> >>> SAN).
> >>>>> The
> >>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data
> >>>>> (like the
> >>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> >>>>>> CloudStack
> >>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
> >>>>> volume
> >>>>>>>>> will be
> >>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the
> >>>>> plug-in
> >>>>>>>>> code
> >>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and
> >>>>> detects
> >>>>>> the
> >>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
> >>> of a
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> >>>>> terminology,
> >>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> >>>>> allocation on
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> >>> mean
> >>>>> 'SAN
> >>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
> >>> can
> >>>>> make in
> >>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
> >>> make
> >>>>> sense, but
> >>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on
> >>>>> this
> >>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> >>> approach I
> >>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> >>> John
> >>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
> >>>>> distinct
> >>>>>>>>> concepts
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols.
> >>> In
> >>>>> both
> >>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
> >>>>>> create/delete
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the
> >>>>> volume to
> >>>>>> be
> >>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
> >>> To
> >>>>> ensure
> >>>>>>>>> that
> >>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> >>>>> definitions,
> >>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
> >>>>> Minimally,
> >>>>>> we
> >>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device
> >>> to
> >>>>> move bits
> >>>>>>>>> from
> >>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >>> Optionally, a
> >>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore,
> >>>>> we can have
> >>>>>> a
> >>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
> >>> we
> >>>>> can have a
> >>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> >>> capable
> >>>>> of
> >>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
> >>> may
> >>>>> be able
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> >>>>> unmanaged by
> >>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
> >>>>> operator has
> >>>>>>>>> chosen
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
> >>> my
> >>>>>> reasoning
> >>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
> >>>>> reasonable, it
> >>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
> >>> from
> >>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
> >>> driver
> >>>>> that
> >>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> >>> establish
> >>>>> a
> >>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that
> >>> is
> >>>>>>>>> associated
> >>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
> >>>>> drivers can
> >>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
> >>> a
> >>>>> device
> >>>>>> managed
> >>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
> >>> by
> >>>>> the driver to
> >>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> >>> sense,
> >>>>> then we can
> >>>>>>>>> sit
> >>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
> >>> the
> >>>>> current
> >>>>>> iSCSI
> >>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
> >>> and
> >>>>> create
> >>>>>> a
> >>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time,
> >>>>> the
> >>>>>> iSCSI
> >>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated).
> >>>>> Future
> >>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created
> >>>>> behind the
> >>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
> >>> go in
> >>>>> and
> >>>>>> create
> >>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete
> >>>>> volumes on
> >>>>>>>>> its
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
> >>>>> storage
> >>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
> >>> was
> >>>>> created,
> >>>>>> the
> >>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
> >>> After
> >>>>> this is
> >>>>>>>>> done,
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called.
> >>>>> No
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
> >>> was
> >>>>> just
> >>>>>>>>> created.
> >>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
> >>> Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should
> >>>>> probably
> >>>>>> have
> >>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> >>> pre-
> >>>>> allocated
> >>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> >>> hypervisor
> >>>>> data
> >>>>>>>>> structure
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
> >>> then
> >>>>> that
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
> >>> (VDI)
> >>>>> are later
> >>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> >>> these
> >>>>> cases, the
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> >>> already
> >>>>> in place
> >>>>>>>>> and
> >>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure
> >>> to
> >>>>> the VM in
> >>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
> >>> not
> >>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
> >>> be a
> >>>>> SAN
> >>>>>> (not
> >>>>>>>>> a
> >>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
> >>>>> structure
> >>>>>>>>> must be
> >>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
> >>> these
> >>>>> types of
> >>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> >>>>> structure by an
> >>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer
> >>> for
> >>>>> this
> >>>>>>>>> volume,
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM
> >>> in
> >>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
> >>>>> John Burwell
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of
> >>> the
> >>>>> Dynamic
> >>>>>> type
> >>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
> >>> ask
> >>>>> about the
> >>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
> >>>>> definition
> >>>>>> of
> >>>>>>>>> a
> >>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> >>> Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type
> >>> is
> >>>>>> uniquely
> >>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> >>> Storage
> >>>>> that is
> >>>>>>>>> directly
> >>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> >>> step of
> >>>>> going
> >>>>>> to
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
> >>> that
> >>>>> iSCSI
> >>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> >>>>> supported for
> >>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go
> >>> to
> >>>>> the
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
> >>> PM,
> >>>>> John
> >>>>>> Burwell
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
> >>> what
> >>>>> is the
> >>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
> >>> RBD
> >>>>> considered
> >>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
> >>> Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
> >>>>> defined in
> >>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> >>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
> >>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
> >>> //
> >>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
> >>> LUN,
> >>>>> with a
> >>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
> >>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
> >>> local
> >>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
> >>> VMFS
> >>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> >>>>> XenServer, Storage
> >>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
> >>> local
> >>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
> >>>>> zone-wide
> >>>>>>>>> storage
> >>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
> >>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41
> >>> PM,
> >>>>> Mike
> >>>>>>>>> Tutkowski
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> >>> another
> >>>>> storage
> >>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> >>> Quality
> >>>>> of
> >>>>>> Service
> >>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well.
> >>>>> They would
> >>>>>>>>> need
> >>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
> >>>>> necessary
> >>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>> 1:39
> >>>>> PM, Mike
> >>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
> >>>>> XenServer code
> >>>>>> is
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
> >>> logic is
> >>>>> now aware
> >>>>>>>>> of
> >>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> >>>>> implementation of
> >>>>>>>>> this
> >>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> >>> with
> >>>>> Edison's
> >>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
> >>>>> framework that
> >>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
> >>>>> However, when I
> >>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
> >>> it
> >>>>> was not
> >>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could
> >>> do
> >>>>> with these
> >>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
> >>> this
> >>>>> new type of
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>> 1:28
> >>>>> PM, John
> >>>>>>>>> Burwell
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me
> >>>>> that any
> >>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> >>> about
> >>>>> details. I
> >>>>>>>>> expect
> >>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> >>> operations
> >>>>> that can be
> >>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> >>>>> allocate
> >>>>>> space/create
> >>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> >>>>> dependency, we are
> >>>>>>>>> going
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make
> >>> the
> >>>>> system
> >>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> >>>>> understanding that the
> >>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >>>>> CitrixResourceBase
> >>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>> 3:12
> >>>>> PM, Mike
> >>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
> >>> bit
> >>>>> more
> >>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
> >>> from
> >>>>> one setup
> >>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> >>> volume on
> >>>>> the
> >>>>>> storage
> >>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> >>> structure
> >>>>> to make
> >>>>>> use
> >>>>>>>>> of
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> >>> XenServer
> >>>>> or a
> >>>>>> datastore
> >>>>>>>>> on
> >>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
> >>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> >>> storage
> >>>>> framework,
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> >>> create
> >>>>> a 1:1
> >>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
> >>> data
> >>>>> disk
> >>>>>>>>> (necessary
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
> >>> plug-
> >>>>> in that is
> >>>>>>>>> called
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
> >>>>> storage
> >>>>>> framework
> >>>>>>>>> did
> >>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >>> hypervisor-
> >>>>> specific data
> >>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
> >>> I've
> >>>>> been
> >>>>>>>>> enhancing
> >>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> >>>>> submitted. I've got
> >>>>>>>>> ESX(i)
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
> >>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in -
> >>> by
> >>>>> itself - is
> >>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
> >>>>> volume-attach
> >>>>>>>>> logic
> >>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
> >>> the
> >>>>> plug-in to
> >>>>>>>>> have
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> >>> comes to
> >>>>> attach the
> >>>>>>>>> volume
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> >>> enough
> >>>>> to recognize
> >>>>>>>>> it's
> >>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
> >>> has
> >>>>> just been
> >>>>>>>>> created)
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> >>>>> XenServer) or a
> >>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just
> >>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
> >>> most
> >>>>> of my time
> >>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> >>> at
> >>>>> 12:48 PM,
> >>>>>> John
> >>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> >>> why
> >>>>> the the storage
> >>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> >>> 1:21
> >>>>> PM, Mike
> >>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
> >>>>> would like to
> >>>>>>>>> support
> >>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
> >>> I'm
> >>>>> just out of
> >>>>>>>>> time
> >>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> >>> plug-in
> >>>>> supports
> >>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> >>> feature
> >>>>> freeze, I
> >>>>>>>>> have
> >>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
> >>> clear,
> >>>>> this is
> >>>>>> all
> >>>>>>>>> in
> >>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute
> >>>>> Offerings post
> >>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>> 2013
> >>>>> at 11:14 AM,
> >>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan
> >>> on
> >>>>> supporting
> >>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> >>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> >>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
> >>> June
> >>>>> 3, 2013
> >>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> >>> [MERGE]
> >>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> >>> merging
> >>>>> Wei's feature
> >>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
> >>> for
> >>>>> KVM only,
> >>>>>>>>> then
> >>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>> 2013
> >>>>> at 8:55 AM,
> >>>>>> Wei
> >>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
> >>> as
> >>>>> no one works
> >>>>>>>>> on
> >>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
> >>>>> themselves. They
> >>>>>>>>> can
> >>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of
> >>> a
> >>>>> VMs and
> >>>>>> volumes
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
> >>>>> exact
> >>>>>> limitation
> >>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
> >>> for
> >>>>> nics as
> >>>>>> well.
> >>>>>>>>> I
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
> >>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> >>>>> misunstand the second
> >>>>>> the
> >>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> >>> what
> >>>>> you said about
> >>>>>> the
> >>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> >>>>> Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> >>> at
> >>>>> 2:13 AM, Wei
> >>>>>>>>> ZHOU
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's
> >>>>> aswer helps you.
> >>>>>>>>> I am
> >>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
> >>> billing
> >>>>> should
> >>>>>>>>> depend
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
> >>> Please
> >>>>> review
> >>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
> >>>>> statistics
> >>>>>> including
> >>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> >>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
> >>> AWS
> >>>>> model,
> >>>>>> customers
> >>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>> IOPS,
> >>>>> as well as,
> >>>>>> for
> >>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://aws.amazon.com/ebs/).
> >>>>>> I
> >>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> >>> employ
> >>>>> similar cost
> >>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
> >>> in
> >>>>> the current
> >>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
> >>> mean
> >>>>> IOPS runtime
> >>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for
> >>> a
> >>>>> running
> >>>>>> virtual
> >>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>> does
> >>>>> not support
> >>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
> >>>>> offering or disk
> >>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the
> >>>>> Java interface
> >>>>>>>>> level.
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> >>>>> generalize allocation
> >>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
> >>>>> service guarantees
> >>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> >>> good
> >>>>> question.
> >>>>>>>>> Maybe it
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> disk_io_throttling as Mike
> >>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
> >>> on
> >>>>> my thoughts
> >>>>>> in a
> >>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
> >>>>> between these two
> >>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be
> >>>>> merged before
> >>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
> >>>>> between the
> >>>>>> branches
> >>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> disk_io_throttling)
> >>>>> to
> >>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> >>> John
> >>>>> Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I
> >>>>> want to
> >>>>>> understand
> >>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
> >>>>> value in
> >>>>>> capturing
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model
> >>> (e.g.
> >>>>> for
> >>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
> >>> there
> >>>>> be a common
> >>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning
> >>> at
> >>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
> >>>>> conflicting
> >>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>> and
> >>>>> storage device
> >>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
> >>>>> believe (and
> >>>>>> billed)
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>> device
> >>>>> has been
> >>>>>>>>> configured
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
> >>>>> configuration
> >>>>>>>>> between a
> >>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
> >>>>> later
> >>>>>> modification
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
> >>> 2013,
> >>>>> at 8:38 PM,
> >>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
> >>> Wei's
> >>>>> feature deals
> >>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>> hypervisor
> >>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature
> >>> is
> >>>>> focused on
> >>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
> >>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun
> >>> 2,
> >>>>> 2013 at 6:35
> >>>>>> PM,
> >>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> >>> opinion is
> >>>>> that no
> >>>>>> features
> >>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> >>> have
> >>>>> been resolved
> >>>>>> and
> >>>>>>>>> it
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
> >>> vs
> >>>>> discrete
> >>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
> >>> don't
> >>>>> think this
> >>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how
> >>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> >>> involves
> >>>>> provisioned
> >>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>> where provisioned
> >>>>>>>>> IOPS
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
> >>> 2013,
> >>>>> at 6:47 AM,
> >>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will
> >>>>> change it next
> >>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
> >>>>> Wido den Hollander
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
> >>> have
> >>>>> pushed the
> >>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
> >>>>> object thems for
> >>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> >>> sake
> >>>>> of clarity I
> >>>>>> just
> >>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
> >>> IOps
> >>>>> to R/W IOps
> >>>>>>>>> asap
> >>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
> >>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
> >>> the
> >>>>> groundwork for
> >>>>>>>>> the
> >>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
> >>>>> prevent that we
> >>>>>> create
> >>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> >>>>> Wido den
> >>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
> >>>>> enhancement must
> >>>>>> to
> >>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>> how
> >>>>> it could be
> >>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
> >>> IOPS
> >>>>> into read
> >>>>>> and
> >>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>> unless/until we
> >>>>>> decided
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei
> >>>>> meant was merging
> >>>>>> it
> >>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch
> >>>>> and add Read /
> >>>>>>>>> Write
> >>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> >>>>> released with Read
> >>>>>> and
> >>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
> >>>>> make the May 31st
> >>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other
> >>>>> threads) then it
> >>>>>>>>> won't
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> >>>>> completely agree that
> >>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31,
> >>>>> 2013, at 9:51 AM,
> >>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> >>> Good
> >>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
> >>>>> about at the
> >>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>> of
> >>>>> read and write
> >>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
> >>>>> difference of sent
> >>>>>>>>> and
> >>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> >>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is
> >>>>> much more
> >>>>>>>>> expensive
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> Implementing
> >>>>> it will be some
> >>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> implemented
> >>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
> >>> For
> >>>>> the deadline
> >>>>>> of
> >>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
> >>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's
> >>> a
> >>>>> feature we
> >>>>>>>>> can't
> >>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>> we
> >>>>> have to make sure
> >>>>>> we
> >>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
> >>> that
> >>>>> will confuse
> >>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> >>>>> Wido den
> >>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
> >>> like to
> >>>>> merge
> >>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
> >>>>> object, I will
> >>>>>> merge
> >>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>> purpose
> >>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> >>>>> machines are running
> >>>>>>>>> on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> >>> strage).
> >>>>> Because of
> >>>>>> the
> >>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
> >>> has
> >>>>> large disk
> >>>>>>>>> operation,
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs
> >>>>> running on the
> >>>>>> same
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>> neccesary
> >>>>> to set the
> >>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at
> >>> the
> >>>>> code I see
> >>>>>> you
> >>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and
> >>>>> libvirt support
> >>>>>>>>> setting
> >>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>> which
> >>>>> could benefit a
> >>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
> >>>>> strange, in the
> >>>>>>>>> polling
> >>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
> >>> on
> >>>>> the throttling
> >>>>>>>>> side
> >>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps
> >>> are
> >>>>> usually much
> >>>>>>>>> more
> >>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid
> >>>>> use-case where
> >>>>>>>>> that
> >>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
> >>> Read
> >>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
> >>> only
> >>>>> supports KVM
> >>>>>>>>> at
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at
> >>>>> least have the
> >>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
> >>>>> would be a lot of
> >>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> >>> hypervisor
> >>>>> doesn't
> >>>>>>>>> support
> >>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
> >>> can
> >>>>> always sum
> >>>>>> both
> >>>>>>>>> up
> >>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> >>>>> explain why you
> >>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>> feature
> >>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> >>>>> maximum rate of
> >>>>>>>>> VMs
> >>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change
> >>> the
> >>>>> maximum rate
> >>>>>> of
> >>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
> >>> the
> >>>>> disk rate
> >>>>>> (total
> >>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
> >>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>
> >>>>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>>
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
> >>> check
> >>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
> >>>>> check the branch's
> >>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> >>> there
> >>>>> new dependencies
> >>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> >>>>> automated testing
> >>>>>> (unit
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
> >>> are
> >>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> >>>>> testing has been done
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> >>>>> bytes rate and
> >>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
> >>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
> >>> stop,
> >>>>> start, reboot,
> >>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> >>> Volume
> >>>>> operations,
> >>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> >>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review
> >>> the
> >>>>> code, you can
> >>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> >>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>
> >>>>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>>
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>> 2071
> >>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>> 2071
> >>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
> >>> Disk
> >>>>> I/O
> >>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >>> the
> >>>>> way the world
> >>>>>>>>> uses
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>> CloudStack
> >>>>> Developer,
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>> way
> >>>>> the world uses
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>> CloudStack
> >>>>> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>> way
> >>>>> the world uses
> >>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>> Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >>> the
> >>>>> world uses the
> >>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>> Developer,
> >>>>>> SolidFire
> >>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >>> the
> >>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>> Developer, SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>>> world uses the
> >>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>> Developer,
> >>>>> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>> world
> >>>>> uses the
> >>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>> Developer,
> >>>>> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>> world
> >>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire
> >>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>> uses
> >>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>> uses
> >>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >>> the
> >>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *(tm)*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *(tm)*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *(tm)*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *(tm)*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *(tm)*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
My understanding is the Xen expects an SR structure on all ISCSI devices -- at least that is how I read the code in your patch. Is my understanding correct? If so, the Xen plugin should be able to query the storage device to determine the presence of the SR structure and create it if it does not exist. Am I missing something in the implementation that makes that type of implementation impossible?
Thanks,
-John
On Jun 7, 2013, at 6:18 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Yeah, if a storage vendor wanted to attach a volume directly to a VM in 4.2
> today, it would probably fail because the attach-volume logic assumes the
> existence of the necessary hypervisor data structure (ex. SR on Xen).
>
> If we wanted to enable such an attach, we could do it the way Edison
> suggests.
>
>
> On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com> wrote:
>
>>
>>
>>> -----Original Message-----
>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>> Sent: Friday, June 07, 2013 2:37 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>
>>> As we only have three weeks until feature freeze, we should come to a
>>> consensus on this design point as soon as possible.
>>>
>>> Right now, if the storage framework asks my driver it is is managed, it
>>> will say 'yes.' This means the framework will tell the driver to perform
>>> its management activities. This then means the driver will call into the
>>> host (it doesn't know which hypervisor, by the way) to perform the
>> activity
>>> of, say, creating an SR on XenServer or a datastore on ESX.
>>>
>>> The driver doesn't know which hypervisor it's talking to, it just sends a
>>> message to the host to perform the necessary pre-attach work.
>>
>> Could we just expose a method like "attachVolume/dettachVolume" on the
>> PrimaryDataStoreDriver driver?
>> For most of cases, the implementation of each driver would just send
>> attachvolumecommand/dettachvolumecmd to hypervisor(we can put the
>> implementation in a base class, so that can be shared by all of this
>> drivers), and in each hypervisor resource code, which may just call
>> hypervisor's api to attach the volume to VM, while for certain storage,
>> like SolidFire, may need to create a SR first, and create volume on it,
>> then call hypervisor's API to attach volume to VM.
>> While for some other storage vendor, may want to bypass hypervisor during
>> attaching volume, so inside driver's attachvolume implementation, the
>> driver can do some magic, such as, directly talk to an agent inside VM
>> instance, then create a disk inside VM.
>>
>> How do you guys think?
>>
>>
>>>
>>>
>>>
>>>
>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com> wrote:
>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>
>>>>> Hi John,
>>>>>
>>>>> How's about this:
>>>>>
>>>>> The driver can implement an isManaged() method. The
>>> VolumeManagerImpl
>>>>> can
>>>>> call into the driver to see if its managed. If it is, the
>>>> VolumeManagerImpl
>>>>> (which is responsible for calling into the hypervisor to attach the
>> disk)
>>>>> can call into the hypervisor to create the necessary hypervisor data
>>>>> structure (ex. for XenServer, a storage repository).
>>>>
>>>> The problem here is that storage vendor may work differently with
>>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
>> other
>>>> vendor wants to totally bypass hypervisor, and assign the LUN directly
>> to
>>>> VM instance, see the discuss(
>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>>> %3E
>>>> ).
>>>> So I would let storage provider to implement attach disk to VM,
>> instead of
>>>> implemented by cloudstack itself.
>>>>
>>>>
>>>>>
>>>>> If that's what you're going for, that works for me. By the way,
>> Edison's
>>>>> default storage plug-in (which handles the default storage behavior
>> in
>>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
>>>>> hypervisors. You might want to contact him and inform him of your
>>>> concerns
>>>>> or that logic (as is) will make it to production.
>>>>>
>>>>> Please let me know if what I wrote in above (for my solution) is OK
>> with
>>>>> you. :)
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Please see my responses in-line below.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey John,
>>>>>>>
>>>>>>> I still have a bit more testing I'd like to do before I build up
>> a
>>>> patch
>>>>>>> file, but this is the gist of what I've done:
>>>>>>>
>>>>>>> * During a volume-attach operation, after VolumeManagerImpl tells
>>>>>>> VolumeServiceImpl to have the driver create a volume, I have
>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
>>>>> managed.
>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
>> whatever
>>>>>>> activity is required. In my case, this includes sending a
>> message to
>>>> the
>>>>>>> host where the VM is running to have, say XenServer, add a
>> storage
>>>>>>> repository (based on the IP address of the SAN, the IQN of the
>> SAN
>>>>>> volume,
>>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can
>> on
>>>> the
>>>>>>> storage repository). After this, the normal attach-volume
>> message is
>>>> sent
>>>>>>> to the host by VolumeManagerImpl.
>>>>>>
>>>>>> There should be **no** code from a storage driver to a hypervisor.
>> I
>>>>>> apologize for the repetition, but we simply can not have hypervisor
>>>>>> specific code in the storage layer. The circular dependencies
>> between
>>>> the
>>>>>> two layers are not sustainable in the long term. Either the
>>>> VirtualManager
>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>>> coordinate
>>>>>> volume creation and then populating the SR. Ideally, we can
>>>> generalize the
>>>>>> process flow for attaching volumes such that the Xen hypervisor
>> plugin
>>>>>> would only implement callbacks to perform the attach action and
>> create
>>>>> the
>>>>>> structure and SR. To my mind, the SolidFire driver should only be
>>>>>> allocating space and providing information about contents (e.g.
>> space
>>>>>> available, space consumed, streams to a URI, file handle for a URI,
>>>> etc)
>>>>>> and capabilities.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> * The reverse is performed for a detach-volume command.
>>>>>>>
>>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
>>>>>> Edison's
>>>>>>> default driver simply does a "return false;". We could add a new
>>>>>> parameter
>>>>>>> to the createStoragePool API command, if we want, to remove the
>>>>>> hard-coded
>>>>>>> return values in the drivers (although my driver will probably
>> just
>>>>>> ignore
>>>>>>> this parameter and always return true since it wouldn't make
>> sense
>>>> for it
>>>>>>> to ever return false). We'd need another column in the
>> storage_pool
>>>>> table
>>>>>>> to store this value.
>>>>>>
>>>>>> Yes, I think we should have a parameter added to the
>>> createStoragePool
>>>>>> surfaced to the HTTP API that allows DataStores to be configured
>> for
>>>>>> management when their underlying drivers support it. To simplify
>>>> things,
>>>>>> this flag should only be mutable when the DataStore is created. It
>>>> would be
>>>>>> a bit crazy to take a DataStore from managed to unmanaged after
>>>> creation.
>>>>>>
>>>>>>>
>>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>> wrote:
>>>>>>>
>>>>>>>> I agree, John. Just wanted to point out that I have a working
>> GUI
>>>> for
>>>>>> you
>>>>>>>> to review (in that document), if you'd like to check it out.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
>> jburwell@basho.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> I would like the UIs of two features reviewed together to
>> ensure
>>>>>>>>> consistency across the concepts of hypervisor throttled IOPs
>> and
>>>>>>>>> storage device provisioned IOPs. I see the potential for
>>>> confusion,
>>>>>>>>> and I think a side-by-side Ui review of these features will
>> help
>>>>>>>>> minimize any potential confusion.
>>>>>>>>>
>>>>>>>>> As I mentioned, the term reconciliation issue will work itself
>> if
>>>> it
>>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
>>>> throttled
>>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi John,
>>>>>>>>>>
>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent to
>>> you
>>>>> the
>>>>>>>>>> other day to see how the GUI looks for provisioned storage
>> IOPS.
>>>>>>>>>>
>>>>>>>>>> Several months ago, I put this topic out on the e-mail list
>> and we
>>>>>>>>> decided
>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
>>>> dialog.
>>>>>>>>>> Other storage vendors are coming out with QoS, so they should
>>> be
>>>>> able
>>>>>> to
>>>>>>>>>> leverage this GUI going forward (even if they, say, only use
>> Max
>>>>>> IOPS).
>>>>>>>>>> These fields are optional and can be ignored for storage that
>>>> does not
>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
>>>> admin can
>>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
>> IOPS.
>>>>>>>>>>
>>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
>> mine,
>>>> as
>>>>>> is
>>>>>>>>>> decided by the admin).
>>>>>>>>>>
>>>>>>>>>> I'm not sure what we can do about these two features
>> expressing
>>>> the
>>>>>>>>> speed
>>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
>>>> QoS in
>>>>>>>>> any
>>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
>>>> IOPS).
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>>> <jb...@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Wei,
>>>>>>>>>>>
>>>>>>>>>>> We have been down the rabbit hole a bit on the
>>> Storage/Hypervisor
>>>>>> layer
>>>>>>>>>>> separation, but we still need to reconcile the behavior of
>>>> hypervisor
>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
>> following
>>>>>> issues
>>>>>>>>>>> outstanding:
>>>>>>>>>>>
>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>>> read/write
>>>>>>>>> values
>>>>>>>>>>> where as storage provisioned IOPS are expressed as total
>> IOPS.
>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
>>> storage
>>>>>> volumes
>>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>>> 3. How should usage data be captured for throttled and
>>>> provisioned
>>>>>> IOPS
>>>>>>>>>>> that will permit providers to discriminate these guaranteed
>>>>>> operations
>>>>>>>>> in
>>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>>> 4. What is the user experience for throttled and provisioned
>>> IOPS
>>>>>> that
>>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>>
>>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
>>>> throttled
>>>>>>>>> IOPS
>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
>>>> items 1
>>>>>>>>> and 2.
>>>>>>>>>>> Since the two facilities would not be permitted to operate
>>>> together,
>>>>>>>>> they
>>>>>>>>>>> do not need to be semantically compatible. I think item 3
>> can be
>>>>>>>>> resolved
>>>>>>>>>>> with an additional flag or two on the usage records. As for
>>>> Item 4,
>>>>>> I
>>>>>>>>> am
>>>>>>>>>>> not familiar with how these two enhancements are depicted in
>>> the
>>>>> user
>>>>>>>>>>> interface. I think we need to review the UI enhancements for
>>>> both
>>>>>>>>>>> enhancements and ensure they are consistent.
>>>>>>>>>>>
>>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com>
>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> John and Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>>> CLOUDSTACK-
>>>>> 2875 ) this
>>>>>>>>> week.
>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and
>> ask
>>>> for
>>>>>>>>> second
>>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>>
>>>>>>>>>>>> -Wei
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>
>>>>>>>>>>>>> I believe I understand where you're going with this, John,
>> and
>>>>> have
>>>>>>>>> been
>>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>>>>> <jb...@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> "I am fine with the VolumeManager determining whether or
>>>>> not a
>>>>>>>>> Volume
>>>>>>>>>>> is
>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>>> actual
>>>>>>>>> isManaged
>>>>>>>>>>>>>> method), and asking the device driver to allocate
>> resources
>>>> for
>>>>>> the
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged() method
>>>>> added to
>>>>>>>>> the
>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
>> then the
>>>>>>>>> storage
>>>>>>>>>>>>>> framework could call the manage() (or whatever name)
>>> method
>>>>> (which
>>>>>>>>>>> would
>>>>>>>>>>>>> be
>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
>>> this
>>>>>> would
>>>>>>>>>>> call
>>>>>>>>>>>>>> into a new method in the hypervisor code to create, say on
>>>>>>>>> XenServer,
>>>>>>>>>>> an
>>>>>>>>>>>>> SR?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>>>>> DataStoreDriver.
>>>>>>>>> Since
>>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage, I
>>>> would
>>>>>> add
>>>>>>>>> a
>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
>>> man
>>>>> term
>>>>>>>>> --
>>>>>>>>>>>>>> something other than volume) methods to
>>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
>>> property
>>>>> of
>>>>>>>>>>>>> DataStore
>>>>>>>>>>>>>> which can be enabled if/when the underlying
>>> DataStoreDriver
>>>>> can be
>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>> This approach allows operators to override manageability
>> of
>>>>>> devices.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
>> plugin
>>>>>> would
>>>>>>>>> be
>>>>>>>>>>>>>> responsible for composing DataStore/Volume methods to
>>>>> create any
>>>>>>>>>>>>>> directories or files necessary for the SR. There should
>> be no
>>>>>>>>>>>>> dependencies
>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
>> earlier,
>>>> such
>>>>>>>>>>>>> circular
>>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>>>>> <jb...@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer that
>>>>>> supports n
>>>>>>>>>>>>>>> devices types with each specific behaviors of m
>> hypervisors.
>>>>>> Such
>>>>>>>>> a
>>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
>>> beast.
>>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve the
>>>>> Storage
>>>>>> layer
>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am fine with the VolumeManager determining whether
>>> or
>>>>> not a
>>>>>>>>> Volume
>>>>>>>>>>> is
>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>>> actual
>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>> method), and asking the device driver to allocate
>> resources
>>>> for
>>>>>> the
>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs
>> to
>>>>>> indicate
>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>> or not it supports management operations. Finally, I
>> think
>>>> we
>>>>>>>>> need to
>>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
>>>>>> something
>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
>>> managing
>>>>> the
>>>>>>>>>>> device,
>>>>>>>>>>>>>>> but the administrator has elected to leave it unmanaged).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Creation of a structure on the volume should be done in
>>> the
>>>>> Xen
>>>>>>>>>>>>>>> hypervisor module using methods exposed by the Storage
>>>>> layer to
>>>>>>>>>>> perform
>>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
>> file,
>>>> etc).
>>>>>>>>>>> This
>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>>>> hypervisor, as
>>>>>>>>> such,
>>>>>>>>>>>>>>> should be confined to its implementation. From my
>>>>> perspective,
>>>>>>>>>>> nothing
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the Storage layer should be concerned with content. From
>>> its
>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>> structure and data are opaque. It provides the means to
>>>> query
>>>>>> the
>>>>>>>>>>> data
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> support the interpretation of the content by higher-level
>>>>> layers
>>>>>>>>> (e.g.
>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a composition
>>> of
>>>>>>>>>>> operations
>>>>>>>>>>>>>>> from the Storage layer that varies based on the Volume
>>>>> storage
>>>>>>>>>>> protocol
>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Alternatively to the way the attach logic is implemented
>> in
>>>> my
>>>>>>>>> patch,
>>>>>>>>>>> we
>>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>> we
>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
>>>>> hypervisor.
>>>>>> Before
>>>>>>>>>>> this
>>>>>>>>>>>>>>> command is sent, we could check to see if we're dealing
>>> with
>>>>>>>>> Dynamic
>>>>>>>>>>> (or
>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
>>>> "Create
>>>>>>>>> SR"
>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we would
>>> then
>>>>>>>>> proceed
>>>>>>>>>>> to
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This way the attach logic remains the same and we just
>> add
>>>>>> another
>>>>>>>>>>>>>>> command to the agent code that is called for this
>> particular
>>>>> type
>>>>>>>>> of
>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I created a document for a customer today that outlines
>>> how
>>>>> the
>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
>> use
>>>> to
>>>>>>>>> you, as
>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have shared this document with you (you should have
>>>>> received
>>>>>>>>> that
>>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>>>>>> jburwell@basho.com
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
>>> but
>>>>>> want
>>>>>>>>> to
>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
>>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
>>>>> coherent
>>>>>>>>> form
>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
>>>> notion
>>>>>> of
>>>>>>>>> a
>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
>> the
>>>>> notion
>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in was
>>>>>>>>> incomplete,
>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
>> device.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
>> this.
>>>> :)
>>>>> Do
>>>>>>>>> you
>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
>>> what
>>>>> you're
>>>>>>>>> looking
>>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
>>>> notion
>>>>>> of
>>>>>>>>> a
>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
>>>>> capabilities
>>>>>>>>> (i.e.
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
>>> managed
>>>>> by
>>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
>>>>> semantic
>>>>>>>>> that
>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed. I
>>> also
>>>>> want
>>>>>>>>> to
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from the
>>>>> hypervisor
>>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following questions
>>> for
>>>>> me
>>>>>> with
>>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
>>> SAN
>>>>>> volumes?
>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN
>>> 0).
>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and
>>>>> CloudStack
>>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
>>> server or
>>>>>>>>> externally
>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
>>> volume
>>>>> (same
>>>>>> as a
>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>>> created by the management server (via the plug-in)
>>>>> the
>>>>>> first
>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
>>>> still
>>>>>>>>> want
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
>>> today
>>>>>> (prior
>>>>>>>>> to
>>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
>>> this
>>>>> case -
>>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While you
>>> can do
>>>>> this
>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management server
>>> or
>>>>> externally
>>>>>>>>> by
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
>>> volumes
>>>>> are
>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in). There
>>> is
>>>>> no
>>>>>> admin
>>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
>>> volume
>>>>> and a
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
>>>>> supports
>>>>>> true
>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
>> related
>>>>> and
>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
>>> VMs
>>>>> with
>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
>>> single
>>>>> SAN
>>>>>>>>> volume
>>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
>>> multiple
>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
>>> house
>>>>> more
>>>>>>>>> than
>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
>>> without
>>>>> any of
>>>>>> my
>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
>>> for
>>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me
>>> --
>>>>>> concepts
>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
>>>> only
>>>>>> used
>>>>>>>>>>> (as
>>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not
>>>>> PreSetup an
>>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
>>>>> VMware, it
>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
>>> KVM,
>>>>> it uses
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
>>> here
>>>>> was to
>>>>>>>>> make a
>>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
>>> Edison
>>>>> added
>>>>>>>>> into
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
>>> specificy,
>>>>> though:
>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
>>> Tutkowski
>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
>>> or
>>>>> care
>>>>>> that
>>>>>>>>> it
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
>>>>> questions
>>>>>>>>> about
>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
>>>>> SolidFire
>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
>>> hypervisor
>>>>> logic
>>>>>>>>>>> knows
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint
>>>>> and just
>>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
>>> the
>>>>> hypervisor
>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
>>> the
>>>>> necessary
>>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
>>> Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to
>>>>> manage a
>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>>> CloudStack
>>>>> (e.g. the
>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
>>> chosen
>>>>> to
>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
>>> make
>>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
>>>>> volume via
>>>>>>>>> the
>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
>>>>> work unless
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
>>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
>>>>> CloudStack, an
>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
>>> exposes it
>>>>>> through
>>>>>>>>> a
>>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
>>>>> CloudStack,
>>>>>>>>> but
>>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
>>>>> changes, we
>>>>>>>>> would
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
>>>>> necessary
>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have
>>> the
>>>>> storage
>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
>>> from
>>>>> the
>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
>>> create
>>>>> the
>>>>>>>>> SR
>>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin
>>>>> should
>>>>>> have a
>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
>>> available
>>>>> for
>>>>>> volume
>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>>> interrogation of
>>>>> the
>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
>>>>> shouldn't
>>>>>>>>> know
>>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able
>>> to
>>>>>> answer
>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
>>> driver
>>>>> (which
>>>>>>>>> is a
>>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
>>> new
>>>>> Primary
>>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
>>> that
>>>>>> references
>>>>>>>>> the
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
>>>>> volume. This
>>>>>>>>> leads
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack
>>>>> volume to a
>>>>>>>>> VM
>>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
>>> the
>>>>> plug-in
>>>>>>>>> to
>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
>>> SAN).
>>>>> The
>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data
>>>>> (like the
>>>>>>>>> IQN of
>>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
>>>>>> CloudStack
>>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
>>>>> volume
>>>>>>>>> will be
>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the
>>>>> plug-in
>>>>>>>>> code
>>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and
>>>>> detects
>>>>>> the
>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
>>> of a
>>>>>>>>> hypervisor
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
>>>>> terminology,
>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
>>>>> allocation on
>>>>>>>>> the
>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
>>> mean
>>>>> 'SAN
>>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
>>> can
>>>>> make in
>>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
>>> make
>>>>> sense, but
>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on
>>>>> this
>>>>>>>>> conversation
>>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>>> approach I
>>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
>>> John
>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
>>>>> distinct
>>>>>>>>> concepts
>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols.
>>> In
>>>>> both
>>>>>>>>> cases, we
>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
>>>>>> create/delete
>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the
>>>>> volume to
>>>>>> be
>>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
>>> To
>>>>> ensure
>>>>>>>>> that
>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
>>>>> definitions,
>>>>>>>>> refers to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
>>>>> Minimally,
>>>>>> we
>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device
>>> to
>>>>> move bits
>>>>>>>>> from
>>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>>> Optionally, a
>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore,
>>>>> we can have
>>>>>> a
>>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
>>> we
>>>>> can have a
>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
>>> capable
>>>>> of
>>>>>>>>> allocating
>>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
>>> may
>>>>> be able
>>>>>>>>> to
>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
>>>>> unmanaged by
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
>>>>> operator has
>>>>>>>>> chosen
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
>>> my
>>>>>> reasoning
>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
>>>>> reasonable, it
>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
>>> from
>>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
>>> driver
>>>>> that
>>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
>>> establish
>>>>> a
>>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that
>>> is
>>>>>>>>> associated
>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
>>>>> drivers can
>>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
>>> a
>>>>> device
>>>>>> managed
>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
>>> by
>>>>> the driver to
>>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
>>> sense,
>>>>> then we can
>>>>>>>>> sit
>>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
>>> the
>>>>> current
>>>>>> iSCSI
>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
>>> and
>>>>> create
>>>>>> a
>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time,
>>>>> the
>>>>>> iSCSI
>>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated).
>>>>> Future
>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created
>>>>> behind the
>>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
>>> go in
>>>>> and
>>>>>> create
>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete
>>>>> volumes on
>>>>>>>>> its
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
>>>>> storage
>>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
>>> was
>>>>> created,
>>>>>> the
>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
>>> After
>>>>> this is
>>>>>>>>> done,
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called.
>>>>> No
>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
>>> was
>>>>> just
>>>>>>>>> created.
>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
>>> Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should
>>>>> probably
>>>>>> have
>>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
>>> pre-
>>>>> allocated
>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
>>> hypervisor
>>>>> data
>>>>>>>>> structure
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
>>> then
>>>>> that
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
>>> (VDI)
>>>>> are later
>>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
>>> these
>>>>> cases, the
>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
>>> already
>>>>> in place
>>>>>>>>> and
>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure
>>> to
>>>>> the VM in
>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
>>> not
>>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
>>> be a
>>>>> SAN
>>>>>> (not
>>>>>>>>> a
>>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
>>>>> structure
>>>>>>>>> must be
>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
>>> these
>>>>> types of
>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
>>>>> structure by an
>>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer
>>> for
>>>>> this
>>>>>>>>> volume,
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM
>>> in
>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
>>>>> John Burwell
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of
>>> the
>>>>> Dynamic
>>>>>> type
>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
>>> ask
>>>>> about the
>>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
>>>>> definition
>>>>>> of
>>>>>>>>> a
>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
>>> Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type
>>> is
>>>>>> uniquely
>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
>>> Storage
>>>>> that is
>>>>>>>>> directly
>>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
>>> step of
>>>>> going
>>>>>> to
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
>>> that
>>>>> iSCSI
>>>>>>>>> target as
>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
>>>>> supported for
>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go
>>> to
>>>>> the
>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
>>> PM,
>>>>> John
>>>>>> Burwell
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
>>> what
>>>>> is the
>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
>>> RBD
>>>>> considered
>>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
>>> Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
>>>>> defined in
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
>>> //
>>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
>>> LUN,
>>>>> with a
>>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
>>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
>>> local
>>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
>>> VMFS
>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
>>>>> XenServer, Storage
>>>>>>>>> Pool
>>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
>>> local
>>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
>>>>> zone-wide
>>>>>>>>> storage
>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
>>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41
>>> PM,
>>>>> Mike
>>>>>>>>> Tutkowski
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
>>> another
>>>>> storage
>>>>>>>>>>> company
>>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
>>> Quality
>>>>> of
>>>>>> Service
>>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well.
>>>>> They would
>>>>>>>>> need
>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
>>>>> necessary
>>>>>>>>> changes to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>> 1:39
>>>>> PM, Mike
>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
>>>>> XenServer code
>>>>>> is
>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
>>> logic is
>>>>> now aware
>>>>>>>>> of
>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
>>>>> implementation of
>>>>>>>>> this
>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
>>> with
>>>>> Edison's
>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
>>>>> framework that
>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
>>>>> However, when I
>>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
>>> it
>>>>> was not
>>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could
>>> do
>>>>> with these
>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
>>> this
>>>>> new type of
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>> 1:28
>>>>> PM, John
>>>>>>>>> Burwell
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me
>>>>> that any
>>>>>>>>>>> operation
>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
>>> about
>>>>> details. I
>>>>>>>>> expect
>>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
>>> operations
>>>>> that can be
>>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
>>>>> allocate
>>>>>> space/create
>>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
>>>>> dependency, we are
>>>>>>>>> going
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make
>>> the
>>>>> system
>>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
>>>>> understanding that the
>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>>>>> CitrixResourceBase
>>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>> 3:12
>>>>> PM, Mike
>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
>>> bit
>>>>> more
>>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
>>> from
>>>>> one setup
>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
>>> volume on
>>>>> the
>>>>>> storage
>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
>>> structure
>>>>> to make
>>>>>> use
>>>>>>>>> of
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>>> XenServer
>>>>> or a
>>>>>> datastore
>>>>>>>>> on
>>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
>>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
>>> storage
>>>>> framework,
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
>>> create
>>>>> a 1:1
>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
>>> data
>>>>> disk
>>>>>>>>> (necessary
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
>>> plug-
>>>>> in that is
>>>>>>>>> called
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
>>>>> storage
>>>>>> framework
>>>>>>>>> did
>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>>> hypervisor-
>>>>> specific data
>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
>>> I've
>>>>> been
>>>>>>>>> enhancing
>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
>>>>> submitted. I've got
>>>>>>>>> ESX(i)
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
>>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in -
>>> by
>>>>> itself - is
>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
>>>>> volume-attach
>>>>>>>>> logic
>>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
>>> the
>>>>> plug-in to
>>>>>>>>> have
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
>>> comes to
>>>>> attach the
>>>>>>>>> volume
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
>>> enough
>>>>> to recognize
>>>>>>>>> it's
>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
>>> has
>>>>> just been
>>>>>>>>> created)
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
>>>>> XenServer) or a
>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just
>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
>>> most
>>>>> of my time
>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>>> at
>>>>> 12:48 PM,
>>>>>> John
>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
>>> why
>>>>> the the storage
>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>>> 1:21
>>>>> PM, Mike
>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
>>>>> would like to
>>>>>>>>> support
>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
>>> I'm
>>>>> just out of
>>>>>>>>> time
>>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
>>> plug-in
>>>>> supports
>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
>>> feature
>>>>> freeze, I
>>>>>>>>> have
>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
>>> clear,
>>>>> this is
>>>>>> all
>>>>>>>>> in
>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute
>>>>> Offerings post
>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>> 2013
>>>>> at 11:14 AM,
>>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan
>>> on
>>>>> supporting
>>>>>>>>> KVM in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
>>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
>>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
>>> June
>>>>> 3, 2013
>>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
>>> [MERGE]
>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
>>> merging
>>>>> Wei's feature
>>>>>>>>>>> first,
>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
>>> for
>>>>> KVM only,
>>>>>>>>> then
>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>> 2013
>>>>> at 8:55 AM,
>>>>>> Wei
>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
>>> as
>>>>> no one works
>>>>>>>>> on
>>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
>>>>> themselves. They
>>>>>>>>> can
>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of
>>> a
>>>>> VMs and
>>>>>> volumes
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
>>>>> exact
>>>>>> limitation
>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
>>> for
>>>>> nics as
>>>>>> well.
>>>>>>>>> I
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
>>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
>>>>> misunstand the second
>>>>>> the
>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
>>> what
>>>>> you said about
>>>>>> the
>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
>>>>> Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
>>> at
>>>>> 2:13 AM, Wei
>>>>>>>>> ZHOU
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's
>>>>> aswer helps you.
>>>>>>>>> I am
>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
>>> billing
>>>>> should
>>>>>>>>> depend
>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
>>> Please
>>>>> review
>>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
>>>>> statistics
>>>>>> including
>>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
>>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
>>> AWS
>>>>> model,
>>>>>> customers
>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>> IOPS,
>>>>> as well as,
>>>>>> for
>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://aws.amazon.com/ebs/).
>>>>>> I
>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
>>> employ
>>>>> similar cost
>>>>>>>>>>> models.
>>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
>>> in
>>>>> the current
>>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
>>> mean
>>>>> IOPS runtime
>>>>>>>>>>> change?
>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for
>>> a
>>>>> running
>>>>>> virtual
>>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>> does
>>>>> not support
>>>>>>>>>>> changing
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
>>>>> offering or disk
>>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the
>>>>> Java interface
>>>>>>>>> level.
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
>>>>> generalize allocation
>>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
>>>>> service guarantees
>>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
>>> good
>>>>> question.
>>>>>>>>> Maybe it
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> disk_io_throttling as Mike
>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
>>> on
>>>>> my thoughts
>>>>>> in a
>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
>>>>> between these two
>>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be
>>>>> merged before
>>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
>>>>> between the
>>>>>> branches
>>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> disk_io_throttling)
>>>>> to
>>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>>> John
>>>>> Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I
>>>>> want to
>>>>>> understand
>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
>>>>> value in
>>>>>> capturing
>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model
>>> (e.g.
>>>>> for
>>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
>>> there
>>>>> be a common
>>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning
>>> at
>>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
>>>>> conflicting
>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>> and
>>>>> storage device
>>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
>>>>> believe (and
>>>>>> billed)
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>> device
>>>>> has been
>>>>>>>>> configured
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
>>>>> configuration
>>>>>>>>> between a
>>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
>>>>> later
>>>>>> modification
>>>>>>>>> to
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
>>> 2013,
>>>>> at 8:38 PM,
>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
>>> Wei's
>>>>> feature deals
>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>> hypervisor
>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature
>>> is
>>>>> focused on
>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
>>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun
>>> 2,
>>>>> 2013 at 6:35
>>>>>> PM,
>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>>> opinion is
>>>>> that no
>>>>>> features
>>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
>>> have
>>>>> been resolved
>>>>>> and
>>>>>>>>> it
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
>>> vs
>>>>> discrete
>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
>>> don't
>>>>> think this
>>>>>>>>> criteria
>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how
>>>>> does this work
>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>> involves
>>>>> provisioned
>>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>> where provisioned
>>>>>>>>> IOPS
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
>>> 2013,
>>>>> at 6:47 AM,
>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will
>>>>> change it next
>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
>>>>> Wido den Hollander
>>>>>> <
>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
>>> have
>>>>> pushed the
>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
>>>>> object thems for
>>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
>>> sake
>>>>> of clarity I
>>>>>> just
>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
>>> IOps
>>>>> to R/W IOps
>>>>>>>>> asap
>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
>>> the
>>>>> groundwork for
>>>>>>>>> the
>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
>>>>> prevent that we
>>>>>> create
>>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
>>>>> Wido den
>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
>>>>> enhancement must
>>>>>> to
>>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>> how
>>>>> it could be
>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
>>> IOPS
>>>>> into read
>>>>>> and
>>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>> unless/until we
>>>>>> decided
>>>>>>>>> to
>>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei
>>>>> meant was merging
>>>>>> it
>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch
>>>>> and add Read /
>>>>>>>>> Write
>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
>>>>> released with Read
>>>>>> and
>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
>>>>> make the May 31st
>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other
>>>>> threads) then it
>>>>>>>>> won't
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
>>>>> completely agree that
>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31,
>>>>> 2013, at 9:51 AM,
>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>> Good
>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
>>>>> about at the
>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>> of
>>>>> read and write
>>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
>>>>> difference of sent
>>>>>>>>> and
>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is
>>>>> much more
>>>>>>>>> expensive
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> Implementing
>>>>> it will be some
>>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> implemented
>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
>>> For
>>>>> the deadline
>>>>>> of
>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's
>>> a
>>>>> feature we
>>>>>>>>> can't
>>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>> we
>>>>> have to make sure
>>>>>> we
>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
>>> that
>>>>> will confuse
>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
>>>>> Wido den
>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
>>> like to
>>>>> merge
>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
>>>>> object, I will
>>>>>> merge
>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>> purpose
>>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
>>>>> machines are running
>>>>>>>>> on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
>>> strage).
>>>>> Because of
>>>>>> the
>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
>>> has
>>>>> large disk
>>>>>>>>> operation,
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs
>>>>> running on the
>>>>>> same
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>> neccesary
>>>>> to set the
>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at
>>> the
>>>>> code I see
>>>>>> you
>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and
>>>>> libvirt support
>>>>>>>>> setting
>>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>> which
>>>>> could benefit a
>>>>>>>>> lot of
>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
>>>>> strange, in the
>>>>>>>>> polling
>>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
>>> on
>>>>> the throttling
>>>>>>>>> side
>>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps
>>> are
>>>>> usually much
>>>>>>>>> more
>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid
>>>>> use-case where
>>>>>>>>> that
>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
>>> Read
>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
>>> only
>>>>> supports KVM
>>>>>>>>> at
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at
>>>>> least have the
>>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
>>>>> would be a lot of
>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
>>> hypervisor
>>>>> doesn't
>>>>>>>>> support
>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
>>> can
>>>>> always sum
>>>>>> both
>>>>>>>>> up
>>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
>>>>> explain why you
>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>> feature
>>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
>>>>> maximum rate of
>>>>>>>>> VMs
>>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change
>>> the
>>>>> maximum rate
>>>>>> of
>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
>>> the
>>>>> disk rate
>>>>>> (total
>>>>>>>>>>> bps
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
>>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>
>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
>>> check
>>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
>>>>> check the branch's
>>>>>>>>> RAT
>>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
>>> there
>>>>> new dependencies
>>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
>>>>> automated testing
>>>>>> (unit
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
>>> are
>>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
>>>>> testing has been done
>>>>>>>>> to
>>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
>>>>> bytes rate and
>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
>>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
>>> stop,
>>>>> start, reboot,
>>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>>> Volume
>>>>> operations,
>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
>>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review
>>> the
>>>>> code, you can
>>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
>>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>
>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>>> 2071
>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>> 2071
>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
>>> Disk
>>>>> I/O
>>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>>> the
>>>>> way the world
>>>>>>>>> uses
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>> CloudStack
>>>>> Developer,
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>> way
>>>>> the world uses
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>> CloudStack
>>>>> Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>> way
>>>>> the world uses
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>> Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>>> the
>>>>> world uses the
>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>> Developer,
>>>>>> SolidFire
>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>>> the
>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>> Developer, SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>>> world uses the
>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>> Developer,
>>>>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>> world
>>>>> uses the
>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>> Developer,
>>>>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>> world
>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>> SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>> uses
>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>> uses
>>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>> SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>>> the
>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>> SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>> SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>
>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *(tm)*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *(tm)*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *(tm)*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *(tm)*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *(tm)*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, if a storage vendor wanted to attach a volume directly to a VM in 4.2
today, it would probably fail because the attach-volume logic assumes the
existence of the necessary hypervisor data structure (ex. SR on Xen).
If we wanted to enable such an attach, we could do it the way Edison
suggests.
On Fri, Jun 7, 2013 at 4:13 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
> > -----Original Message-----
> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > Sent: Friday, June 07, 2013 2:37 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > As we only have three weeks until feature freeze, we should come to a
> > consensus on this design point as soon as possible.
> >
> > Right now, if the storage framework asks my driver it is is managed, it
> > will say 'yes.' This means the framework will tell the driver to perform
> > its management activities. This then means the driver will call into the
> > host (it doesn't know which hypervisor, by the way) to perform the
> activity
> > of, say, creating an SR on XenServer or a datastore on ESX.
> >
> > The driver doesn't know which hypervisor it's talking to, it just sends a
> > message to the host to perform the necessary pre-attach work.
>
> Could we just expose a method like "attachVolume/dettachVolume" on the
> PrimaryDataStoreDriver driver?
> For most of cases, the implementation of each driver would just send
> attachvolumecommand/dettachvolumecmd to hypervisor(we can put the
> implementation in a base class, so that can be shared by all of this
> drivers), and in each hypervisor resource code, which may just call
> hypervisor's api to attach the volume to VM, while for certain storage,
> like SolidFire, may need to create a SR first, and create volume on it,
> then call hypervisor's API to attach volume to VM.
> While for some other storage vendor, may want to bypass hypervisor during
> attaching volume, so inside driver's attachvolume implementation, the
> driver can do some magic, such as, directly talk to an agent inside VM
> instance, then create a disk inside VM.
>
> How do you guys think?
>
>
> >
> >
> >
> >
> > On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com> wrote:
> >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > > Sent: Friday, June 07, 2013 1:14 PM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: Re: [MERGE] disk_io_throttling to MASTER
> > > >
> > > > Hi John,
> > > >
> > > > How's about this:
> > > >
> > > > The driver can implement an isManaged() method. The
> > VolumeManagerImpl
> > > > can
> > > > call into the driver to see if its managed. If it is, the
> > > VolumeManagerImpl
> > > > (which is responsible for calling into the hypervisor to attach the
> disk)
> > > > can call into the hypervisor to create the necessary hypervisor data
> > > > structure (ex. for XenServer, a storage repository).
> > >
> > > The problem here is that storage vendor may work differently with
> > > hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> other
> > > vendor wants to totally bypass hypervisor, and assign the LUN directly
> to
> > > VM instance, see the discuss(
> > > http://mail-archives.apache.org/mod_mbox/cloudstack-
> > dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> > %3E
> > > ).
> > > So I would let storage provider to implement attach disk to VM,
> instead of
> > > implemented by cloudstack itself.
> > >
> > >
> > > >
> > > > If that's what you're going for, that works for me. By the way,
> Edison's
> > > > default storage plug-in (which handles the default storage behavior
> in
> > > > CloudStack (ex. how pre 4.2 works)) does include code that talks to
> > > > hypervisors. You might want to contact him and inform him of your
> > > concerns
> > > > or that logic (as is) will make it to production.
> > > >
> > > > Please let me know if what I wrote in above (for my solution) is OK
> with
> > > > you. :)
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
> > wrote:
> > > >
> > > > > Mike,
> > > > >
> > > > > Please see my responses in-line below.
> > > > >
> > > > > Thanks,
> > > > > -John
> > > > >
> > > > > On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > > mike.tutkowski@solidfire.com>
> > > > > wrote:
> > > > >
> > > > > > Hey John,
> > > > > >
> > > > > > I still have a bit more testing I'd like to do before I build up
> a
> > > patch
> > > > > > file, but this is the gist of what I've done:
> > > > > >
> > > > > > * During a volume-attach operation, after VolumeManagerImpl tells
> > > > > > VolumeServiceImpl to have the driver create a volume, I have
> > > > > > VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> > > > managed.
> > > > > > If it is managed, VolumeServiceImpl has the driver perform
> whatever
> > > > > > activity is required. In my case, this includes sending a
> message to
> > > the
> > > > > > host where the VM is running to have, say XenServer, add a
> storage
> > > > > > repository (based on the IP address of the SAN, the IQN of the
> SAN
> > > > > volume,
> > > > > > etc.) and a single VDI (the VDI consumes all of the space it can
> on
> > > the
> > > > > > storage repository). After this, the normal attach-volume
> message is
> > > sent
> > > > > > to the host by VolumeManagerImpl.
> > > > >
> > > > > There should be **no** code from a storage driver to a hypervisor.
> I
> > > > > apologize for the repetition, but we simply can not have hypervisor
> > > > > specific code in the storage layer. The circular dependencies
> between
> > > the
> > > > > two layers are not sustainable in the long term. Either the
> > > VirtualManager
> > > > > or Xen hypervisor plugin needs to be refactored/modified to
> > coordinate
> > > > > volume creation and then populating the SR. Ideally, we can
> > > generalize the
> > > > > process flow for attaching volumes such that the Xen hypervisor
> plugin
> > > > > would only implement callbacks to perform the attach action and
> create
> > > > the
> > > > > structure and SR. To my mind, the SolidFire driver should only be
> > > > > allocating space and providing information about contents (e.g.
> space
> > > > > available, space consumed, streams to a URI, file handle for a URI,
> > > etc)
> > > > > and capabilities.
> > > > >
> > > > >
> > > > > >
> > > > > > * The reverse is performed for a detach-volume command.
> > > > > >
> > > > > > * Right now I simply "return true;" for isManaged() in my driver.
> > > > > Edison's
> > > > > > default driver simply does a "return false;". We could add a new
> > > > > parameter
> > > > > > to the createStoragePool API command, if we want, to remove the
> > > > > hard-coded
> > > > > > return values in the drivers (although my driver will probably
> just
> > > > > ignore
> > > > > > this parameter and always return true since it wouldn't make
> sense
> > > for it
> > > > > > to ever return false). We'd need another column in the
> storage_pool
> > > > table
> > > > > > to store this value.
> > > > >
> > > > > Yes, I think we should have a parameter added to the
> > createStoragePool
> > > > > surfaced to the HTTP API that allows DataStores to be configured
> for
> > > > > management when their underlying drivers support it. To simplify
> > > things,
> > > > > this flag should only be mutable when the DataStore is created. It
> > > would be
> > > > > a bit crazy to take a DataStore from managed to unmanaged after
> > > creation.
> > > > >
> > > > > >
> > > > > > Sound like I'm in sync with what you were thinking?
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > > > > mike.tutkowski@solidfire.com
> > > > > >> wrote:
> > > > > >
> > > > > >> I agree, John. Just wanted to point out that I have a working
> GUI
> > > for
> > > > > you
> > > > > >> to review (in that document), if you'd like to check it out.
> > > > > >>
> > > > > >> Thanks!
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <
> jburwell@basho.com>
> > > > > wrote:
> > > > > >>
> > > > > >>> Mike,
> > > > > >>>
> > > > > >>> I would like the UIs of two features reviewed together to
> ensure
> > > > > >>> consistency across the concepts of hypervisor throttled IOPs
> and
> > > > > >>> storage device provisioned IOPs. I see the potential for
> > > confusion,
> > > > > >>> and I think a side-by-side Ui review of these features will
> help
> > > > > >>> minimize any potential confusion.
> > > > > >>>
> > > > > >>> As I mentioned, the term reconciliation issue will work itself
> if
> > > it
> > > > > >>> is acceptable that a VM is only permitted utilize hypervisor
> > > throttled
> > > > > >>> IOPs or storage provisioned IOPs.
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> -John
> > > > > >>>
> > > > > >>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > > > > >>> <mi...@solidfire.com> wrote:
> > > > > >>>
> > > > > >>>> Hi John,
> > > > > >>>>
> > > > > >>>> Yeah, when you get a chance, refer to the Google doc I sent to
> > you
> > > > the
> > > > > >>>> other day to see how the GUI looks for provisioned storage
> IOPS.
> > > > > >>>>
> > > > > >>>> Several months ago, I put this topic out on the e-mail list
> and we
> > > > > >>> decided
> > > > > >>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
> > > dialog.
> > > > > >>>> Other storage vendors are coming out with QoS, so they should
> > be
> > > > able
> > > > > to
> > > > > >>>> leverage this GUI going forward (even if they, say, only use
> Max
> > > > > IOPS).
> > > > > >>>> These fields are optional and can be ignored for storage that
> > > does not
> > > > > >>>> support provisioned IOPS. Just like the Disk Size field, the
> > > admin can
> > > > > >>>> choose to allow the end user to fill in Min, Max, and Burst
> IOPS.
> > > > > >>>>
> > > > > >>>> I'm OK if we do an either/or model (either Wei's feature or
> mine,
> > > as
> > > > > is
> > > > > >>>> decided by the admin).
> > > > > >>>>
> > > > > >>>> I'm not sure what we can do about these two features
> expressing
> > > the
> > > > > >>> speed
> > > > > >>>> in different terms. I've never seen a SAN express the IOPS for
> > > QoS in
> > > > > >>> any
> > > > > >>>> way other than total IOPS (i.e. not broken in into read/write
> > > IOPS).
> > > > > >>>>
> > > > > >>>> Thanks
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> > <jb...@basho.com>
> > > > > >>> wrote:
> > > > > >>>>
> > > > > >>>>> Wei,
> > > > > >>>>>
> > > > > >>>>> We have been down the rabbit hole a bit on the
> > Storage/Hypervisor
> > > > > layer
> > > > > >>>>> separation, but we still need to reconcile the behavior of
> > > hypervisor
> > > > > >>>>> throttled I/O and storage provisioned IOPS. I see the
> following
> > > > > issues
> > > > > >>>>> outstanding:
> > > > > >>>>>
> > > > > >>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> > read/write
> > > > > >>> values
> > > > > >>>>> where as storage provisioned IOPS are expressed as total
> IOPS.
> > > > > >>>>> 2. How do we handle VMs with throttled IOPS attached to
> > storage
> > > > > volumes
> > > > > >>>>> with provisioned IOPS?
> > > > > >>>>> 3. How should usage data be captured for throttled and
> > > provisioned
> > > > > IOPS
> > > > > >>>>> that will permit providers to discriminate these guaranteed
> > > > > operations
> > > > > >>> in
> > > > > >>>>> the event they want to bill for it differently?
> > > > > >>>>> 4. What is the user experience for throttled and provisioned
> > IOPS
> > > > > that
> > > > > >>>>> minimizes confusion of these concepts?
> > > > > >>>>>
> > > > > >>>>> My thinking is that a VM can have either utilize hypervisor
> > > throttled
> > > > > >>> IOPS
> > > > > >>>>> or storage provisioned IOPS. This policy would neatly solve
> > > items 1
> > > > > >>> and 2.
> > > > > >>>>> Since the two facilities would not be permitted to operate
> > > together,
> > > > > >>> they
> > > > > >>>>> do not need to be semantically compatible. I think item 3
> can be
> > > > > >>> resolved
> > > > > >>>>> with an additional flag or two on the usage records. As for
> > > Item 4,
> > > > > I
> > > > > >>> am
> > > > > >>>>> not familiar with how these two enhancements are depicted in
> > the
> > > > user
> > > > > >>>>> interface. I think we need to review the UI enhancements for
> > > both
> > > > > >>>>> enhancements and ensure they are consistent.
> > > > > >>>>>
> > > > > >>>>> Do these solutions make sense?
> > > > > >>>>>
> > > > > >>>>> Thanks,
> > > > > >>>>> -John
> > > > > >>>>>
> > > > > >>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com>
> > > > wrote:
> > > > > >>>>>
> > > > > >>>>>> John and Mike,
> > > > > >>>>>>
> > > > > >>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> > > > > >>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> > CLOUDSTACK-
> > > > 2875 ) this
> > > > > >>> week.
> > > > > >>>>>> I will start to develop on iops/bps changes tomorrow, and
> ask
> > > for
> > > > > >>> second
> > > > > >>>>>> merge request after testing.
> > > > > >>>>>>
> > > > > >>>>>> -Wei
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > > > > >>>>>>
> > > > > >>>>>>> I believe I understand where you're going with this, John,
> and
> > > > have
> > > > > >>> been
> > > > > >>>>>>> re-working this section of code today.
> > > > > >>>>>>>
> > > > > >>>>>>> I should be able to run it by you tomorrow.
> > > > > >>>>>>>
> > > > > >>>>>>> Thanks for the comments,
> > > > > >>>>>>> Mike
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > > > <jb...@basho.com>
> > > > > >>>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> Mike,
> > > > > >>>>>>>>
> > > > > >>>>>>>> See my responses in-line below.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Thanks,
> > > > > >>>>>>>> -John
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > > > > >>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>> Hi John,
> > > > > >>>>>>>>
> > > > > >>>>>>>> Thanks for the response.
> > > > > >>>>>>>>
> > > > > >>>>>>>> "I am fine with the VolumeManager determining whether or
> > > > not a
> > > > > >>> Volume
> > > > > >>>>> is
> > > > > >>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> > actual
> > > > > >>> isManaged
> > > > > >>>>>>>> method), and asking the device driver to allocate
> resources
> > > for
> > > > > the
> > > > > >>>>>>> volume
> > > > > >>>>>>>> if it is managed."
> > > > > >>>>>>>>
> > > > > >>>>>>>> Are you thinking you'd like to see an isManaged() method
> > > > added to
> > > > > >>> the
> > > > > >>>>>>>> PrimaryDataStoreDriver interface? If it returns true,
> then the
> > > > > >>> storage
> > > > > >>>>>>>> framework could call the manage() (or whatever name)
> > method
> > > > (which
> > > > > >>>>> would
> > > > > >>>>>>> be
> > > > > >>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
> > this
> > > > > would
> > > > > >>>>> call
> > > > > >>>>>>>> into a new method in the hypervisor code to create, say on
> > > > > >>> XenServer,
> > > > > >>>>> an
> > > > > >>>>>>> SR?
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> I would like to see canBeManaged() : boolean on
> > > > DataStoreDriver.
> > > > > >>> Since
> > > > > >>>>>>>> the notion of Volumes only pertains to primary storage, I
> > > would
> > > > > add
> > > > > >>> a
> > > > > >>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
> > man
> > > > term
> > > > > >>> --
> > > > > >>>>>>>> something other than volume) methods to
> > > > > >>>>> allocate/create/deallocate/delete
> > > > > >>>>>>>> underlying storage. To my mind, managed is a mutable
> > property
> > > > of
> > > > > >>>>>>> DataStore
> > > > > >>>>>>>> which can be enabled if/when the underlying
> > DataStoreDriver
> > > > can be
> > > > > >>>>>>> managed.
> > > > > >>>>>>>> This approach allows operators to override manageability
> of
> > > > > devices.
> > > > > >>>>>>>>
> > > > > >>>>>>>> In terms of orchestration/process flow for SR, the Xen
> plugin
> > > > > would
> > > > > >>> be
> > > > > >>>>>>>> responsible for composing DataStore/Volume methods to
> > > > create any
> > > > > >>>>>>>> directories or files necessary for the SR. There should
> be no
> > > > > >>>>>>> dependencies
> > > > > >>>>>>>> from the Storage to the Hypervisor layer. As I said
> earlier,
> > > such
> > > > > >>>>>>> circular
> > > > > >>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> Just want to make sure I'm on the same page with you.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Thanks again, John
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > > > <jb...@basho.com>
> > > > > >>>>> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>>> Mike,
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> > > > > supports n
> > > > > >>>>>>>>> devices types with each specific behaviors of m
> hypervisors.
> > > > > Such
> > > > > >>> a
> > > > > >>>>>>>>> scenario will create an unmaintainable and untestable
> > beast.
> > > > > >>>>>>> Therefore, my
> > > > > >>>>>>>>> thoughts and recommendations are driven to evolve the
> > > > Storage
> > > > > layer
> > > > > >>>>>>> towards
> > > > > >>>>>>>>> this separation.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> I am fine with the VolumeManager determining whether
> > or
> > > > not a
> > > > > >>> Volume
> > > > > >>>>> is
> > > > > >>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> > actual
> > > > > >>>>> isManaged
> > > > > >>>>>>>>> method), and asking the device driver to allocate
> resources
> > > for
> > > > > the
> > > > > >>>>>>> volume
> > > > > >>>>>>>>> if it is managed. Furthermore, the device driver needs
> to
> > > > > indicate
> > > > > >>>>>>> whether
> > > > > >>>>>>>>> or not it supports management operations. Finally, I
> think
> > > we
> > > > > >>> need to
> > > > > >>>>>>>>> provide the ability for an administrator to elect to have
> > > > > something
> > > > > >>>>>>> that is
> > > > > >>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> > managing
> > > > the
> > > > > >>>>> device,
> > > > > >>>>>>>>> but the administrator has elected to leave it unmanaged).
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Creation of a structure on the volume should be done in
> > the
> > > > Xen
> > > > > >>>>>>>>> hypervisor module using methods exposed by the Storage
> > > > layer to
> > > > > >>>>> perform
> > > > > >>>>>>>>> low-level operations (e.g. make directories, create a
> file,
> > > etc).
> > > > > >>>>> This
> > > > > >>>>>>>>> structure is specific to the operation of the Xen
> > > hypervisor, as
> > > > > >>> such,
> > > > > >>>>>>>>> should be confined to its implementation. From my
> > > > perspective,
> > > > > >>>>> nothing
> > > > > >>>>>>> in
> > > > > >>>>>>>>> the Storage layer should be concerned with content. From
> > its
> > > > > >>>>>>> perspective,
> > > > > >>>>>>>>> structure and data are opaque. It provides the means to
> > > query
> > > > > the
> > > > > >>>>> data
> > > > > >>>>>>> to
> > > > > >>>>>>>>> support the interpretation of the content by higher-level
> > > > layers
> > > > > >>> (e.g.
> > > > > >>>>>>>>> Hypervisors). To my mind, attach should be a composition
> > of
> > > > > >>>>> operations
> > > > > >>>>>>>>> from the Storage layer that varies based on the Volume
> > > > storage
> > > > > >>>>> protocol
> > > > > >>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Thanks,
> > > > > >>>>>>>>> -John
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > > > > >>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Hi John,
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Alternatively to the way the attach logic is implemented
> in
> > > my
> > > > > >>> patch,
> > > > > >>>>> we
> > > > > >>>>>>>>> could do the following:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Leave the attach logic in the agent code alone. In
> > > > > >>> VolumeManagerImpl
> > > > > >>>>> we
> > > > > >>>>>>>>> create an AttachVolumeCommand and send it to the
> > > > hypervisor.
> > > > > Before
> > > > > >>>>> this
> > > > > >>>>>>>>> command is sent, we could check to see if we're dealing
> > with
> > > > > >>> Dynamic
> > > > > >>>>> (or
> > > > > >>>>>>>>> whatever we want to call it) storage and - if so - send a
> > > "Create
> > > > > >>> SR"
> > > > > >>>>>>>>> command to the hypervisor. If this returns OK, we would
> > then
> > > > > >>> proceed
> > > > > >>>>> to
> > > > > >>>>>>> the
> > > > > >>>>>>>>> AttachVolumeCommand, as usual.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> This way the attach logic remains the same and we just
> add
> > > > > another
> > > > > >>>>>>>>> command to the agent code that is called for this
> particular
> > > > type
> > > > > >>> of
> > > > > >>>>>>>>> storage.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> What do you think?
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > > > > >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>> Hey John,
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> I created a document for a customer today that outlines
> > how
> > > > the
> > > > > >>>>> plug-in
> > > > > >>>>>>>>>> works from a user standpoint. This will probably be of
> use
> > > to
> > > > > >>> you, as
> > > > > >>>>>>> well,
> > > > > >>>>>>>>>> as you perform the code review.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> I have shared this document with you (you should have
> > > > received
> > > > > >>> that
> > > > > >>>>>>>>>> information in a separate e-mail).
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Talk to you later!
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > > > > >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>> Oh, OK, that sounds really good, John.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > > > > jburwell@basho.com
> > > > > >>>>>>>> wrote:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
> > but
> > > > > want
> > > > > >>> to
> > > > > >>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > > > > >>>>>>> disk_io_throttle,
> > > > > >>>>>>>>>>>> and
> > > > > >>>>>>>>>>>> object_store branches. I hope to collect them in a
> > > > coherent
> > > > > >>> form
> > > > > >>>>>>>>>>>> tomorrow (5 June 2013).
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > > > > >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
> > > notion
> > > > > of
> > > > > >>> a
> > > > > >>>>>>>>>>>> managed
> > > > > >>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces
> the
> > > > notion
> > > > > >>> of
> > > > > >>>>>>>>>>>>> provisioned storage IOPS.
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> The storage framework that leverages the plug-in was
> > > > > >>> incomplete,
> > > > > >>>>> so
> > > > > >>>>>>>>>>>> I had
> > > > > >>>>>>>>>>>>> to try to add in the notion of a managed iSCSI
> device.
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> I appreciate all the time you've been spending on
> this.
> > > :)
> > > > Do
> > > > > >>> you
> > > > > >>>>>>>>>>>> have a
> > > > > >>>>>>>>>>>>> recommendation as to how we should accomplish
> > what
> > > > you're
> > > > > >>> looking
> > > > > >>>>>>>>>>>> for?
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> Thanks!
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > > > > >>> jburwell@basho.com>
> > > > > >>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
> > > notion
> > > > > of
> > > > > >>> a
> > > > > >>>>>>>>>>>> managed
> > > > > >>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> I want to see a separation of the management
> > > > capabilities
> > > > > >>> (i.e.
> > > > > >>>>>>> can
> > > > > >>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>> device be managed/does an operator want it
> > managed
> > > > by
> > > > > >>> CloudStack)
> > > > > >>>>>>>>>>>> from the
> > > > > >>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
> > > > semantic
> > > > > >>> that
> > > > > >>>>>>>>>>>> will
> > > > > >>>>>>>>>>>>>> allow any type of storage device to be managed. I
> > also
> > > > want
> > > > > >>> to
> > > > > >>>>>>> make
> > > > > >>>>>>>>>>>>>> progress on decoupling the storage types from the
> > > > hypervisor
> > > > > >>>>>>>>>>>> definitions.
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > > > > >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> Hi John,
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> No problem. Answers are below in red.
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> Thanks!
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > > > > >>>>> jburwell@basho.com
> > > > > >>>>>>>>
> > > > > >>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Could you please answer the following questions
> > for
> > > > me
> > > > > with
> > > > > >>>>>>>>>>>> regards to
> > > > > >>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
> > SAN
> > > > > volumes?
> > > > > >>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN
> > 0).
> > > > > >>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and
> > > > CloudStack
> > > > > >>>>>>>>>>>> Volumes?
> > > > > >>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> > server or
> > > > > >>> externally
> > > > > >>>>> by
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>> operator?
> > > > > >>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> > volume
> > > > (same
> > > > > as a
> > > > > >>>>> SAN
> > > > > >>>>>>>>>>>> LUN) is
> > > > > >>>>>>>>>>>>>>> created by the management server (via the plug-in)
> > > > the
> > > > > first
> > > > > >>>>> time
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
> > > still
> > > > > >>> want
> > > > > >>>>> to
> > > > > >>>>>>>>>>>> use a
> > > > > >>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> > today
> > > > > (prior
> > > > > >>> to
> > > > > >>>>>>>>>>>> 4.2). The
> > > > > >>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
> > this
> > > > case -
> > > > > >>>>>>>>>>>> multiple VMs
> > > > > >>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>> data disks can share this SAN volume. While you
> > can do
> > > > this
> > > > > >>>>>>> today,
> > > > > >>>>>>>>>>>> it is
> > > > > >>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Are the SAN volumes by the management server
> > or
> > > > externally
> > > > > >>> by
> > > > > >>>>>>> the
> > > > > >>>>>>>>>>>>>> operator?
> > > > > >>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> > volumes
> > > > are
> > > > > >>>>>>> completely
> > > > > >>>>>>>>>>>>>> managed
> > > > > >>>>>>>>>>>>>>> by the management server (via the plug-in). There
> > is
> > > > no
> > > > > admin
> > > > > >>>>>>>>>>>>>> interaction.
> > > > > >>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> > volume
> > > > and a
> > > > > >>>>>>> CloudStack
> > > > > >>>>>>>>>>>>>> volume,
> > > > > >>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > > > supports
> > > > > true
> > > > > >>>>> QoS.
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> related
> > > > and
> > > > > >>>>>>> expected
> > > > > >>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>> operate.
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > > > > >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
> > VMs
> > > > with
> > > > > >>>>>>>>>>>> provisioned
> > > > > >>>>>>>>>>>>>> IOPS
> > > > > >>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> > single
> > > > SAN
> > > > > >>> volume
> > > > > >>>>>>>>>>>> (LUN) to a
> > > > > >>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> > multiple
> > > > > >>> CloudStack
> > > > > >>>>>>>>>>>> volumes
> > > > > >>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
> > QoS.
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
> > house
> > > > more
> > > > > >>> than
> > > > > >>>>>>> one
> > > > > >>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> > without
> > > > any of
> > > > > my
> > > > > >>>>>>>>>>>> plug-in
> > > > > >>>>>>>>>>>>>> code.
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> > > > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
> > for
> > > > > >>>>> CloudStack's
> > > > > >>>>>>>>>>>> use
> > > > > >>>>>>>>>>>>>> onto
> > > > > >>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> > > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me
> > --
> > > > > concepts
> > > > > >>>>> from
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> > > > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
> > > only
> > > > > used
> > > > > >>>>> (as
> > > > > >>>>>>>>>>>> far as
> > > > > >>>>>>>>>>>>>> I
> > > > > >>>>>>>>>>>>>>>>>>> know)
> > > > > >>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not
> > > > PreSetup an
> > > > > >>> SR).
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> > > > VMware, it
> > > > > >>> uses
> > > > > >>>>>>> the
> > > > > >>>>>>>>>>>> vmfs
> > > > > >>>>>>>>>>>>>>>> type.
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> > KVM,
> > > > it uses
> > > > > >>> the
> > > > > >>>>>>>>>>>>>>>>>>> SharedMountPoint
> > > > > >>>>>>>>>>>>>>>>>>>> type.
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> > here
> > > > was to
> > > > > >>> make a
> > > > > >>>>>>>>>>>> new type
> > > > > >>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> > Edison
> > > > added
> > > > > >>> into
> > > > > >>>>>>> the
> > > > > >>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> > specificy,
> > > > though:
> > > > > >>>>>>>>>>>> Dynamic_iSCSI
> > > > > >>>>>>>>>>>>>>>>>>> versus,
> > > > > >>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> > Tutkowski
> > > > <
> > > > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
> > or
> > > > care
> > > > > that
> > > > > >>> it
> > > > > >>>>>>> is
> > > > > >>>>>>>>>>>> being
> > > > > >>>>>>>>>>>>>>>> used
> > > > > >>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > > > questions
> > > > > >>> about
> > > > > >>>>> it
> > > > > >>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>> storing."
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > > > SolidFire
> > > > > >>> plug-in
> > > > > >>>>>>>>>>>> needs to
> > > > > >>>>>>>>>>>>>>>>>>> call
> > > > > >>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> > hypervisor
> > > > logic
> > > > > >>>>> knows
> > > > > >>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>> treat
> > > > > >>>>>>>>>>>>>>>>>>> it as
> > > > > >>>>>>>>>>>>>>>>>>>>> such.
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint
> > > > and just
> > > > > >>>>>>>>>>>> calling it
> > > > > >>>>>>>>>>>>>>>> iSCSI
> > > > > >>>>>>>>>>>>>>>>>>> or
> > > > > >>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
> > the
> > > > hypervisor
> > > > > >>>>>>> attach
> > > > > >>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
> > the
> > > > necessary
> > > > > >>>>>>>>>>>> activities.
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> > Burwell <
> > > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to
> > > > manage a
> > > > > >>>>>>> device,
> > > > > >>>>>>>>>>>> an
> > > > > >>>>>>>>>>>>>>>>>>> operator
> > > > > >>>>>>>>>>>>>>>>>>>>>> may
> > > > > >>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> > CloudStack
> > > > (e.g. the
> > > > > >>>>>>> device
> > > > > >>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>> shared
> > > > > >>>>>>>>>>>>>>>>>>> by
> > > > > >>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> > chosen
> > > > to
> > > > > >>> dedicate
> > > > > >>>>>>>>>>>> only a
> > > > > >>>>>>>>>>>>>>>> portion
> > > > > >>>>>>>>>>>>>>>>>>>>>> of it
> > > > > >>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> > make
> > > > sense?"
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
> > > > volume via
> > > > > >>> the
> > > > > >>>>>>>>>>>> plug-in
> > > > > >>>>>>>>>>>>>>>>>>> (before
> > > > > >>>>>>>>>>>>>>>>>>>>>> an
> > > > > >>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
> > > > work unless
> > > > > >>> the
> > > > > >>>>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>>>>> consumes
> > > > > >>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
> > SR.
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > > > CloudStack, an
> > > > > >>>>>>> operator
> > > > > >>>>>>>>>>>>>>>> allocates
> > > > > >>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> > exposes it
> > > > > through
> > > > > >>> a
> > > > > >>>>>>>>>>>> LUN. They
> > > > > >>>>>>>>>>>>>>>>>>> simply
> > > > > >>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
> > > > CloudStack,
> > > > > >>> but
> > > > > >>>>>>>>>>>> not allow
> > > > > >>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > > > changes, we
> > > > > >>> would
> > > > > >>>>>>> be
> > > > > >>>>>>>>>>>>>> passing
> > > > > >>>>>>>>>>>>>>>>>>> in a
> > > > > >>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > > > necessary
> > > > > >>> hypervisor
> > > > > >>>>>>>>>>>> support
> > > > > >>>>>>>>>>>>>>>> (like
> > > > > >>>>>>>>>>>>>>>>>>> an
> > > > > >>>>>>>>>>>>>>>>>>>>>> SR)
> > > > > >>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have
> > the
> > > > storage
> > > > > >>>>>>> framework
> > > > > >>>>>>>>>>>>>> itself
> > > > > >>>>>>>>>>>>>>>>>>>>>> detect
> > > > > >>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
> > from
> > > > the
> > > > > >>> hypervisor
> > > > > >>>>>>>>>>>> side and
> > > > > >>>>>>>>>>>>>>>>>>> have
> > > > > >>>>>>>>>>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
> > create
> > > > the
> > > > > >>> SR
> > > > > >>>>>>>>>>>> before the
> > > > > >>>>>>>>>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin
> > > > should
> > > > > have a
> > > > > >>>>>>> rich
> > > > > >>>>>>>>>>>> enough
> > > > > >>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> > available
> > > > for
> > > > > volume
> > > > > >>>>>>>>>>>> storage.
> > > > > >>>>>>>>>>>>>> For
> > > > > >>>>>>>>>>>>>>>>>>> Xen,
> > > > > >>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> > interrogation of
> > > > the
> > > > > >>>>> device
> > > > > >>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>> determine the
> > > > > >>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
> > > > shouldn't
> > > > > >>> know
> > > > > >>>>>>>>>>>> or care
> > > > > >>>>>>>>>>>>>>>>>>> that it
> > > > > >>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able
> > to
> > > > > answer
> > > > > >>>>>>>>>>>> questions
> > > > > >>>>>>>>>>>>>>>>>>> about it
> > > > > >>>>>>>>>>>>>>>>>>>>>> is storing.
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > > > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> > driver
> > > > (which
> > > > > >>> is a
> > > > > >>>>>>>>>>>> type of
> > > > > >>>>>>>>>>>>>>>>>>>>>> so-called
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
> > new
> > > > Primary
> > > > > >>>>>>>>>>>> Storage shows
> > > > > >>>>>>>>>>>>>>>> up
> > > > > >>>>>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> > that
> > > > > references
> > > > > >>> the
> > > > > >>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>> tag
> > > > > >>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
> > > > volume. This
> > > > > >>> leads
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>> a new
> > > > > >>>>>>>>>>>>>>>> row
> > > > > >>>>>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack
> > > > volume to a
> > > > > >>> VM
> > > > > >>>>>>>>>>>> (attach
> > > > > >>>>>>>>>>>>>>>> disk).
> > > > > >>>>>>>>>>>>>>>>>>>>>> This
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
> > the
> > > > plug-in
> > > > > >>> to
> > > > > >>>>>>>>>>>> create a
> > > > > >>>>>>>>>>>>>> new
> > > > > >>>>>>>>>>>>>>>>>>>>>> volume
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> > SAN).
> > > > The
> > > > > >>> plug-in
> > > > > >>>>>>>>>>>> also
> > > > > >>>>>>>>>>>>>>>> updates
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data
> > > > (like the
> > > > > >>> IQN of
> > > > > >>>>>>>>>>>> the SAN
> > > > > >>>>>>>>>>>>>>>>>>>>>> volume).
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> > > > > CloudStack
> > > > > >>>>>>>>>>>> volume is in
> > > > > >>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
> > > > volume
> > > > > >>> will be
> > > > > >>>>>>>>>>>> in the
> > > > > >>>>>>>>>>>>>>>>>>> 'Ready'
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the
> > > > plug-in
> > > > > >>> code
> > > > > >>>>>>>>>>>> will not
> > > > > >>>>>>>>>>>>>> be
> > > > > >>>>>>>>>>>>>>>>>>>>>> called
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and
> > > > detects
> > > > > the
> > > > > >>>>>>>>>>>> CloudStack
> > > > > >>>>>>>>>>>>>>>>>>> volume
> > > > > >>>>>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
> > of a
> > > > > >>> hypervisor
> > > > > >>>>>>> data
> > > > > >>>>>>>>>>>>>>>> structure
> > > > > >>>>>>>>>>>>>>>>>>>>>> (ex.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
> > > > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > > > terminology,
> > > > > >>> volume,
> > > > > >>>>>>>>>>>> in these
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > > > allocation on
> > > > > >>> the
> > > > > >>>>>>>>>>>> device,
> > > > > >>>>>>>>>>>>>>>>>>>>>> correct?"
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> > mean
> > > > 'SAN
> > > > > >>> volume'.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
> > can
> > > > make in
> > > > > >>>>>>>>>>>> CloudStack, I
> > > > > >>>>>>>>>>>>>>>>>>> try to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike
> > > > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
> > make
> > > > sense, but
> > > > > >>> I'm
> > > > > >>>>>>>>>>>> having a
> > > > > >>>>>>>>>>>>>>>> hard
> > > > > >>>>>>>>>>>>>>>>>>>>>> time
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on
> > > > this
> > > > > >>> conversation
> > > > > >>>>>>>>>>>> as he
> > > > > >>>>>>>>>>>>>> was
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> > approach I
> > > > took.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> > John
> > > > Burwell <
> > > > > >>>>>>>>>>>>>>>> jburwell@basho.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
> > > > distinct
> > > > > >>> concepts
> > > > > >>>>>>> --
> > > > > >>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>> device
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols.
> > In
> > > > both
> > > > > >>> cases, we
> > > > > >>>>>>>>>>>> are
> > > > > >>>>>>>>>>>>>>>>>>>>>> communicating with
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
> > > > > create/delete
> > > > > >>>>>>>>>>>> volumes
> > > > > >>>>>>>>>>>>>>>>>>> (Dynamic)
> > > > > >>>>>>>>>>>>>>>>>>>>>> on the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the
> > > > volume to
> > > > > be
> > > > > >>>>>>>>>>>> volume to be
> > > > > >>>>>>>>>>>>>>>>>>>>>> managed
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
> > To
> > > > ensure
> > > > > >>> that
> > > > > >>>>> we
> > > > > >>>>>>>>>>>> are in
> > > > > >>>>>>>>>>>>>>>>>>> sync on
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > > > definitions,
> > > > > >>> refers to
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>> physical
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
> > > > Minimally,
> > > > > we
> > > > > >>>>>>> must
> > > > > >>>>>>>>>>>> be
> > > > > >>>>>>>>>>>>>> able
> > > > > >>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device
> > to
> > > > move bits
> > > > > >>> from
> > > > > >>>>>>>>>>>> one place
> > > > > >>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>> another,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> > Optionally, a
> > > > > >>> storage
> > > > > >>>>>>>>>>>> device
> > > > > >>>>>>>>>>>>>> may
> > > > > >>>>>>>>>>>>>>>> be
> > > > > >>>>>>>>>>>>>>>>>>>>>> able to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore,
> > > > we can have
> > > > > a
> > > > > >>>>>>>>>>>> unmanaged
> > > > > >>>>>>>>>>>>>>>> iSCSI
> > > > > >>>>>>>>>>>>>>>>>>>>>> device
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
> > we
> > > > can have a
> > > > > >>>>>>> managed
> > > > > >>>>>>>>>>>>>>>> SolidFire
> > > > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> > capable
> > > > of
> > > > > >>> allocating
> > > > > >>>>>>>>>>>> LUNs and
> > > > > >>>>>>>>>>>>>>>>>>> storing
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
> > may
> > > > be able
> > > > > >>> to
> > > > > >>>>>>>>>>>> manage a
> > > > > >>>>>>>>>>>>>>>>>>> device,
> > > > > >>>>>>>>>>>>>>>>>>>>>> an
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > > > unmanaged by
> > > > > >>>>>>> CloudStack
> > > > > >>>>>>>>>>>> (e.g.
> > > > > >>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>> device is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
> > > > operator has
> > > > > >>> chosen
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>>>>>> dedicate
> > > > > >>>>>>>>>>>>>>>>>>>>>> only a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
> > my
> > > > > reasoning
> > > > > >>>>> make
> > > > > >>>>>>>>>>>> sense?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
> > > > reasonable, it
> > > > > >>> seems
> > > > > >>>>>>>>>>>>>> appropriate
> > > > > >>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
> > from
> > > > > >>> StoragePoolType,
> > > > > >>>>>>>>>>>> add the
> > > > > >>>>>>>>>>>>>>>>>>> notion
> > > > > >>>>>>>>>>>>>>>>>>>>>> of a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
> > driver
> > > > that
> > > > > >>>>> indicates
> > > > > >>>>>>>>>>>> whether
> > > > > >>>>>>>>>>>>>>>> or
> > > > > >>>>>>>>>>>>>>>>>>>>>> not is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> > establish
> > > > a
> > > > > >>> abstraction
> > > > > >>>>>>>>>>>>>>>> representing a
> > > > > >>>>>>>>>>>>>>>>>>>>>> physical
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that
> > is
> > > > > >>> associated
> > > > > >>>>>>>>>>>> with a
> > > > > >>>>>>>>>>>>>>>> volume.
> > > > > >>>>>>>>>>>>>>>>>>>>>> With
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
> > > > drivers can
> > > > > >>>>>>> declare
> > > > > >>>>>>>>>>>> which
> > > > > >>>>>>>>>>>>>>>>>>>>>> protocols they
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
> > a
> > > > device
> > > > > managed
> > > > > >>> by
> > > > > >>>>>>>>>>>>>> CloudStack,
> > > > > >>>>>>>>>>>>>>>>>>>>>> utilize the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
> > by
> > > > the driver to
> > > > > >>>>>>> automate
> > > > > >>>>>>>>>>>>>>>>>>> allocation.
> > > > > >>>>>>>>>>>>>>>>>>>>>> If
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> > sense,
> > > > then we can
> > > > > >>> sit
> > > > > >>>>>>>>>>>> down and
> > > > > >>>>>>>>>>>>>>>>>>> drill
> > > > > >>>>>>>>>>>>>>>>>>>>>> down to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
> > > > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
> > the
> > > > current
> > > > > iSCSI
> > > > > >>>>>>> type
> > > > > >>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>> Dynamic
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
> > and
> > > > create
> > > > > a
> > > > > >>>>>>>>>>>> Primary
> > > > > >>>>>>>>>>>>>>>> Storage
> > > > > >>>>>>>>>>>>>>>>>>>>>> based
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time,
> > > > the
> > > > > iSCSI
> > > > > >>>>>>>>>>>> volume must
> > > > > >>>>>>>>>>>>>>>>>>> exist
> > > > > >>>>>>>>>>>>>>>>>>>>>> on
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated).
> > > > Future
> > > > > >>>>>>>>>>>> CloudStack
> > > > > >>>>>>>>>>>>>>>> volumes
> > > > > >>>>>>>>>>>>>>>>>>> are
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created
> > > > behind the
> > > > > >>>>>>> scenes.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
> > go in
> > > > and
> > > > > create
> > > > > >>>>>>>>>>>> Primary
> > > > > >>>>>>>>>>>>>>>> Storage
> > > > > >>>>>>>>>>>>>>>>>>>>>> based
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete
> > > > volumes on
> > > > > >>> its
> > > > > >>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>> system
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
> > > > storage
> > > > > >>>>>>>>>>>> framework). When
> > > > > >>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>> user
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
> > was
> > > > created,
> > > > > the
> > > > > >>>>>>>>>>>> framework
> > > > > >>>>>>>>>>>>>>>> tells
> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
> > After
> > > > this is
> > > > > >>> done,
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called.
> > > > No
> > > > > >>> hypervisor
> > > > > >>>>>>>>>>>> data
> > > > > >>>>>>>>>>>>>>>>>>> structure
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
> > was
> > > > just
> > > > > >>> created.
> > > > > >>>>>>> The
> > > > > >>>>>>>>>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>>>>> data
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
> > Mike
> > > > > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should
> > > > probably
> > > > > have
> > > > > >>>>>>>>>>>> defined them
> > > > > >>>>>>>>>>>>>>>> up
> > > > > >>>>>>>>>>>>>>>>>>>>>> front
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> > pre-
> > > > allocated
> > > > > >>> (ex.
> > > > > >>>>>>>>>>>> an admin
> > > > > >>>>>>>>>>>>>>>>>>>>>> creates a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> > hypervisor
> > > > data
> > > > > >>> structure
> > > > > >>>>>>> is
> > > > > >>>>>>>>>>>>>> created
> > > > > >>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
> > then
> > > > that
> > > > > >>>>>>> hypervisor
> > > > > >>>>>>>>>>>> data
> > > > > >>>>>>>>>>>>>>>>>>>>>> structure
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
> > (VDI)
> > > > are later
> > > > > >>>>>>> placed
> > > > > >>>>>>>>>>>> on
> > > > > >>>>>>>>>>>>>> this
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> > these
> > > > cases, the
> > > > > >>>>>>> attach
> > > > > >>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>>>>>>> assumes
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> > already
> > > > in place
> > > > > >>> and
> > > > > >>>>>>>>>>>> simply
> > > > > >>>>>>>>>>>>>>>>>>> attaches
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure
> > to
> > > > the VM in
> > > > > >>>>>>>>>>>> question.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
> > not
> > > > > >>>>> pre-allocated.
> > > > > >>>>>>>>>>>> Instead
> > > > > >>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
> > be a
> > > > SAN
> > > > > (not
> > > > > >>> a
> > > > > >>>>>>>>>>>> volume on
> > > > > >>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>> SAN,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
> > > > structure
> > > > > >>> must be
> > > > > >>>>>>>>>>>> created
> > > > > >>>>>>>>>>>>>>>>>>> when an
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
> > these
> > > > types of
> > > > > >>> volumes
> > > > > >>>>>>>>>>>> have not
> > > > > >>>>>>>>>>>>>>>>>>> been
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > > > structure by an
> > > > > >>>>> admin.
> > > > > >>>>>>>>>>>> Once
> > > > > >>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer
> > for
> > > > this
> > > > > >>> volume,
> > > > > >>>>>>> it
> > > > > >>>>>>>>>>>>>> attaches
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM
> > in
> > > > question.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
> > > > John Burwell
> > > > > <
> > > > > >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of
> > the
> > > > Dynamic
> > > > > type
> > > > > >>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>> behavior
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
> > ask
> > > > about the
> > > > > >>>>>>>>>>>> difference.
> > > > > >>>>>>>>>>>>>>>>>>> Another
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
> > > > definition
> > > > > of
> > > > > >>> a
> > > > > >>>>>>>>>>>> Dynamic
> > > > > >>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> > Mike
> > > > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type
> > is
> > > > > uniquely
> > > > > >>>>> used
> > > > > >>>>>>>>>>>> by
> > > > > >>>>>>>>>>>>>>>>>>> XenServer
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> > Storage
> > > > that is
> > > > > >>> directly
> > > > > >>>>>>>>>>>> based on
> > > > > >>>>>>>>>>>>>> an
> > > > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> > step of
> > > > going
> > > > > to
> > > > > >>> the
> > > > > >>>>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
> > that
> > > > iSCSI
> > > > > >>> target as
> > > > > >>>>>>>>>>>>>> CloudStack
> > > > > >>>>>>>>>>>>>>>>>>> does
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > > > supported for
> > > > > >>>>>>>>>>>> XenServer.
> > > > > >>>>>>>>>>>>>> For
> > > > > >>>>>>>>>>>>>>>>>>> all
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go
> > to
> > > > the
> > > > > >>> hypervisor
> > > > > >>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>> perform
> > > > > >>>>>>>>>>>>>>>>>>>>>> this
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
> > PM,
> > > > John
> > > > > Burwell
> > > > > >>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
> > what
> > > > is the
> > > > > >>>>> difference
> > > > > >>>>>>>>>>>> between
> > > > > >>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
> > RBD
> > > > considered
> > > > > >>>>>>> Dynamic?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
> > Mike
> > > > > Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
> > > > defined in
> > > > > the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > > > StoragePoolType {
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
> > > > directory
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
> > //
> > > > NFS or CIFS
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
> > LUN,
> > > > with a
> > > > > >>>>> clusterfs
> > > > > >>>>>>>>>>>>>> overlay
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
> > > > Comstar
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
> > local
> > > > LVM SR
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
> > VMFS
> > > > storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > > > XenServer, Storage
> > > > > >>> Pool
> > > > > >>>>>>>>>>>> is set
> > > > > >>>>>>>>>>>>>>>> up
> > > > > >>>>>>>>>>>>>>>>>>>>>> by
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
> > local
> > > > EXT SR
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
> > > > zone-wide
> > > > > >>> storage
> > > > > >>>>>>> (ex.
> > > > > >>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
> > > > shared) {
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41
> > PM,
> > > > Mike
> > > > > >>> Tutkowski
> > > > > >>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> > another
> > > > storage
> > > > > >>>>> company
> > > > > >>>>>>>>>>>> wants
> > > > > >>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> > Quality
> > > > of
> > > > > Service
> > > > > >>>>>>>>>>>> feature. It
> > > > > >>>>>>>>>>>>>>>>>>> would
> > > > > >>>>>>>>>>>>>>>>>>>>>> be
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well.
> > > > They would
> > > > > >>> need
> > > > > >>>>>>> only
> > > > > >>>>>>>>>>>>>>>>>>> implement a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
> > > > necessary
> > > > > >>> changes to
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 1:39
> > > > PM, Mike
> > > > > >>>>>>> Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > mike.tutkowski@solidfire.com> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
> > > > XenServer code
> > > > > is
> > > > > >>>>>>>>>>>> SolidFire
> > > > > >>>>>>>>>>>>>>>>>>> specific.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
> > logic is
> > > > now aware
> > > > > >>> of
> > > > > >>>>>>>>>>>> dynamic,
> > > > > >>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > > > implementation of
> > > > > >>> this
> > > > > >>>>>>>>>>>> kind of
> > > > > >>>>>>>>>>>>>>>>>>>>>> storage).
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> > with
> > > > Edison's
> > > > > >>> storage
> > > > > >>>>>>>>>>>> framework
> > > > > >>>>>>>>>>>>>>>>>>>>>> changes.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
> > > > framework that
> > > > > >>>>> supported
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>> creation
> > > > > >>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
> > > > However, when I
> > > > > >>>>>>>>>>>> visited with
> > > > > >>>>>>>>>>>>>>>> him
> > > > > >>>>>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
> > it
> > > > was not
> > > > > >>>>>>>>>>>> complete. We
> > > > > >>>>>>>>>>>>>>>>>>> realized
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could
> > do
> > > > with these
> > > > > >>>>>>> volumes
> > > > > >>>>>>>>>>>> unless
> > > > > >>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
> > this
> > > > new type of
> > > > > >>>>> storage
> > > > > >>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>> create
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 1:28
> > > > PM, John
> > > > > >>> Burwell
> > > > > >>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me
> > > > that any
> > > > > >>>>> operation
> > > > > >>>>>>>>>>>> in the
> > > > > >>>>>>>>>>>>>>>>>>> Storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> > about
> > > > details. I
> > > > > >>> expect
> > > > > >>>>>>>>>>>> to see
> > > > > >>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> > operations
> > > > that can be
> > > > > >>>>>>>>>>>>>> composed/driven
> > > > > >>>>>>>>>>>>>>>>>>> by
> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > > > allocate
> > > > > space/create
> > > > > >>>>>>>>>>>> structures
> > > > > >>>>>>>>>>>>>>>> per
> > > > > >>>>>>>>>>>>>>>>>>>>>> their
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > > > dependency, we are
> > > > > >>> going
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>> end
> > > > > >>>>>>>>>>>>>>>> with a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make
> > the
> > > > system
> > > > > >>>>>>> increasingly
> > > > > >>>>>>>>>>>>>>>>>>> difficult to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > > > understanding that the
> > > > > >>> Xen
> > > > > >>>>>>>>>>>> specific
> > > > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > > > CitrixResourceBase
> > > > > class?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 3:12
> > > > PM, Mike
> > > > > >>>>>>>>>>>> Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
> > bit
> > > > more
> > > > > >>> detail:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
> > from
> > > > one setup
> > > > > >>>>>>> method
> > > > > >>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>> XenServer,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> > volume on
> > > > the
> > > > > storage
> > > > > >>>>>>>>>>>> system,
> > > > > >>>>>>>>>>>>>> then
> > > > > >>>>>>>>>>>>>>>> go
> > > > > >>>>>>>>>>>>>>>>>>>>>> into
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> > structure
> > > > to make
> > > > > use
> > > > > >>> of
> > > > > >>>>>>> the
> > > > > >>>>>>>>>>>>>> volume
> > > > > >>>>>>>>>>>>>>>>>>> (ex.
> > > > > >>>>>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> > XenServer
> > > > or a
> > > > > datastore
> > > > > >>> on
> > > > > >>>>>>>>>>>> ESX(i)).
> > > > > >>>>>>>>>>>>>>>> VMs
> > > > > >>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
> > > > system's volume.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> > storage
> > > > framework,
> > > > > >>>>>>> storage
> > > > > >>>>>>>>>>>> need
> > > > > >>>>>>>>>>>>>> no
> > > > > >>>>>>>>>>>>>>>>>>>>>> longer
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> > create
> > > > a 1:1
> > > > > >>>>>>> relationship
> > > > > >>>>>>>>>>>>>> between
> > > > > >>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
> > data
> > > > disk
> > > > > >>> (necessary
> > > > > >>>>>>> for
> > > > > >>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>> Quality
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
> > plug-
> > > > in that is
> > > > > >>> called
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>> dynamically
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
> > > > storage
> > > > > framework
> > > > > >>> did
> > > > > >>>>>>>>>>>> not
> > > > > >>>>>>>>>>>>>>>> address
> > > > > >>>>>>>>>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> > hypervisor-
> > > > specific data
> > > > > >>>>>>>>>>>> structure
> > > > > >>>>>>>>>>>>>> when
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
> > I've
> > > > been
> > > > > >>> enhancing
> > > > > >>>>> it
> > > > > >>>>>>>>>>>> to do
> > > > > >>>>>>>>>>>>>> so.
> > > > > >>>>>>>>>>>>>>>>>>> I've
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > > > submitted. I've got
> > > > > >>> ESX(i)
> > > > > >>>>>>> in
> > > > > >>>>>>>>>>>> my
> > > > > >>>>>>>>>>>>>>>> sandbox
> > > > > >>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
> > > > freeze date.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > > > 1:03 PM, Mike
> > > > > >>>>>>>>>>>> Tutkowski <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in -
> > by
> > > > itself - is
> > > > > >>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>> agnostic.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
> > > > volume-attach
> > > > > >>> logic
> > > > > >>>>>>>>>>>> (in the
> > > > > >>>>>>>>>>>>>>>>>>> agent
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
> > the
> > > > plug-in to
> > > > > >>> have
> > > > > >>>>>>> it
> > > > > >>>>>>>>>>>>>> create a
> > > > > >>>>>>>>>>>>>>>>>>>>>> volume
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> > comes to
> > > > attach the
> > > > > >>> volume
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>> hypervisor,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> > enough
> > > > to recognize
> > > > > >>> it's
> > > > > >>>>>>>>>>>> being
> > > > > >>>>>>>>>>>>>>>>>>> invoked on
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
> > has
> > > > just been
> > > > > >>> created)
> > > > > >>>>>>> and
> > > > > >>>>>>>>>>>>>> create,
> > > > > >>>>>>>>>>>>>>>>>>>>>> say, a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > > > XenServer) or a
> > > > > >>> datastore
> > > > > >>>>>>>>>>>> (for
> > > > > >>>>>>>>>>>>>>>>>>> VMware) to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just
> > > > created.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
> > most
> > > > of my time
> > > > > >>>>>>> recently
> > > > > >>>>>>>>>>>> making
> > > > > >>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at
> > > > 12:48 PM,
> > > > > John
> > > > > >>>>>>>>>>>> Burwell <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> > why
> > > > the the storage
> > > > > >>>>>>> driver
> > > > > >>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> > 1:21
> > > > PM, Mike
> > > > > >>>>>>> Tutkowski
> > > > > >>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
> > > > would like to
> > > > > >>> support
> > > > > >>>>>>>>>>>> all
> > > > > >>>>>>>>>>>>>>>>>>> hypervisors
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
> > I'm
> > > > just out of
> > > > > >>> time
> > > > > >>>>>>>>>>>> for 4.2
> > > > > >>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>> get
> > > > > >>>>>>>>>>>>>>>>>>>>>> KVM
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> > plug-in
> > > > supports
> > > > > >>>>>>> XenServer.
> > > > > >>>>>>>>>>>>>>>>>>> Depending on
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> > feature
> > > > freeze, I
> > > > > >>> have
> > > > > >>>>> it
> > > > > >>>>>>>>>>>> working
> > > > > >>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
> > clear,
> > > > this is
> > > > > all
> > > > > >>> in
> > > > > >>>>>>>>>>>> regards
> > > > > >>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>> Disk
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute
> > > > Offerings post
> > > > > 4.2.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > > > at 11:14 AM,
> > > > > >>> Kelcey
> > > > > >>>>>>>>>>>> Jamison
> > > > > >>>>>>>>>>>>>>>>>>> Damage
> > > > > >>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan
> > on
> > > > supporting
> > > > > >>> KVM in
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>> patch
> > > > > >>>>>>>>>>>>>>>>>>> cycle
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > > > Message -----
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > > > Tutkowski" <
> > > > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > > > dev@cloudstack.apache.org
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
> > June
> > > > 3, 2013
> > > > > >>> 10:12:32 AM
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> > [MERGE]
> > > > > >>>>> disk_io_throttling
> > > > > >>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>> MASTER
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> > merging
> > > > Wei's feature
> > > > > >>>>> first,
> > > > > >>>>>>>>>>>> then
> > > > > >>>>>>>>>>>>>>>> mine.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
> > for
> > > > KVM only,
> > > > > >>> then
> > > > > >>>>> it
> > > > > >>>>>>>>>>>> is a
> > > > > >>>>>>>>>>>>>> non
> > > > > >>>>>>>>>>>>>>>>>>> issue
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> > 2013
> > > > at 8:55 AM,
> > > > > Wei
> > > > > >>>>>>> ZHOU
> > > > > >>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
> > as
> > > > no one works
> > > > > >>> on
> > > > > >>>>>>>>>>>> billing
> > > > > >>>>>>>>>>>>>> now,
> > > > > >>>>>>>>>>>>>>>>>>>>>> users
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
> > > > themselves. They
> > > > > >>> can
> > > > > >>>>>>> get
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of
> > a
> > > > VMs and
> > > > > volumes
> > > > > >>>>> for
> > > > > >>>>>>>>>>>>>>>>>>> calculation.
> > > > > >>>>>>>>>>>>>>>>>>>>>> Of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
> > > > exact
> > > > > limitation
> > > > > >>>>>>> value
> > > > > >>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>> individual
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
> > for
> > > > nics as
> > > > > well.
> > > > > >>> I
> > > > > >>>>>>> can
> > > > > >>>>>>>>>>>> work
> > > > > >>>>>>>>>>>>>> on
> > > > > >>>>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
> > > > throttling?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > > > misunstand the second
> > > > > the
> > > > > >>>>>>>>>>>> question.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> > what
> > > > you said about
> > > > > the
> > > > > >>>>> two
> > > > > >>>>>>>>>>>>>> features.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> > > > Burwell <
> > > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> > at
> > > > 2:13 AM, Wei
> > > > > >>> ZHOU
> > > > > >>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's
> > > > aswer helps you.
> > > > > >>> I am
> > > > > >>>>>>>>>>>> trying
> > > > > >>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>> adding
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
> > billing
> > > > should
> > > > > >>> depend
> > > > > >>>>> on
> > > > > >>>>>>>>>>>> IO
> > > > > >>>>>>>>>>>>>>>>>>> statistics
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
> > Please
> > > > review
> > > > > >>>>>>>>>>>> disk_io_stat if
> > > > > >>>>>>>>>>>>>> you
> > > > > >>>>>>>>>>>>>>>>>>> have
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
> > > > statistics
> > > > > including
> > > > > >>>>>>>>>>>> bytes/iops
> > > > > >>>>>>>>>>>>>>>>>>>>>> read/write
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> > machine.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
> > AWS
> > > > model,
> > > > > customers
> > > > > >>>>> are
> > > > > >>>>>>>>>>>> billed
> > > > > >>>>>>>>>>>>>>>> more
> > > > > >>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> > IOPS,
> > > > as well as,
> > > > > for
> > > > > >>>>>>> those
> > > > > >>>>>>>>>>>>>>>>>>> operations (
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > http://aws.amazon.com/ebs/).
> > > > > I
> > > > > >>>>>>> would
> > > > > >>>>>>>>>>>>>> imagine
> > > > > >>>>>>>>>>>>>>>>>>> our
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> > employ
> > > > similar cost
> > > > > >>>>> models.
> > > > > >>>>>>>>>>>> Could
> > > > > >>>>>>>>>>>>>> an
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
> > in
> > > > the current
> > > > > >>> patch?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
> > mean
> > > > IOPS runtime
> > > > > >>>>> change?
> > > > > >>>>>>>>>>>> KVM
> > > > > >>>>>>>>>>>>>>>>>>> supports
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for
> > a
> > > > running
> > > > > virtual
> > > > > >>>>>>>>>>>> machine
> > > > > >>>>>>>>>>>>>>>> through
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > does
> > > > not support
> > > > > >>>>> changing
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>> parameters
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
> > > > offering or disk
> > > > > >>>>>>> offering).
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the
> > > > Java interface
> > > > > >>> level.
> > > > > >>>>>>> I
> > > > > >>>>>>>>>>>>>>>> apologize
> > > > > >>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > > > generalize allocation
> > > > > >>>>>>>>>>>> algorithms
> > > > > >>>>>>>>>>>>>> with a
> > > > > >>>>>>>>>>>>>>>>>>> set
> > > > > >>>>>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
> > > > service guarantees
> > > > > >>>>>>>>>>>> provided by a
> > > > > >>>>>>>>>>>>>>>>>>>>>> resource?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> > good
> > > > question.
> > > > > >>> Maybe it
> > > > > >>>>>>> is
> > > > > >>>>>>>>>>>>>> better
> > > > > >>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > disk_io_throttling as Mike
> > > > > >>> needs to
> > > > > >>>>>>>>>>>> consider
> > > > > >>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
> > on
> > > > my thoughts
> > > > > in a
> > > > > >>>>>>> later
> > > > > >>>>>>>>>>>>>>>> response
> > > > > >>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
> > > > between these two
> > > > > >>>>>>>>>>>> features. I
> > > > > >>>>>>>>>>>>>>>> think
> > > > > >>>>>>>>>>>>>>>>>>>>>> that
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_io_throttling
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be
> > > > merged before
> > > > > >>>>>>>>>>>> SolidFire, but
> > > > > >>>>>>>>>>>>>> I
> > > > > >>>>>>>>>>>>>>>>>>> think
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
> > > > between the
> > > > > branches
> > > > > >>>>>>>>>>>> (possibly
> > > > > >>>>>>>>>>>>>>>> have
> > > > > >>>>>>>>>>>>>>>>>>>>>> have
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_io_throttling)
> > > > to
> > > > > >>> coordinate on
> > > > > >>>>>>>>>>>> this
> > > > > >>>>>>>>>>>>>>>> issue.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> > John
> > > > Burwell <
> > > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I
> > > > want to
> > > > > understand
> > > > > >>>>> are
> > > > > >>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>> following:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
> > > > value in
> > > > > capturing
> > > > > >>>>> IOPS
> > > > > >>>>>>>>>>>>>> policies
> > > > > >>>>>>>>>>>>>>>> be
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model
> > (e.g.
> > > > for
> > > > > >>> billing/usage
> > > > > >>>>>>>>>>>>>> purposes,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
> > there
> > > > be a common
> > > > > >>>>>>>>>>>> interface model
> > > > > >>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning
> > at
> > > > runtime?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
> > > > conflicting
> > > > > >>> provisioned
> > > > > >>>>>>>>>>>> IOPS
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > and
> > > > storage device
> > > > > >>>>>>>>>>>> reconciled?
> > > > > >>>>>>>>>>>>>> In
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
> > > > believe (and
> > > > > billed)
> > > > > >>>>> for
> > > > > >>>>>>>>>>>> more
> > > > > >>>>>>>>>>>>>> IOPS
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > device
> > > > has been
> > > > > >>> configured
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>>>>>> deliver.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
> > > > configuration
> > > > > >>> between a
> > > > > >>>>>>>>>>>> VM and a
> > > > > >>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
> > > > later
> > > > > modification
> > > > > >>> to
> > > > > >>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>> device
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > inconsistency.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
> > 2013,
> > > > at 8:38 PM,
> > > > > >>> Mike
> > > > > >>>>>>>>>>>> Tutkowski
> > > > > >>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > mike.tutkowski@solidfire.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
> > Wei's
> > > > feature deals
> > > > > >>> with
> > > > > >>>>>>>>>>>>>>>> controlling
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > hypervisor
> > > > side.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature
> > is
> > > > focused on
> > > > > >>>>>>> controlling
> > > > > >>>>>>>>>>>> IOPS
> > > > > >>>>>>>>>>>>>>>> from
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
> > > > helps. :)
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun
> > 2,
> > > > 2013 at 6:35
> > > > > PM,
> > > > > >>>>>>> John
> > > > > >>>>>>>>>>>>>> Burwell
> > > > > >>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> > opinion is
> > > > that no
> > > > > features
> > > > > >>>>>>>>>>>> should be
> > > > > >>>>>>>>>>>>>>>>>>> merged
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> > have
> > > > been resolved
> > > > > and
> > > > > >>> it
> > > > > >>>>>>> is
> > > > > >>>>>>>>>>>> ready
> > > > > >>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>> turn
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
> > vs
> > > > discrete
> > > > > >>> read/write
> > > > > >>>>>>>>>>>> ops issue
> > > > > >>>>>>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
> > don't
> > > > think this
> > > > > >>> criteria
> > > > > >>>>>>>>>>>> has been
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how
> > > > does this work
> > > > > >>>>>>>>>>>>>>>> intersect/compliment
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>> https://reviews.apache.org/r/11479/
> > > > > >>>>>>>>>>>> )?
> > > > > >>>>>>>>>>>>>> As I
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> > involves
> > > > provisioned
> > > > > >>> IOPS. I
> > > > > >>>>>>>>>>>> would
> > > > > >>>>>>>>>>>>>>>> like
> > > > > >>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > > where provisioned
> > > > > >>> IOPS
> > > > > >>>>> in
> > > > > >>>>>>>>>>>> KVM and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > incompatible.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
> > 2013,
> > > > at 6:47 AM,
> > > > > >>> Wei
> > > > > >>>>>>>>>>>> ZHOU <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will
> > > > change it next
> > > > > >>> week.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
> > > > Wido den Hollander
> > > > > <
> > > > > >>>>>>>>>>>>>> wido@widodh.nl
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > > 06/01/2013 08:24 AM, Wei
> > > > > >>> ZHOU
> > > > > >>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
> > have
> > > > pushed the
> > > > > >>>>>>> features
> > > > > >>>>>>>>>>>> into
> > > > > >>>>>>>>>>>>>>>>>>> master.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
> > > > object thems for
> > > > > >>>>>>> technical
> > > > > >>>>>>>>>>>>>> reason
> > > > > >>>>>>>>>>>>>>>>>>> till
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> > sake
> > > > of clarity I
> > > > > just
> > > > > >>>>>>> want
> > > > > >>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>> mention
> > > > > >>>>>>>>>>>>>>>>>>>>>> again
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
> > IOps
> > > > to R/W IOps
> > > > > >>> asap
> > > > > >>>>> so
> > > > > >>>>>>>>>>>> that we
> > > > > >>>>>>>>>>>>>>>>>>> never
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
> > IOps.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
> > the
> > > > groundwork for
> > > > > >>> the
> > > > > >>>>>>> I/O
> > > > > >>>>>>>>>>>>>>>> throttling
> > > > > >>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
> > > > prevent that we
> > > > > create
> > > > > >>>>>>>>>>>> legacy from
> > > > > >>>>>>>>>>>>>>>> day
> > > > > >>>>>>>>>>>>>>>>>>>>>> #1.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> > > > Wido den
> > > > > Hollander <
> > > > > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > > 05/31/2013 03:59 PM, John
> > > > > >>>>>>> Burwell
> > > > > >>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
> > > > enhancement must
> > > > > to
> > > > > >>>>>>>>>>>> discretely
> > > > > >>>>>>>>>>>>>>>>>>> support
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > how
> > > > it could be
> > > > > >>> fixed
> > > > > >>>>>>>>>>>> later
> > > > > >>>>>>>>>>>>>>>> because
> > > > > >>>>>>>>>>>>>>>>>>> I
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
> > IOPS
> > > > into read
> > > > > and
> > > > > >>>>>>>>>>>> write.
> > > > > >>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > > unless/until we
> > > > > decided
> > > > > >>> to
> > > > > >>>>>>>>>>>> break
> > > > > >>>>>>>>>>>>>>>>>>> backwards
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei
> > > > meant was merging
> > > > > it
> > > > > >>>>> into
> > > > > >>>>>>>>>>>> master
> > > > > >>>>>>>>>>>>>>>> now
> > > > > >>>>>>>>>>>>>>>>>>> so
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch
> > > > and add Read /
> > > > > >>> Write
> > > > > >>>>>>> IOps
> > > > > >>>>>>>>>>>>>> before
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>> 4.2
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > > > released with Read
> > > > > and
> > > > > >>>>>>> Write
> > > > > >>>>>>>>>>>>>> instead
> > > > > >>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
> > > > make the May 31st
> > > > > >>>>>>> feature
> > > > > >>>>>>>>>>>>>> freeze
> > > > > >>>>>>>>>>>>>>>>>>> date.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other
> > > > threads) then it
> > > > > >>> won't
> > > > > >>>>>>> be
> > > > > >>>>>>>>>>>>>>>> necessary
> > > > > >>>>>>>>>>>>>>>>>>>>>> to do
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > > > completely agree that
> > > > > >>>>> there
> > > > > >>>>>>>>>>>> is no
> > > > > >>>>>>>>>>>>>>>>>>>>>> association
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31,
> > > > 2013, at 9:51 AM,
> > > > > >>> Wido
> > > > > >>>>>>>>>>>> den
> > > > > >>>>>>>>>>>>>>>>>>> Hollander <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > > 05/31/2013 03:13 PM, Wei
> > > > > >>> ZHOU
> > > > > >>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> > Good
> > > > question.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
> > > > about at the
> > > > > >>>>> beginning.
> > > > > >>>>>>>>>>>>>> Finally I
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> > of
> > > > read and write
> > > > > >>>>>>> mainly
> > > > > >>>>>>>>>>>>>> because
> > > > > >>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
> > > > difference of sent
> > > > > >>> and
> > > > > >>>>>>>>>>>> received
> > > > > >>>>>>>>>>>>>>>>>>> bytes as
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > > > reasoning seems odd.
> > > > > >>>>>>>>>>>> Networking and
> > > > > >>>>>>>>>>>>>>>> disk
> > > > > >>>>>>>>>>>>>>>>>>> I/O
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is
> > > > much more
> > > > > >>> expensive
> > > > > >>>>> in
> > > > > >>>>>>>>>>>> most
> > > > > >>>>>>>>>>>>>>>>>>>>>> situations
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > Implementing
> > > > it will be some
> > > > > >>>>>>>>>>>> copy-paste
> > > > > >>>>>>>>>>>>>>>> work.
> > > > > >>>>>>>>>>>>>>>>>>> It
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > implemented
> > > > in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
> > For
> > > > the deadline
> > > > > of
> > > > > >>>>>>>>>>>> feature
> > > > > >>>>>>>>>>>>>>>> freeze,
> > > > > >>>>>>>>>>>>>>>>>>> I
> > > > > >>>>>>>>>>>>>>>>>>>>>> will
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
> > > > needed.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's
> > a
> > > > feature we
> > > > > >>> can't
> > > > > >>>>>>>>>>>> miss. But
> > > > > >>>>>>>>>>>>>>>> if
> > > > > >>>>>>>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > we
> > > > have to make sure
> > > > > we
> > > > > >>>>>>> don't
> > > > > >>>>>>>>>>>>>> release
> > > > > >>>>>>>>>>>>>>>>>>> with
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
> > that
> > > > will confuse
> > > > > >>>>>>> users.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> > > > Wido den
> > > > > Hollander <
> > > > > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > > 05/30/2013 06:03 PM, Wei
> > > > > >>> ZHOU
> > > > > >>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
> > like to
> > > > merge
> > > > > >>>>>>>>>>>> disk_io_throttling
> > > > > >>>>>>>>>>>>>>>>>>> branch
> > > > > >>>>>>>>>>>>>>>>>>>>>> into
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
> > > > object, I will
> > > > > merge
> > > > > >>>>>>> into
> > > > > >>>>>>>>>>>> master
> > > > > >>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>> 48
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > purpose
> > > > is :
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > > > machines are running
> > > > > >>> on
> > > > > >>>>>>> the
> > > > > >>>>>>>>>>>> same
> > > > > >>>>>>>>>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> > strage).
> > > > Because of
> > > > > the
> > > > > >>>>> rate
> > > > > >>>>>>>>>>>>>>>> limitation
> > > > > >>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
> > has
> > > > large disk
> > > > > >>> operation,
> > > > > >>>>>>> it
> > > > > >>>>>>>>>>>> may
> > > > > >>>>>>>>>>>>>>>> affect
> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs
> > > > running on the
> > > > > same
> > > > > >>>>>>>>>>>> storage
> > > > > >>>>>>>>>>>>>>>> device.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > neccesary
> > > > to set the
> > > > > >>>>> maximum
> > > > > >>>>>>>>>>>> rate
> > > > > >>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>> limit
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at
> > the
> > > > code I see
> > > > > you
> > > > > >>>>> make
> > > > > >>>>>>>>>>>> no
> > > > > >>>>>>>>>>>>>>>>>>> difference
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and
> > > > libvirt support
> > > > > >>> setting
> > > > > >>>>>>>>>>>> both a
> > > > > >>>>>>>>>>>>>>>>>>> different
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > which
> > > > could benefit a
> > > > > >>> lot of
> > > > > >>>>>>>>>>>> users.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
> > > > strange, in the
> > > > > >>> polling
> > > > > >>>>>>>>>>>> side you
> > > > > >>>>>>>>>>>>>>>>>>>>>> collect
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
> > on
> > > > the throttling
> > > > > >>> side
> > > > > >>>>>>>>>>>> you only
> > > > > >>>>>>>>>>>>>> go
> > > > > >>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps
> > are
> > > > usually much
> > > > > >>> more
> > > > > >>>>>>>>>>>> expensive
> > > > > >>>>>>>>>>>>>>>>>>> then
> > > > > >>>>>>>>>>>>>>>>>>>>>> Read
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid
> > > > use-case where
> > > > > >>> that
> > > > > >>>>>>> an
> > > > > >>>>>>>>>>>> admin
> > > > > >>>>>>>>>>>>>>>>>>> would
> > > > > >>>>>>>>>>>>>>>>>>>>>> set
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
> > Read
> > > > IOps.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
> > only
> > > > supports KVM
> > > > > >>> at
> > > > > >>>>>>> this
> > > > > >>>>>>>>>>>>>> point I
> > > > > >>>>>>>>>>>>>>>>>>>>>> think
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at
> > > > least have the
> > > > > >>>>>>>>>>>> mechanism in
> > > > > >>>>>>>>>>>>>>>> place
> > > > > >>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > implementing
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
> > > > would be a lot of
> > > > > >>>>> work.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> > hypervisor
> > > > doesn't
> > > > > >>> support
> > > > > >>>>>>>>>>>> setting
> > > > > >>>>>>>>>>>>>>>>>>> different
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
> > can
> > > > always sum
> > > > > both
> > > > > >>> up
> > > > > >>>>>>>>>>>> and set
> > > > > >>>>>>>>>>>>>>>> that
> > > > > >>>>>>>>>>>>>>>>>>> as
> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> > > > explain why you
> > > > > >>>>>>> implemented
> > > > > >>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>> this
> > > > > >>>>>>>>>>>>>>>>>>> way?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > feature
> > > > includes:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> > > > maximum rate of
> > > > > >>> VMs
> > > > > >>>>>>> (in
> > > > > >>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > configuration)
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change
> > the
> > > > maximum rate
> > > > > of
> > > > > >>>>> VMs
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
> > the
> > > > disk rate
> > > > > (total
> > > > > >>>>> bps
> > > > > >>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>> iops)
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> > > > > >>>>>>>>>>>>>> https://issues.apache.org/****
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > issues.apache.org/****
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > jira/browse/CLOUDSTACK-1192<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > > 1192>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <ht**tps://
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > 1192<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > > 1192<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
> > > > update later) :
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>
> > > > >
> > > >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>
> > > > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > > > >>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>
> > > > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > VM+Disk+IO+Throttling<https://
> > > > > >>>>>>> ****
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > cwiki.apache.org/confluence/****
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>
> > > >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>
> > > > > >>>
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > > rottling
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
> > check
> > > > list :-
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
> > > > check the branch's
> > > > > >>> RAT
> > > > > >>>>>>>>>>>> execution
> > > > > >>>>>>>>>>>>>>>>>>>>>> success?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> > there
> > > > new dependencies
> > > > > >>>>>>>>>>>> introduced?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> > > > automated testing
> > > > > (unit
> > > > > >>>>> and
> > > > > >>>>>>>>>>>>>>>>>>> integration)
> > > > > >>>>>>>>>>>>>>>>>>>>>> is
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
> > are
> > > > added.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> > > > testing has been done
> > > > > >>> to
> > > > > >>>>>>>>>>>> check for
> > > > > >>>>>>>>>>>>>>>>>>>>>> potential
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> > > > bytes rate and
> > > > > >>> IOPS
> > > > > >>>>>>>>>>>> rate on
> > > > > >>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
> > > > operations, including
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
> > stop,
> > > > start, reboot,
> > > > > >>>>>>>>>>>> destroy,
> > > > > >>>>>>>>>>>>>>>> expunge.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> > Volume
> > > > operations,
> > > > > >>> including
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > > > Detach
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review
> > the
> > > > code, you can
> > > > > >>> try
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > c30057635d04a2396f84c588127d7e******be42e503a7
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > f2e5591b710d04cc86815044f5823e******73a4a58944
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> > regards,
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>
> > > > >
> > > >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>
> > > > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > > > >>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>
> > > > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > VM+Disk+IO+Throttling<https://
> > > > > >>>>>>> ****
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > cwiki.apache.org/confluence/****
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>
> > > >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>
> > > > > >>>
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > > rottling
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > > > > >>> refs/heads/disk_io_throttling
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > > 1301
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <ht**tps://
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> > 1301<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > > 1301>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <ht**tps://
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> > 1301<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > > 1301<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <ht**tps://
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> > 2071
> > > > > >>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > > 2071
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > > 2071
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > > 2071
> > > > > >>>>>>>>>>>>>> <
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > > 2071>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > <h**ttps://
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > (**CLOUDSTACK-1301
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
> > Disk
> > > > I/O
> > > > > Throttling)
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > > Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > > CloudStack Developer,
> > > > > >>>>>>>>>>>> SolidFire
> > > > > >>>>>>>>>>>>>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > > >>> mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> > 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> > the
> > > > way the world
> > > > > >>> uses
> > > > > >>>>>>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > > Developer,
> > > > > >>>>> SolidFire
> > > > > >>>>>>>>>>>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > > the world uses
> > > > > >>> the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > http://solidfire.com/solution/overview/?video=play
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack
> > > > Developer,
> > > > > >>> SolidFire
> > > > > >>>>>>>>>>>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way
> > > > the world uses
> > > > > the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > > Developer,
> > > > > >>> SolidFire
> > > > > >>>>>>>>>>>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > > world uses the
> > > > > >>>>>>> cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > > Developer,
> > > > > SolidFire
> > > > > >>>>>>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the
> > > > world uses the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > > Developer, SolidFire
> > > > > >>>>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > > > world uses the
> > > > > >>> cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > > SolidFire
> > > > > >>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > > uses the
> > > > > >>> cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > > SolidFire
> > > > > >>> Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world
> > > > uses the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > > SolidFire
> > > > > Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > > the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > > SolidFire Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses
> > > > the cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > > SolidFire Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > > Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > > cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > > Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > > Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > cloud<
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > > > Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > > >>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>>>> http://solidfire.com/solution/overview/?video=play
> > > > > >>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > cloud<
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > > > >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > > >>>>>>>>>>>>>>>>>>>> cloud<
> > > > > >>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > > > >>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > > >>>>>>>>>>>>>>>>>
> > > > cloud<http://solidfire.com/solution/overview/?video=play
> > > > > >
> > > > > >>>>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > > >>>>>>>>>>>>>>>
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>>>> Advancing the way the world uses the
> > > > > >>>>>>>>>>>>>
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> --
> > > > > >>>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>>> Advancing the way the world uses the cloud<
> > > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> --
> > > > > >>>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>>> Advancing the way the world uses the cloud<
> > > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>>> *(tm)*
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> --
> > > > > >>>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>>> o: 303.746.7302
> > > > > >>>>>>>>> Advancing the way the world uses the cloud<
> > > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>>> *(tm)*
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> --
> > > > > >>>>>>>> *Mike Tutkowski*
> > > > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>>> o: 303.746.7302
> > > > > >>>>>>>> Advancing the way the world uses the cloud<
> > > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>>> *(tm)*
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> --
> > > > > >>>>>>> *Mike Tutkowski*
> > > > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>>>>> e: mike.tutkowski@solidfire.com
> > > > > >>>>>>> o: 303.746.7302
> > > > > >>>>>>> Advancing the way the world uses the
> > > > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > > > >>>>>>> *(tm)*
> > > > > >>>>>>>
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> --
> > > > > >>>> *Mike Tutkowski*
> > > > > >>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >>>> e: mike.tutkowski@solidfire.com
> > > > > >>>> o: 303.746.7302
> > > > > >>>> Advancing the way the world uses the
> > > > > >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > > > >>>> *(tm)*
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> *Mike Tutkowski*
> > > > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > > > >> e: mike.tutkowski@solidfire.com
> > > > > >> o: 303.746.7302
> > > > > >> Advancing the way the world uses the cloud<
> > > > > http://solidfire.com/solution/overview/?video=play>
> > > > > >> *(tm)*
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Mike Tutkowski*
> > > > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > > > e: mike.tutkowski@solidfire.com
> > > > > > o: 303.746.7302
> > > > > > Advancing the way the world uses the
> > > > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > > > *(tm)*
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkowski@solidfire.com
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > *(tm)*
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I don't understand this line:
"If the DataStore is not managed and the underlying storage is not managed,
an exception is thrown."
>From what I know, a DataStore is just an object that describes a row in the
storage_pool table. Is that incorrect?
I could see the underlying storage being usable in both a managed or
unmanaged mode, but the DataStore itself (a row in the storage_pool table)
would always be in only one of these states, right?
On Fri, Jun 7, 2013 at 5:20 PM, John Burwell <jb...@basho.com> wrote:
> Edison,
>
> Please see my comments in-line below.
>
> Thanks,
> -John
>
> On Jun 7, 2013, at 6:49 PM, Edison Su <Ed...@citrix.com> wrote:
>
> >
> >
> >> -----Original Message-----
> >> From: John Burwell [mailto:jburwell@basho.com]
> >> Sent: Friday, June 07, 2013 3:26 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>
> >>
> >> On Jun 7, 2013, at 6:13 PM, Edison Su <Ed...@citrix.com> wrote:
> >>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>> Sent: Friday, June 07, 2013 2:37 PM
> >>>> To: dev@cloudstack.apache.org
> >>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>
> >>>> As we only have three weeks until feature freeze, we should come to a
> >>>> consensus on this design point as soon as possible.
> >>>>
> >>>> Right now, if the storage framework asks my driver it is is managed,
> it
> >>>> will say 'yes.' This means the framework will tell the driver to
> perform
> >>>> its management activities. This then means the driver will call into
> the
> >>>> host (it doesn't know which hypervisor, by the way) to perform the
> >> activity
> >>>> of, say, creating an SR on XenServer or a datastore on ESX.
> >>>>
> >>>> The driver doesn't know which hypervisor it's talking to, it just
> sends a
> >>>> message to the host to perform the necessary pre-attach work.
> >>>
> >>> Could we just expose a method like "attachVolume/dettachVolume" on
> >> the PrimaryDataStoreDriver driver?
> >>> For most of cases, the implementation of each driver would just send
> >> attachvolumecommand/dettachvolumecmd to hypervisor(we can put the
> >> implementation in a base class, so that can be shared by all of this
> drivers),
> >> and in each hypervisor resource code, which may just call hypervisor's
> api to
> >> attach the volume to VM, while for certain storage, like SolidFire, may
> need
> >> to create a SR first, and create volume on it, then call hypervisor's
> API to
> >> attach volume to VM.
> >>> While for some other storage vendor, may want to bypass hypervisor
> >> during attaching volume, so inside driver's attachvolume implementation,
> >> the driver can do some magic, such as, directly talk to an agent inside
> VM
> >> instance, then create a disk inside VM.
> >>>
> >>> How do you guys think?
> >>
> >> This behavior is the opposite of my expectations. I would expect the
> >> VirtualMachineManager to coordinate this process. Why would Storage
> >> need to know or care about a Hypervisor? My understanding is that Xen
> >> needs to create an SR in a particular manner on all ISCSI devices upon
> >> allocation. On attach, the Xen plugin would query the datastore to
> >> determine the presence of an SR, and create it as necessary. It would
> >> proceed with the creating the volume, etc. In the "real world", SANs
> have no
> >> knowledge about the concept of a hypervisor let alone specific
> >> implementations. I think we would be wise to replicate that model in
> >> CloudStack.
> >
> > I think there are volumes managed/created by hypervisor(the normal
> case), and there are volumes managed by storage vendor(solidfire case). How
> to expose the volume to user/VM may be quite different in both cases.
> > If the volume is managed by storage vendor, then vendor wants the volume
> to be used by hypervisor, won't it be the storage vendor's responsibility
> to do whatever the hypervisor wants?
> > It's the same, if volume is managed by storage vendor, and vendor wants
> the volume to used by VM directly, won't it be the storage vendor's
> responsibility to do whatever magic they want?
> > If the responsibility is on the storage vendor, then why we can't let
> storage vendor to do whatever necessary steps to get the things done?
>
> It is not the responsibility of the storage vendor to understand the needs
> of the hypervisor. It is the responsibility of the storage vendor to
> provide interfaces to perform certain low level functions (e.g. create a
> volume, delete a volume, etc). Users of the DataStore compose these
> operations to achieve certain goals (e.g. attach a volume to a hypervisor,
> create structures, create volumes, etc).
>
> You are describing the managed vs. unmanaged device concept that the
> SolidFire patch introduces. For DataStoreDrivers that can be managed, the
> associated DataStores can have management enabled. When a device is
> manageable, then the Storage layer recognizes that it can do more things
> with it. For example, when executing attachVolume, the orchestration
> pieces can asked managed DataStores to ensure that the underlying storage
> has been allocated to support the volume. From a CloudStack perspective,
> we know when management functions need to be invoked -- we only need the
> driver to provide the discrete operations. For the volume attach scenario
> we are discussing, I am thinking the basic would be as follows -- starting
> in a VirtualMachineManager:
>
> 1. VirtualMachineManager initiates the allocation process --
> creating records, networks, etc (hand wavy a bit) until we get to storage
> 2. For each Disk Offering invoke VolumeManager
> A. Allocate through VolumeManager
> i. VolumeManager acquires the DataStore, and if
> managed, allocates the underlying storage. If the DataStore is not managed
> and the underlying storage is not managed, an exception is thrown.
> B. The resulting volume is attached to the new VM through
> the hypervisor plugin (callback point for Xen to create an SR structure ->
> query the new volume to determine whether or not the SR is present)
> 3. The rest of the VM allocation process completes
>
> Based on this process, the DataStoreDriver provides the discrete
> operations necessary to be composed by the Storage and Hypervisor
> orchestration components to complete various goals.
>
> Does this process make sense?
>
> >
> >>
> >>>
> >>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com>
> wrote:
> >>>>
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>>>> To: dev@cloudstack.apache.org
> >>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>
> >>>>>> Hi John,
> >>>>>>
> >>>>>> How's about this:
> >>>>>>
> >>>>>> The driver can implement an isManaged() method. The
> >>>> VolumeManagerImpl
> >>>>>> can
> >>>>>> call into the driver to see if its managed. If it is, the
> >>>>> VolumeManagerImpl
> >>>>>> (which is responsible for calling into the hypervisor to attach the
> disk)
> >>>>>> can call into the hypervisor to create the necessary hypervisor data
> >>>>>> structure (ex. for XenServer, a storage repository).
> >>>>>
> >>>>> The problem here is that storage vendor may work differently with
> >>>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> >> other
> >>>>> vendor wants to totally bypass hypervisor, and assign the LUN
> directly to
> >>>>> VM instance, see the discuss(
> >>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>>>
> >> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >>>> %3E
> >>>>> ).
> >>>>> So I would let storage provider to implement attach disk to VM,
> instead
> >> of
> >>>>> implemented by cloudstack itself.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> If that's what you're going for, that works for me. By the way,
> Edison's
> >>>>>> default storage plug-in (which handles the default storage behavior
> in
> >>>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
> >>>>>> hypervisors. You might want to contact him and inform him of your
> >>>>> concerns
> >>>>>> or that logic (as is) will make it to production.
> >>>>>>
> >>>>>> Please let me know if what I wrote in above (for my solution) is OK
> with
> >>>>>> you. :)
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> Please see my responses in-line below.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hey John,
> >>>>>>>>
> >>>>>>>> I still have a bit more testing I'd like to do before I build up a
> >>>>> patch
> >>>>>>>> file, but this is the gist of what I've done:
> >>>>>>>>
> >>>>>>>> * During a volume-attach operation, after VolumeManagerImpl tells
> >>>>>>>> VolumeServiceImpl to have the driver create a volume, I have
> >>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> >>>>>> managed.
> >>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
> >> whatever
> >>>>>>>> activity is required. In my case, this includes sending a message
> to
> >>>>> the
> >>>>>>>> host where the VM is running to have, say XenServer, add a storage
> >>>>>>>> repository (based on the IP address of the SAN, the IQN of the SAN
> >>>>>>> volume,
> >>>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can
> on
> >>>>> the
> >>>>>>>> storage repository). After this, the normal attach-volume message
> is
> >>>>> sent
> >>>>>>>> to the host by VolumeManagerImpl.
> >>>>>>>
> >>>>>>> There should be **no** code from a storage driver to a hypervisor.
> I
> >>>>>>> apologize for the repetition, but we simply can not have hypervisor
> >>>>>>> specific code in the storage layer. The circular dependencies
> between
> >>>>> the
> >>>>>>> two layers are not sustainable in the long term. Either the
> >>>>> VirtualManager
> >>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >>>> coordinate
> >>>>>>> volume creation and then populating the SR. Ideally, we can
> >>>>> generalize the
> >>>>>>> process flow for attaching volumes such that the Xen hypervisor
> >> plugin
> >>>>>>> would only implement callbacks to perform the attach action and
> >> create
> >>>>>> the
> >>>>>>> structure and SR. To my mind, the SolidFire driver should only be
> >>>>>>> allocating space and providing information about contents (e.g.
> space
> >>>>>>> available, space consumed, streams to a URI, file handle for a URI,
> >>>>> etc)
> >>>>>>> and capabilities.
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> * The reverse is performed for a detach-volume command.
> >>>>>>>>
> >>>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
> >>>>>>> Edison's
> >>>>>>>> default driver simply does a "return false;". We could add a new
> >>>>>>> parameter
> >>>>>>>> to the createStoragePool API command, if we want, to remove the
> >>>>>>> hard-coded
> >>>>>>>> return values in the drivers (although my driver will probably
> just
> >>>>>>> ignore
> >>>>>>>> this parameter and always return true since it wouldn't make sense
> >>>>> for it
> >>>>>>>> to ever return false). We'd need another column in the
> storage_pool
> >>>>>> table
> >>>>>>>> to store this value.
> >>>>>>>
> >>>>>>> Yes, I think we should have a parameter added to the
> >>>> createStoragePool
> >>>>>>> surfaced to the HTTP API that allows DataStores to be configured
> for
> >>>>>>> management when their underlying drivers support it. To simplify
> >>>>> things,
> >>>>>>> this flag should only be mutable when the DataStore is created. It
> >>>>> would be
> >>>>>>> a bit crazy to take a DataStore from managed to unmanaged after
> >>>>> creation.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I agree, John. Just wanted to point out that I have a working GUI
> >>>>> for
> >>>>>>> you
> >>>>>>>>> to review (in that document), if you'd like to check it out.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jburwell@basho.com
> >
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> I would like the UIs of two features reviewed together to ensure
> >>>>>>>>>> consistency across the concepts of hypervisor throttled IOPs and
> >>>>>>>>>> storage device provisioned IOPs. I see the potential for
> >>>>> confusion,
> >>>>>>>>>> and I think a side-by-side Ui review of these features will help
> >>>>>>>>>> minimize any potential confusion.
> >>>>>>>>>>
> >>>>>>>>>> As I mentioned, the term reconciliation issue will work itself
> if
> >>>>> it
> >>>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> >>>>> throttled
> >>>>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi John,
> >>>>>>>>>>>
> >>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent to
> >>>> you
> >>>>>> the
> >>>>>>>>>>> other day to see how the GUI looks for provisioned storage
> IOPS.
> >>>>>>>>>>>
> >>>>>>>>>>> Several months ago, I put this topic out on the e-mail list
> and we
> >>>>>>>>>> decided
> >>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
> >>>>> dialog.
> >>>>>>>>>>> Other storage vendors are coming out with QoS, so they should
> >>>> be
> >>>>>> able
> >>>>>>> to
> >>>>>>>>>>> leverage this GUI going forward (even if they, say, only use
> Max
> >>>>>>> IOPS).
> >>>>>>>>>>> These fields are optional and can be ignored for storage that
> >>>>> does not
> >>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> >>>>> admin can
> >>>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst
> IOPS.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or
> mine,
> >>>>> as
> >>>>>>> is
> >>>>>>>>>>> decided by the admin).
> >>>>>>>>>>>
> >>>>>>>>>>> I'm not sure what we can do about these two features
> >> expressing
> >>>>> the
> >>>>>>>>>> speed
> >>>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
> >>>>> QoS in
> >>>>>>>>>> any
> >>>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
> >>>>> IOPS).
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >>>> <jb...@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Wei,
> >>>>>>>>>>>>
> >>>>>>>>>>>> We have been down the rabbit hole a bit on the
> >>>> Storage/Hypervisor
> >>>>>>> layer
> >>>>>>>>>>>> separation, but we still need to reconcile the behavior of
> >>>>> hypervisor
> >>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the
> following
> >>>>>>> issues
> >>>>>>>>>>>> outstanding:
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >>>> read/write
> >>>>>>>>>> values
> >>>>>>>>>>>> where as storage provisioned IOPS are expressed as total IOPS.
> >>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> >>>> storage
> >>>>>>> volumes
> >>>>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>>>> 3. How should usage data be captured for throttled and
> >>>>> provisioned
> >>>>>>> IOPS
> >>>>>>>>>>>> that will permit providers to discriminate these guaranteed
> >>>>>>> operations
> >>>>>>>>>> in
> >>>>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>>>> 4. What is the user experience for throttled and provisioned
> >>>> IOPS
> >>>>>>> that
> >>>>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>>>
> >>>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> >>>>> throttled
> >>>>>>>>>> IOPS
> >>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
> >>>>> items 1
> >>>>>>>>>> and 2.
> >>>>>>>>>>>> Since the two facilities would not be permitted to operate
> >>>>> together,
> >>>>>>>>>> they
> >>>>>>>>>>>> do not need to be semantically compatible. I think item 3 can
> >> be
> >>>>>>>>>> resolved
> >>>>>>>>>>>> with an additional flag or two on the usage records. As for
> >>>>> Item 4,
> >>>>>>> I
> >>>>>>>>>> am
> >>>>>>>>>>>> not familiar with how these two enhancements are depicted in
> >>>> the
> >>>>>> user
> >>>>>>>>>>>> interface. I think we need to review the UI enhancements for
> >>>>> both
> >>>>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> >> <us...@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> John and Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> >>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >>>> CLOUDSTACK-
> >>>>>> 2875 ) this
> >>>>>>>>>> week.
> >>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and ask
> >>>>> for
> >>>>>>>>>> second
> >>>>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I believe I understand where you're going with this, John,
> >> and
> >>>>>> have
> >>>>>>>>>> been
> >>>>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>>>> <jb...@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> "I am fine with the VolumeManager determining whether
> >> or
> >>>>>> not a
> >>>>>>>>>> Volume
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> >>>> actual
> >>>>>>>>>> isManaged
> >>>>>>>>>>>>>>> method), and asking the device driver to allocate resources
> >>>>> for
> >>>>>>> the
> >>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged() method
> >>>>>> added to
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then
> >> the
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>> framework could call the manage() (or whatever name)
> >>>> method
> >>>>>> (which
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
> >>>> this
> >>>>>>> would
> >>>>>>>>>>>> call
> >>>>>>>>>>>>>>> into a new method in the hypervisor code to create, say on
> >>>>>>>>>> XenServer,
> >>>>>>>>>>>> an
> >>>>>>>>>>>>>> SR?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >>>>>> DataStoreDriver.
> >>>>>>>>>> Since
> >>>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage, I
> >>>>> would
> >>>>>>> add
> >>>>>>>>>> a
> >>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
> >>>> man
> >>>>>> term
> >>>>>>>>>> --
> >>>>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> >>>> property
> >>>>>> of
> >>>>>>>>>>>>>> DataStore
> >>>>>>>>>>>>>>> which can be enabled if/when the underlying
> >>>> DataStoreDriver
> >>>>>> can be
> >>>>>>>>>>>>>> managed.
> >>>>>>>>>>>>>>> This approach allows operators to override manageability of
> >>>>>>> devices.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> >> plugin
> >>>>>>> would
> >>>>>>>>>> be
> >>>>>>>>>>>>>>> responsible for composing DataStore/Volume methods to
> >>>>>> create any
> >>>>>>>>>>>>>>> directories or files necessary for the SR. There should be
> >> no
> >>>>>>>>>>>>>> dependencies
> >>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said
> earlier,
> >>>>> such
> >>>>>>>>>>>>>> circular
> >>>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>>>> <jb...@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> >>>>>>> supports n
> >>>>>>>>>>>>>>>> devices types with each specific behaviors of m
> >> hypervisors.
> >>>>>>> Such
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> >>>> beast.
> >>>>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve the
> >>>>>> Storage
> >>>>>>> layer
> >>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I am fine with the VolumeManager determining whether
> >>>> or
> >>>>>> not a
> >>>>>>>>>> Volume
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> >>>> actual
> >>>>>>>>>>>> isManaged
> >>>>>>>>>>>>>>>> method), and asking the device driver to allocate
> >> resources
> >>>>> for
> >>>>>>> the
> >>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs to
> >>>>>>> indicate
> >>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>> or not it supports management operations. Finally, I
> think
> >>>>> we
> >>>>>>>>>> need to
> >>>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
> >>>>>>> something
> >>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> >>>> managing
> >>>>>> the
> >>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>> but the administrator has elected to leave it unmanaged).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Creation of a structure on the volume should be done in
> >>>> the
> >>>>>> Xen
> >>>>>>>>>>>>>>>> hypervisor module using methods exposed by the
> >> Storage
> >>>>>> layer to
> >>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a
> file,
> >>>>> etc).
> >>>>>>>>>>>> This
> >>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >>>>> hypervisor, as
> >>>>>>>>>> such,
> >>>>>>>>>>>>>>>> should be confined to its implementation. From my
> >>>>>> perspective,
> >>>>>>>>>>>> nothing
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> the Storage layer should be concerned with content.
> >> From
> >>>> its
> >>>>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>>>> structure and data are opaque. It provides the means to
> >>>>> query
> >>>>>>> the
> >>>>>>>>>>>> data
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> support the interpretation of the content by higher-level
> >>>>>> layers
> >>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a composition
> >>>> of
> >>>>>>>>>>>> operations
> >>>>>>>>>>>>>>>> from the Storage layer that varies based on the Volume
> >>>>>> storage
> >>>>>>>>>>>> protocol
> >>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Alternatively to the way the attach logic is implemented
> in
> >>>>> my
> >>>>>>>>>> patch,
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >>>>>>>>>> VolumeManagerImpl
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> >>>>>> hypervisor.
> >>>>>>> Before
> >>>>>>>>>>>> this
> >>>>>>>>>>>>>>>> command is sent, we could check to see if we're dealing
> >>>> with
> >>>>>>>>>> Dynamic
> >>>>>>>>>>>> (or
> >>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
> >>>>> "Create
> >>>>>>>>>> SR"
> >>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we would
> >>>> then
> >>>>>>>>>> proceed
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> This way the attach logic remains the same and we just
> >> add
> >>>>>>> another
> >>>>>>>>>>>>>>>> command to the agent code that is called for this
> >> particular
> >>>>>> type
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I created a document for a customer today that outlines
> >>>> how
> >>>>>> the
> >>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> >> use
> >>>>> to
> >>>>>>>>>> you, as
> >>>>>>>>>>>>>> well,
> >>>>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I have shared this document with you (you should have
> >>>>>> received
> >>>>>>>>>> that
> >>>>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
> >>>> but
> >>>>>>> want
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> >>>>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> >>>>>> coherent
> >>>>>>>>>> form
> >>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
> >>>>> notion
> >>>>>>> of
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the
> >>>>>> notion
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
> >> was
> >>>>>>>>>> incomplete,
> >>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> >> this.
> >>>>> :)
> >>>>>> Do
> >>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> >>>> what
> >>>>>> you're
> >>>>>>>>>> looking
> >>>>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
> >>>>> notion
> >>>>>>> of
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> >>>>>> capabilities
> >>>>>>>>>> (i.e.
> >>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> >>>> managed
> >>>>>> by
> >>>>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
> >>>>>> semantic
> >>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed. I
> >>>> also
> >>>>>> want
> >>>>>>>>>> to
> >>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from the
> >>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following questions
> >>>> for
> >>>>>> me
> >>>>>>> with
> >>>>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
> >>>> SAN
> >>>>>>> volumes?
> >>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
> >> (LUN
> >>>> 0).
> >>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
> >> and
> >>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> >>>> server or
> >>>>>>>>>> externally
> >>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> >>>> volume
> >>>>>> (same
> >>>>>>> as a
> >>>>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>>>> created by the management server (via the plug-
> >> in)
> >>>>>> the
> >>>>>>> first
> >>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
> >>>>> still
> >>>>>>>>>> want
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> >>>> today
> >>>>>>> (prior
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
> >>>> this
> >>>>>> case -
> >>>>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While you
> >>>> can do
> >>>>>> this
> >>>>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management server
> >>>> or
> >>>>>> externally
> >>>>>>>>>> by
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> >>>> volumes
> >>>>>> are
> >>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
> >> There
> >>>> is
> >>>>>> no
> >>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> >>>> volume
> >>>>>> and a
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> >>>>>> supports
> >>>>>>> true
> >>>>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> >> related
> >>>>>> and
> >>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
> >>>> VMs
> >>>>>> with
> >>>>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> >>>> single
> >>>>>> SAN
> >>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> >>>> multiple
> >>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
> >>>> QoS.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
> >>>> house
> >>>>>> more
> >>>>>>>>>> than
> >>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> >>>> without
> >>>>>> any of
> >>>>>>> my
> >>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
> >>>> for
> >>>>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which concerns
> >> me
> >>>> --
> >>>>>>> concepts
> >>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
> >>>>> only
> >>>>>>> used
> >>>>>>>>>>>> (as
> >>>>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
> >> not
> >>>>>> PreSetup an
> >>>>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> >>>>>> VMware, it
> >>>>>>>>>> uses
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> >>>> KVM,
> >>>>>> it uses
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> >>>> here
> >>>>>> was to
> >>>>>>>>>> make a
> >>>>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> >>>> Edison
> >>>>>> added
> >>>>>>>>>> into
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> >>>> specificy,
> >>>>>> though:
> >>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> >>>> Tutkowski
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
> >>>> or
> >>>>>> care
> >>>>>>> that
> >>>>>>>>>> it
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> >>>>>> questions
> >>>>>>>>>> about
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> >>>>>> SolidFire
> >>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> >>>> hypervisor
> >>>>>> logic
> >>>>>>>>>>>> knows
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> >> constraint
> >>>>>> and just
> >>>>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
> >>>> the
> >>>>>> hypervisor
> >>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
> >>>> the
> >>>>>> necessary
> >>>>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able
> >> to
> >>>>>> manage a
> >>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> >>>> CloudStack
> >>>>>> (e.g. the
> >>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> >>>> chosen
> >>>>>> to
> >>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> >>>> make
> >>>>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
> >>>>>> volume via
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
> >>>>>> work unless
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
> >>>> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> >>>>>> CloudStack, an
> >>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> >>>> exposes it
> >>>>>>> through
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
> >>>>>> CloudStack,
> >>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> >>>>>> changes, we
> >>>>>>>>>> would
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> >>>>>> necessary
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
> >> have
> >>>> the
> >>>>>> storage
> >>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
> >>>> from
> >>>>>> the
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
> >>>> create
> >>>>>> the
> >>>>>>>>>> SR
> >>>>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
> >> plugin
> >>>>>> should
> >>>>>>> have a
> >>>>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> >>>> available
> >>>>>> for
> >>>>>>> volume
> >>>>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >>>> interrogation of
> >>>>>> the
> >>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
> >>>>>> shouldn't
> >>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
> >> able
> >>>> to
> >>>>>>> answer
> >>>>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> >>>> driver
> >>>>>> (which
> >>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
> >>>> new
> >>>>>> Primary
> >>>>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> >>>> that
> >>>>>>> references
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
> >>>>>> volume. This
> >>>>>>>>>> leads
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> >> CloudStack
> >>>>>> volume to a
> >>>>>>>>>> VM
> >>>>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
> >>>> the
> >>>>>> plug-in
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> >>>> SAN).
> >>>>>> The
> >>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
> >> data
> >>>>>> (like the
> >>>>>>>>>> IQN of
> >>>>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> >>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
> >>>>>> volume
> >>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
> >> the
> >>>>>> plug-in
> >>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
> >> and
> >>>>>> detects
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
> >>>> of a
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> >>>>>> terminology,
> >>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> >>>>>> allocation on
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> >>>> mean
> >>>>>> 'SAN
> >>>>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
> >>>> can
> >>>>>> make in
> >>>>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
> >> Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
> >>>> make
> >>>>>> sense, but
> >>>>>>>>>> I'm
> >>>>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in
> >> on
> >>>>>> this
> >>>>>>>>>> conversation
> >>>>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> >>>> approach I
> >>>>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> >>>> John
> >>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
> >>>>>> distinct
> >>>>>>>>>> concepts
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> >> protocols.
> >>>> In
> >>>>>> both
> >>>>>>>>>> cases, we
> >>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system
> >> to
> >>>>>>> create/delete
> >>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
> >> the
> >>>>>> volume to
> >>>>>>> be
> >>>>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
> >>>> To
> >>>>>> ensure
> >>>>>>>>>> that
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> >>>>>> definitions,
> >>>>>>>>>> refers to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
> >>>>>> Minimally,
> >>>>>>> we
> >>>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
> >> device
> >>>> to
> >>>>>> move bits
> >>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >>>> Optionally, a
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> >> Therefore,
> >>>>>> we can have
> >>>>>>> a
> >>>>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
> >>>> we
> >>>>>> can have a
> >>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> >>>> capable
> >>>>>> of
> >>>>>>>>>> allocating
> >>>>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
> >>>> may
> >>>>>> be able
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> >>>>>> unmanaged by
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
> >>>>>> operator has
> >>>>>>>>>> chosen
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
> >>>> my
> >>>>>>> reasoning
> >>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
> >>>>>> reasonable, it
> >>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
> >>>> from
> >>>>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
> >>>> driver
> >>>>>> that
> >>>>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> >>>> establish
> >>>>>> a
> >>>>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
> >> that
> >>>> is
> >>>>>>>>>> associated
> >>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
> >>>>>> drivers can
> >>>>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
> >>>> a
> >>>>>> device
> >>>>>>> managed
> >>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
> >>>> by
> >>>>>> the driver to
> >>>>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> >>>> sense,
> >>>>>> then we can
> >>>>>>>>>> sit
> >>>>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
> >>>> the
> >>>>>> current
> >>>>>>> iSCSI
> >>>>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
> >>>> and
> >>>>>> create
> >>>>>>> a
> >>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
> >> time,
> >>>>>> the
> >>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> >> allocated).
> >>>>>> Future
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
> >> created
> >>>>>> behind the
> >>>>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
> >>>> go in
> >>>>>> and
> >>>>>>> create
> >>>>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
> >> delete
> >>>>>> volumes on
> >>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
> >>>>>> storage
> >>>>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
> >>>> was
> >>>>>> created,
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
> >>>> After
> >>>>>> this is
> >>>>>>>>>> done,
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
> >> called.
> >>>>>> No
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
> >>>> was
> >>>>>> just
> >>>>>>>>>> created.
> >>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
> >>>> Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
> >> should
> >>>>>> probably
> >>>>>>> have
> >>>>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> >>>> pre-
> >>>>>> allocated
> >>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> >>>> hypervisor
> >>>>>> data
> >>>>>>>>>> structure
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
> >>>> then
> >>>>>> that
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
> >>>> (VDI)
> >>>>>> are later
> >>>>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> >>>> these
> >>>>>> cases, the
> >>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> >>>> already
> >>>>>> in place
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> >> structure
> >>>> to
> >>>>>> the VM in
> >>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
> >>>> not
> >>>>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
> >>>> be a
> >>>>>> SAN
> >>>>>>> (not
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
> >>>>>> structure
> >>>>>>>>>> must be
> >>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
> >>>> these
> >>>>>> types of
> >>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> >>>>>> structure by an
> >>>>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> >> XenServer
> >>>> for
> >>>>>> this
> >>>>>>>>>> volume,
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
> >> VM
> >>>> in
> >>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
> >>>>>> John Burwell
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation
> >> of
> >>>> the
> >>>>>> Dynamic
> >>>>>>> type
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
> >>>> ask
> >>>>>> about the
> >>>>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
> >>>>>> definition
> >>>>>>> of
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> >>>> Mike
> >>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
> >> type
> >>>> is
> >>>>>>> uniquely
> >>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> >>>> Storage
> >>>>>> that is
> >>>>>>>>>> directly
> >>>>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> >>>> step of
> >>>>>> going
> >>>>>>> to
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
> >>>> that
> >>>>>> iSCSI
> >>>>>>>>>> target as
> >>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> >>>>>> supported for
> >>>>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first
> >> go
> >>>> to
> >>>>>> the
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD
> >> is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
> >>>> PM,
> >>>>>> John
> >>>>>>> Burwell
> >>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
> >>>> what
> >>>>>> is the
> >>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
> >>>> RBD
> >>>>>> considered
> >>>>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
> >>>> Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
> >>>>>> defined in
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> >>>>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
> >>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
> >>>> //
> >>>>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
> >>>> LUN,
> >>>>>> with a
> >>>>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
> >>>>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
> >>>> local
> >>>>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
> >>>> VMFS
> >>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> >>>>>> XenServer, Storage
> >>>>>>>>>> Pool
> >>>>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
> >>>> local
> >>>>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
> >>>>>> zone-wide
> >>>>>>>>>> storage
> >>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
> >>>>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 1:41
> >>>> PM,
> >>>>>> Mike
> >>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> >>>> another
> >>>>>> storage
> >>>>>>>>>>>> company
> >>>>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> >>>> Quality
> >>>>>> of
> >>>>>>> Service
> >>>>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
> >> well.
> >>>>>> They would
> >>>>>>>>>> need
> >>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
> >>>>>> necessary
> >>>>>>>>>> changes to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>>> 1:39
> >>>>>> PM, Mike
> >>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
> >>>>>> XenServer code
> >>>>>>> is
> >>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
> >>>> logic is
> >>>>>> now aware
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> >>>>>> implementation of
> >>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> >>>> with
> >>>>>> Edison's
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
> >>>>>> framework that
> >>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
> >>>>>> However, when I
> >>>>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
> >>>> it
> >>>>>> was not
> >>>>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
> >> could
> >>>> do
> >>>>>> with these
> >>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
> >>>> this
> >>>>>> new type of
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> >> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>>> 1:28
> >>>>>> PM, John
> >>>>>>>>>> Burwell
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to
> >> me
> >>>>>> that any
> >>>>>>>>>>>> operation
> >>>>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> >>>> about
> >>>>>> details. I
> >>>>>>>>>> expect
> >>>>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> >>>> operations
> >>>>>> that can be
> >>>>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> >>>>>> allocate
> >>>>>>> space/create
> >>>>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> >>>>>> dependency, we are
> >>>>>>>>>> going
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
> >> make
> >>>> the
> >>>>>> system
> >>>>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> >>>>>> understanding that the
> >>>>>>>>>> Xen
> >>>>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >>>>>> CitrixResourceBase
> >>>>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >>>> 3:12
> >>>>>> PM, Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
> >>>> bit
> >>>>>> more
> >>>>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
> >>>> from
> >>>>>> one setup
> >>>>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> >>>> volume on
> >>>>>> the
> >>>>>>> storage
> >>>>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> >>>> structure
> >>>>>> to make
> >>>>>>> use
> >>>>>>>>>> of
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> >>>> XenServer
> >>>>>> or a
> >>>>>>> datastore
> >>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
> >>>>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> >>>> storage
> >>>>>> framework,
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> >>>> create
> >>>>>> a 1:1
> >>>>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
> >>>> data
> >>>>>> disk
> >>>>>>>>>> (necessary
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
> >>>> plug-
> >>>>>> in that is
> >>>>>>>>>> called
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
> >>>>>> storage
> >>>>>>> framework
> >>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >>>> hypervisor-
> >>>>>> specific data
> >>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
> >>>> I've
> >>>>>> been
> >>>>>>>>>> enhancing
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> >>>>>> submitted. I've got
> >>>>>>>>>> ESX(i)
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
> >>>>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
> >> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> >> at
> >>>>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in
> >> -
> >>>> by
> >>>>>> itself - is
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
> >>>>>> volume-attach
> >>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
> >>>> the
> >>>>>> plug-in to
> >>>>>>>>>> have
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> >>>> comes to
> >>>>>> attach the
> >>>>>>>>>> volume
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> >>>> enough
> >>>>>> to recognize
> >>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
> >>>> has
> >>>>>> just been
> >>>>>>>>>> created)
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> >>>>>> XenServer) or a
> >>>>>>>>>> datastore
> >>>>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
> >> just
> >>>>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
> >>>> most
> >>>>>> of my time
> >>>>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> >>>> at
> >>>>>> 12:48 PM,
> >>>>>>> John
> >>>>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> >>>> why
> >>>>>> the the storage
> >>>>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> >>>> 1:21
> >>>>>> PM, Mike
> >>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
> >>>>>> would like to
> >>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
> >>>> I'm
> >>>>>> just out of
> >>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> >>>> plug-in
> >>>>>> supports
> >>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> >>>> feature
> >>>>>> freeze, I
> >>>>>>>>>> have
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
> >>>> clear,
> >>>>>> this is
> >>>>>>> all
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >> Compute
> >>>>>> Offerings post
> >>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>> 2013
> >>>>>> at 11:14 AM,
> >>>>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
> >> plan
> >>>> on
> >>>>>> supporting
> >>>>>>>>>> KVM in
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> >>>>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> >>>>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >>>>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
> >>>> June
> >>>>>> 3, 2013
> >>>>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> >>>> [MERGE]
> >>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> >>>> merging
> >>>>>> Wei's feature
> >>>>>>>>>>>> first,
> >>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
> >>>> for
> >>>>>> KVM only,
> >>>>>>>>>> then
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >>>> 2013
> >>>>>> at 8:55 AM,
> >>>>>>> Wei
> >>>>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
> >>>> as
> >>>>>> no one works
> >>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
> >>>>>> themselves. They
> >>>>>>>>>> can
> >>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering
> >> of
> >>>> a
> >>>>>> VMs and
> >>>>>>> volumes
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
> >>>>>> exact
> >>>>>>> limitation
> >>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
> >>>> for
> >>>>>> nics as
> >>>>>>> well.
> >>>>>>>>>> I
> >>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
> >>>>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> >>>>>> misunstand the second
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> >>>> what
> >>>>>> you said about
> >>>>>>> the
> >>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> >>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> >> 2013,
> >>>> at
> >>>>>> 2:13 AM, Wei
> >>>>>>>>>> ZHOU
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> >> Mike's
> >>>>>> aswer helps you.
> >>>>>>>>>> I am
> >>>>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
> >>>> billing
> >>>>>> should
> >>>>>>>>>> depend
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
> >>>> Please
> >>>>>> review
> >>>>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
> >>>>>> statistics
> >>>>>>> including
> >>>>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> >>>> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
> >>>> AWS
> >>>>>> model,
> >>>>>>> customers
> >>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >>>> IOPS,
> >>>>>> as well as,
> >>>>>>> for
> >>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://aws.amazon.com/ebs/).
> >>>>>>> I
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> >>>> employ
> >>>>>> similar cost
> >>>>>>>>>>>> models.
> >>>>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
> >>>> in
> >>>>>> the current
> >>>>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
> >>>> mean
> >>>>>> IOPS runtime
> >>>>>>>>>>>> change?
> >>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> >> for
> >>>> a
> >>>>>> running
> >>>>>>> virtual
> >>>>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>> does
> >>>>>> not support
> >>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
> >>>>>> offering or disk
> >>>>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
> >> the
> >>>>>> Java interface
> >>>>>>>>>> level.
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> >>>>>> generalize allocation
> >>>>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
> >>>>>> service guarantees
> >>>>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> >>>> good
> >>>>>> question.
> >>>>>>>>>> Maybe it
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> disk_io_throttling as Mike
> >>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
> >>>> on
> >>>>>> my thoughts
> >>>>>>> in a
> >>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
> >>>>>> between these two
> >>>>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to
> >> be
> >>>>>> merged before
> >>>>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
> >>>>>> between the
> >>>>>>> branches
> >>>>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> disk_io_throttling)
> >>>>>> to
> >>>>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> >>>> John
> >>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things
> >> I
> >>>>>> want to
> >>>>>>> understand
> >>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
> >>>>>> value in
> >>>>>>> capturing
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >> model
> >>>> (e.g.
> >>>>>> for
> >>>>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
> >>>> there
> >>>>>> be a common
> >>>>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> provisioning
> >>>> at
> >>>>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
> >>>>>> conflicting
> >>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>> and
> >>>>>> storage device
> >>>>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
> >>>>>> believe (and
> >>>>>>> billed)
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>> device
> >>>>>> has been
> >>>>>>>>>> configured
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
> >>>>>> configuration
> >>>>>>>>>> between a
> >>>>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
> >>>>>> later
> >>>>>>> modification
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
> >>>> 2013,
> >>>>>> at 8:38 PM,
> >>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
> >>>> Wei's
> >>>>>> feature deals
> >>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>> hypervisor
> >>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> >> feature
> >>>> is
> >>>>>> focused on
> >>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
> >>>>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
> >> Jun
> >>>> 2,
> >>>>>> 2013 at 6:35
> >>>>>>> PM,
> >>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> >>>> opinion is
> >>>>>> that no
> >>>>>>> features
> >>>>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> >>>> have
> >>>>>> been resolved
> >>>>>>> and
> >>>>>>>>>> it
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
> >>>> vs
> >>>>>> discrete
> >>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
> >>>> don't
> >>>>>> think this
> >>>>>>>>>> criteria
> >>>>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> >> how
> >>>>>> does this work
> >>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> >>>> involves
> >>>>>> provisioned
> >>>>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>> where provisioned
> >>>>>>>>>> IOPS
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
> >>>> 2013,
> >>>>>> at 6:47 AM,
> >>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
> >> will
> >>>>>> change it next
> >>>>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
> >>>>>> Wido den Hollander
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
> >>>> have
> >>>>>> pushed the
> >>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
> >>>>>> object thems for
> >>>>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> >>>> sake
> >>>>>> of clarity I
> >>>>>>> just
> >>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
> >>>> IOps
> >>>>>> to R/W IOps
> >>>>>>>>>> asap
> >>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
> >>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
> >>>> the
> >>>>>> groundwork for
> >>>>>>>>>> the
> >>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
> >>>>>> prevent that we
> >>>>>>> create
> >>>>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 2013/5/31
> >>>>>> Wido den
> >>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
> >>>>>> enhancement must
> >>>>>>> to
> >>>>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>> how
> >>>>>> it could be
> >>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
> >>>> IOPS
> >>>>>> into read
> >>>>>>> and
> >>>>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>> unless/until we
> >>>>>>> decided
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
> >> Wei
> >>>>>> meant was merging
> >>>>>>> it
> >>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >> branch
> >>>>>> and add Read /
> >>>>>>>>>> Write
> >>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> >>>>>> released with Read
> >>>>>>> and
> >>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
> >>>>>> make the May 31st
> >>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
> >> other
> >>>>>> threads) then it
> >>>>>>>>>> won't
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> >>>>>> completely agree that
> >>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May
> >> 31,
> >>>>>> 2013, at 9:51 AM,
> >>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> >>>> Good
> >>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
> >>>>>> about at the
> >>>>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> difference
> >>>> of
> >>>>>> read and write
> >>>>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
> >>>>>> difference of sent
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> >>>>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O
> >> is
> >>>>>> much more
> >>>>>>>>>> expensive
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> Implementing
> >>>>>> it will be some
> >>>>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> implemented
> >>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
> >>>> For
> >>>>>> the deadline
> >>>>>>> of
> >>>>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
> >>>>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think
> >> it's
> >>>> a
> >>>>>> feature we
> >>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>> we
> >>>>>> have to make sure
> >>>>>>> we
> >>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
> >>>> that
> >>>>>> will confuse
> >>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> 2013/5/31
> >>>>>> Wido den
> >>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
> >>>> like to
> >>>>>> merge
> >>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
> >>>>>> object, I will
> >>>>>>> merge
> >>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>> purpose
> >>>>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> >>>>>> machines are running
> >>>>>>>>>> on
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> >>>> strage).
> >>>>>> Because of
> >>>>>>> the
> >>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
> >>>> has
> >>>>>> large disk
> >>>>>>>>>> operation,
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >> VMs
> >>>>>> running on the
> >>>>>>> same
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>> neccesary
> >>>>>> to set the
> >>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking
> >> at
> >>>> the
> >>>>>> code I see
> >>>>>>> you
> >>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
> >> and
> >>>>>> libvirt support
> >>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>> which
> >>>>>> could benefit a
> >>>>>>>>>> lot of
> >>>>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
> >>>>>> strange, in the
> >>>>>>>>>> polling
> >>>>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
> >>>> on
> >>>>>> the throttling
> >>>>>>>>>> side
> >>>>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >> IOps
> >>>> are
> >>>>>> usually much
> >>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
> >> valid
> >>>>>> use-case where
> >>>>>>>>>> that
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
> >>>> Read
> >>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
> >>>> only
> >>>>>> supports KVM
> >>>>>>>>>> at
> >>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to
> >> at
> >>>>>> least have the
> >>>>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
> >>>>>> would be a lot of
> >>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> >>>> hypervisor
> >>>>>> doesn't
> >>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
> >>>> can
> >>>>>> always sum
> >>>>>>> both
> >>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> >>>>>> explain why you
> >>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>> feature
> >>>>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> >> the
> >>>>>> maximum rate of
> >>>>>>>>>> VMs
> >>>>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> >> change
> >>>> the
> >>>>>> maximum rate
> >>>>>>> of
> >>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
> >>>> the
> >>>>>> disk rate
> >>>>>>> (total
> >>>>>>>>>>>> bps
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
> >> ticket:
> >>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
> >>>>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>
> >>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
> >>>> check
> >>>>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
> >>>>>> check the branch's
> >>>>>>>>>> RAT
> >>>>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> >>>> there
> >>>>>> new dependencies
> >>>>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> >>>>>> automated testing
> >>>>>>> (unit
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
> >>>> are
> >>>>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> >>>>>> testing has been done
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> >> the
> >>>>>> bytes rate and
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
> >>>>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
> >>>> stop,
> >>>>>> start, reboot,
> >>>>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> >>>> Volume
> >>>>>> operations,
> >>>>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> >>>>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> >> review
> >>>> the
> >>>>>> code, you can
> >>>>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> >>>> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>
> >>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >>>> 2071
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>>>> 2071
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
> >>>> Disk
> >>>>>> I/O
> >>>>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >>>> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >>>> the
> >>>>>> way the world
> >>>>>>>>>> uses
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>> Developer,
> >>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>>> way
> >>>>>> the world uses
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack
> >>>>>> Developer,
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >>>> way
> >>>>>> the world uses
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>> Developer,
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >>>> the
> >>>>>> world uses the
> >>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>> Developer,
> >>>>>>> SolidFire
> >>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >>>> the
> >>>>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>>>> Developer, SolidFire
> >>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>>>> world uses the
> >>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>> SolidFire
> >>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>> world
> >>>>>> uses the
> >>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>> SolidFire
> >>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>> world
> >>>>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>>>> SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>>> uses
> >>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>>> uses
> >>>>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >>>> the
> >>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >> the
> >>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire
> >>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *(tm)*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *(tm)*
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *(tm)*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*
> >
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Edison,
Please see my comments in-line below.
Thanks,
-John
On Jun 7, 2013, at 6:49 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: John Burwell [mailto:jburwell@basho.com]
>> Sent: Friday, June 07, 2013 3:26 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>>
>> On Jun 7, 2013, at 6:13 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>> Sent: Friday, June 07, 2013 2:37 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>
>>>> As we only have three weeks until feature freeze, we should come to a
>>>> consensus on this design point as soon as possible.
>>>>
>>>> Right now, if the storage framework asks my driver it is is managed, it
>>>> will say 'yes.' This means the framework will tell the driver to perform
>>>> its management activities. This then means the driver will call into the
>>>> host (it doesn't know which hypervisor, by the way) to perform the
>> activity
>>>> of, say, creating an SR on XenServer or a datastore on ESX.
>>>>
>>>> The driver doesn't know which hypervisor it's talking to, it just sends a
>>>> message to the host to perform the necessary pre-attach work.
>>>
>>> Could we just expose a method like "attachVolume/dettachVolume" on
>> the PrimaryDataStoreDriver driver?
>>> For most of cases, the implementation of each driver would just send
>> attachvolumecommand/dettachvolumecmd to hypervisor(we can put the
>> implementation in a base class, so that can be shared by all of this drivers),
>> and in each hypervisor resource code, which may just call hypervisor's api to
>> attach the volume to VM, while for certain storage, like SolidFire, may need
>> to create a SR first, and create volume on it, then call hypervisor's API to
>> attach volume to VM.
>>> While for some other storage vendor, may want to bypass hypervisor
>> during attaching volume, so inside driver's attachvolume implementation,
>> the driver can do some magic, such as, directly talk to an agent inside VM
>> instance, then create a disk inside VM.
>>>
>>> How do you guys think?
>>
>> This behavior is the opposite of my expectations. I would expect the
>> VirtualMachineManager to coordinate this process. Why would Storage
>> need to know or care about a Hypervisor? My understanding is that Xen
>> needs to create an SR in a particular manner on all ISCSI devices upon
>> allocation. On attach, the Xen plugin would query the datastore to
>> determine the presence of an SR, and create it as necessary. It would
>> proceed with the creating the volume, etc. In the "real world", SANs have no
>> knowledge about the concept of a hypervisor let alone specific
>> implementations. I think we would be wise to replicate that model in
>> CloudStack.
>
> I think there are volumes managed/created by hypervisor(the normal case), and there are volumes managed by storage vendor(solidfire case). How to expose the volume to user/VM may be quite different in both cases.
> If the volume is managed by storage vendor, then vendor wants the volume to be used by hypervisor, won't it be the storage vendor's responsibility to do whatever the hypervisor wants?
> It's the same, if volume is managed by storage vendor, and vendor wants the volume to used by VM directly, won't it be the storage vendor's responsibility to do whatever magic they want?
> If the responsibility is on the storage vendor, then why we can't let storage vendor to do whatever necessary steps to get the things done?
It is not the responsibility of the storage vendor to understand the needs of the hypervisor. It is the responsibility of the storage vendor to provide interfaces to perform certain low level functions (e.g. create a volume, delete a volume, etc). Users of the DataStore compose these operations to achieve certain goals (e.g. attach a volume to a hypervisor, create structures, create volumes, etc).
You are describing the managed vs. unmanaged device concept that the SolidFire patch introduces. For DataStoreDrivers that can be managed, the associated DataStores can have management enabled. When a device is manageable, then the Storage layer recognizes that it can do more things with it. For example, when executing attachVolume, the orchestration pieces can asked managed DataStores to ensure that the underlying storage has been allocated to support the volume. From a CloudStack perspective, we know when management functions need to be invoked -- we only need the driver to provide the discrete operations. For the volume attach scenario we are discussing, I am thinking the basic would be as follows -- starting in a VirtualMachineManager:
1. VirtualMachineManager initiates the allocation process -- creating records, networks, etc (hand wavy a bit) until we get to storage
2. For each Disk Offering invoke VolumeManager
A. Allocate through VolumeManager
i. VolumeManager acquires the DataStore, and if managed, allocates the underlying storage. If the DataStore is not managed and the underlying storage is not managed, an exception is thrown.
B. The resulting volume is attached to the new VM through the hypervisor plugin (callback point for Xen to create an SR structure -> query the new volume to determine whether or not the SR is present)
3. The rest of the VM allocation process completes
Based on this process, the DataStoreDriver provides the discrete operations necessary to be composed by the Storage and Hypervisor orchestration components to complete various goals.
Does this process make sense?
>
>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> How's about this:
>>>>>>
>>>>>> The driver can implement an isManaged() method. The
>>>> VolumeManagerImpl
>>>>>> can
>>>>>> call into the driver to see if its managed. If it is, the
>>>>> VolumeManagerImpl
>>>>>> (which is responsible for calling into the hypervisor to attach the disk)
>>>>>> can call into the hypervisor to create the necessary hypervisor data
>>>>>> structure (ex. for XenServer, a storage repository).
>>>>>
>>>>> The problem here is that storage vendor may work differently with
>>>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
>> other
>>>>> vendor wants to totally bypass hypervisor, and assign the LUN directly to
>>>>> VM instance, see the discuss(
>>>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>>>>
>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>>>> %3E
>>>>> ).
>>>>> So I would let storage provider to implement attach disk to VM, instead
>> of
>>>>> implemented by cloudstack itself.
>>>>>
>>>>>
>>>>>>
>>>>>> If that's what you're going for, that works for me. By the way, Edison's
>>>>>> default storage plug-in (which handles the default storage behavior in
>>>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
>>>>>> hypervisors. You might want to contact him and inform him of your
>>>>> concerns
>>>>>> or that logic (as is) will make it to production.
>>>>>>
>>>>>> Please let me know if what I wrote in above (for my solution) is OK with
>>>>>> you. :)
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> Please see my responses in-line below.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hey John,
>>>>>>>>
>>>>>>>> I still have a bit more testing I'd like to do before I build up a
>>>>> patch
>>>>>>>> file, but this is the gist of what I've done:
>>>>>>>>
>>>>>>>> * During a volume-attach operation, after VolumeManagerImpl tells
>>>>>>>> VolumeServiceImpl to have the driver create a volume, I have
>>>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
>>>>>> managed.
>>>>>>>> If it is managed, VolumeServiceImpl has the driver perform
>> whatever
>>>>>>>> activity is required. In my case, this includes sending a message to
>>>>> the
>>>>>>>> host where the VM is running to have, say XenServer, add a storage
>>>>>>>> repository (based on the IP address of the SAN, the IQN of the SAN
>>>>>>> volume,
>>>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can on
>>>>> the
>>>>>>>> storage repository). After this, the normal attach-volume message is
>>>>> sent
>>>>>>>> to the host by VolumeManagerImpl.
>>>>>>>
>>>>>>> There should be **no** code from a storage driver to a hypervisor. I
>>>>>>> apologize for the repetition, but we simply can not have hypervisor
>>>>>>> specific code in the storage layer. The circular dependencies between
>>>>> the
>>>>>>> two layers are not sustainable in the long term. Either the
>>>>> VirtualManager
>>>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>>>> coordinate
>>>>>>> volume creation and then populating the SR. Ideally, we can
>>>>> generalize the
>>>>>>> process flow for attaching volumes such that the Xen hypervisor
>> plugin
>>>>>>> would only implement callbacks to perform the attach action and
>> create
>>>>>> the
>>>>>>> structure and SR. To my mind, the SolidFire driver should only be
>>>>>>> allocating space and providing information about contents (e.g. space
>>>>>>> available, space consumed, streams to a URI, file handle for a URI,
>>>>> etc)
>>>>>>> and capabilities.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> * The reverse is performed for a detach-volume command.
>>>>>>>>
>>>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
>>>>>>> Edison's
>>>>>>>> default driver simply does a "return false;". We could add a new
>>>>>>> parameter
>>>>>>>> to the createStoragePool API command, if we want, to remove the
>>>>>>> hard-coded
>>>>>>>> return values in the drivers (although my driver will probably just
>>>>>>> ignore
>>>>>>>> this parameter and always return true since it wouldn't make sense
>>>>> for it
>>>>>>>> to ever return false). We'd need another column in the storage_pool
>>>>>> table
>>>>>>>> to store this value.
>>>>>>>
>>>>>>> Yes, I think we should have a parameter added to the
>>>> createStoragePool
>>>>>>> surfaced to the HTTP API that allows DataStores to be configured for
>>>>>>> management when their underlying drivers support it. To simplify
>>>>> things,
>>>>>>> this flag should only be mutable when the DataStore is created. It
>>>>> would be
>>>>>>> a bit crazy to take a DataStore from managed to unmanaged after
>>>>> creation.
>>>>>>>
>>>>>>>>
>>>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I agree, John. Just wanted to point out that I have a working GUI
>>>>> for
>>>>>>> you
>>>>>>>>> to review (in that document), if you'd like to check it out.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> I would like the UIs of two features reviewed together to ensure
>>>>>>>>>> consistency across the concepts of hypervisor throttled IOPs and
>>>>>>>>>> storage device provisioned IOPs. I see the potential for
>>>>> confusion,
>>>>>>>>>> and I think a side-by-side Ui review of these features will help
>>>>>>>>>> minimize any potential confusion.
>>>>>>>>>>
>>>>>>>>>> As I mentioned, the term reconciliation issue will work itself if
>>>>> it
>>>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
>>>>> throttled
>>>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi John,
>>>>>>>>>>>
>>>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent to
>>>> you
>>>>>> the
>>>>>>>>>>> other day to see how the GUI looks for provisioned storage IOPS.
>>>>>>>>>>>
>>>>>>>>>>> Several months ago, I put this topic out on the e-mail list and we
>>>>>>>>>> decided
>>>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
>>>>> dialog.
>>>>>>>>>>> Other storage vendors are coming out with QoS, so they should
>>>> be
>>>>>> able
>>>>>>> to
>>>>>>>>>>> leverage this GUI going forward (even if they, say, only use Max
>>>>>>> IOPS).
>>>>>>>>>>> These fields are optional and can be ignored for storage that
>>>>> does not
>>>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
>>>>> admin can
>>>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
>>>>>>>>>>>
>>>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or mine,
>>>>> as
>>>>>>> is
>>>>>>>>>>> decided by the admin).
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure what we can do about these two features
>> expressing
>>>>> the
>>>>>>>>>> speed
>>>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
>>>>> QoS in
>>>>>>>>>> any
>>>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
>>>>> IOPS).
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>>>> <jb...@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Wei,
>>>>>>>>>>>>
>>>>>>>>>>>> We have been down the rabbit hole a bit on the
>>>> Storage/Hypervisor
>>>>>>> layer
>>>>>>>>>>>> separation, but we still need to reconcile the behavior of
>>>>> hypervisor
>>>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the following
>>>>>>> issues
>>>>>>>>>>>> outstanding:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>>>> read/write
>>>>>>>>>> values
>>>>>>>>>>>> where as storage provisioned IOPS are expressed as total IOPS.
>>>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
>>>> storage
>>>>>>> volumes
>>>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>>>> 3. How should usage data be captured for throttled and
>>>>> provisioned
>>>>>>> IOPS
>>>>>>>>>>>> that will permit providers to discriminate these guaranteed
>>>>>>> operations
>>>>>>>>>> in
>>>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>>>> 4. What is the user experience for throttled and provisioned
>>>> IOPS
>>>>>>> that
>>>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>>>
>>>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
>>>>> throttled
>>>>>>>>>> IOPS
>>>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
>>>>> items 1
>>>>>>>>>> and 2.
>>>>>>>>>>>> Since the two facilities would not be permitted to operate
>>>>> together,
>>>>>>>>>> they
>>>>>>>>>>>> do not need to be semantically compatible. I think item 3 can
>> be
>>>>>>>>>> resolved
>>>>>>>>>>>> with an additional flag or two on the usage records. As for
>>>>> Item 4,
>>>>>>> I
>>>>>>>>>> am
>>>>>>>>>>>> not familiar with how these two enhancements are depicted in
>>>> the
>>>>>> user
>>>>>>>>>>>> interface. I think we need to review the UI enhancements for
>>>>> both
>>>>>>>>>>>> enhancements and ensure they are consistent.
>>>>>>>>>>>>
>>>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
>> <us...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> John and Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
>>>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>>>> CLOUDSTACK-
>>>>>> 2875 ) this
>>>>>>>>>> week.
>>>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and ask
>>>>> for
>>>>>>>>>> second
>>>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I believe I understand where you're going with this, John,
>> and
>>>>>> have
>>>>>>>>>> been
>>>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>>>>>> <jb...@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> "I am fine with the VolumeManager determining whether
>> or
>>>>>> not a
>>>>>>>>>> Volume
>>>>>>>>>>>> is
>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>>>> actual
>>>>>>>>>> isManaged
>>>>>>>>>>>>>>> method), and asking the device driver to allocate resources
>>>>> for
>>>>>>> the
>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged() method
>>>>>> added to
>>>>>>>>>> the
>>>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then
>> the
>>>>>>>>>> storage
>>>>>>>>>>>>>>> framework could call the manage() (or whatever name)
>>>> method
>>>>>> (which
>>>>>>>>>>>> would
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
>>>> this
>>>>>>> would
>>>>>>>>>>>> call
>>>>>>>>>>>>>>> into a new method in the hypervisor code to create, say on
>>>>>>>>>> XenServer,
>>>>>>>>>>>> an
>>>>>>>>>>>>>> SR?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>>>>>> DataStoreDriver.
>>>>>>>>>> Since
>>>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage, I
>>>>> would
>>>>>>> add
>>>>>>>>>> a
>>>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
>>>> man
>>>>>> term
>>>>>>>>>> --
>>>>>>>>>>>>>>> something other than volume) methods to
>>>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
>>>> property
>>>>>> of
>>>>>>>>>>>>>> DataStore
>>>>>>>>>>>>>>> which can be enabled if/when the underlying
>>>> DataStoreDriver
>>>>>> can be
>>>>>>>>>>>>>> managed.
>>>>>>>>>>>>>>> This approach allows operators to override manageability of
>>>>>>> devices.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
>> plugin
>>>>>>> would
>>>>>>>>>> be
>>>>>>>>>>>>>>> responsible for composing DataStore/Volume methods to
>>>>>> create any
>>>>>>>>>>>>>>> directories or files necessary for the SR. There should be
>> no
>>>>>>>>>>>>>> dependencies
>>>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said earlier,
>>>>> such
>>>>>>>>>>>>>> circular
>>>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>>>>>> <jb...@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer that
>>>>>>> supports n
>>>>>>>>>>>>>>>> devices types with each specific behaviors of m
>> hypervisors.
>>>>>>> Such
>>>>>>>>>> a
>>>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
>>>> beast.
>>>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve the
>>>>>> Storage
>>>>>>> layer
>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am fine with the VolumeManager determining whether
>>>> or
>>>>>> not a
>>>>>>>>>> Volume
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>>>> actual
>>>>>>>>>>>> isManaged
>>>>>>>>>>>>>>>> method), and asking the device driver to allocate
>> resources
>>>>> for
>>>>>>> the
>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs to
>>>>>>> indicate
>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>> or not it supports management operations. Finally, I think
>>>>> we
>>>>>>>>>> need to
>>>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
>>>>>>> something
>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
>>>> managing
>>>>>> the
>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>> but the administrator has elected to leave it unmanaged).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Creation of a structure on the volume should be done in
>>>> the
>>>>>> Xen
>>>>>>>>>>>>>>>> hypervisor module using methods exposed by the
>> Storage
>>>>>> layer to
>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a file,
>>>>> etc).
>>>>>>>>>>>> This
>>>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>>>>> hypervisor, as
>>>>>>>>>> such,
>>>>>>>>>>>>>>>> should be confined to its implementation. From my
>>>>>> perspective,
>>>>>>>>>>>> nothing
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> the Storage layer should be concerned with content.
>> From
>>>> its
>>>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>>>> structure and data are opaque. It provides the means to
>>>>> query
>>>>>>> the
>>>>>>>>>>>> data
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> support the interpretation of the content by higher-level
>>>>>> layers
>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a composition
>>>> of
>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>> from the Storage layer that varies based on the Volume
>>>>>> storage
>>>>>>>>>>>> protocol
>>>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alternatively to the way the attach logic is implemented in
>>>>> my
>>>>>>>>>> patch,
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>>>>>>>>>> VolumeManagerImpl
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
>>>>>> hypervisor.
>>>>>>> Before
>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> command is sent, we could check to see if we're dealing
>>>> with
>>>>>>>>>> Dynamic
>>>>>>>>>>>> (or
>>>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
>>>>> "Create
>>>>>>>>>> SR"
>>>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we would
>>>> then
>>>>>>>>>> proceed
>>>>>>>>>>>> to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This way the attach logic remains the same and we just
>> add
>>>>>>> another
>>>>>>>>>>>>>>>> command to the agent code that is called for this
>> particular
>>>>>> type
>>>>>>>>>> of
>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I created a document for a customer today that outlines
>>>> how
>>>>>> the
>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
>> use
>>>>> to
>>>>>>>>>> you, as
>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have shared this document with you (you should have
>>>>>> received
>>>>>>>>>> that
>>>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
>>>> but
>>>>>>> want
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
>>>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
>>>>>> coherent
>>>>>>>>>> form
>>>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
>>>>> notion
>>>>>>> of
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the
>>>>>> notion
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
>> was
>>>>>>>>>> incomplete,
>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
>> this.
>>>>> :)
>>>>>> Do
>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
>>>> what
>>>>>> you're
>>>>>>>>>> looking
>>>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
>>>>> notion
>>>>>>> of
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
>>>>>> capabilities
>>>>>>>>>> (i.e.
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
>>>> managed
>>>>>> by
>>>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
>>>>>> semantic
>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed. I
>>>> also
>>>>>> want
>>>>>>>>>> to
>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from the
>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Could you please answer the following questions
>>>> for
>>>>>> me
>>>>>>> with
>>>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
>>>> SAN
>>>>>>> volumes?
>>>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
>> (LUN
>>>> 0).
>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
>> and
>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
>>>> server or
>>>>>>>>>> externally
>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
>>>> volume
>>>>>> (same
>>>>>>> as a
>>>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>>>> created by the management server (via the plug-
>> in)
>>>>>> the
>>>>>>> first
>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
>>>>> still
>>>>>>>>>> want
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
>>>> today
>>>>>>> (prior
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
>>>> this
>>>>>> case -
>>>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While you
>>>> can do
>>>>>> this
>>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management server
>>>> or
>>>>>> externally
>>>>>>>>>> by
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
>>>> volumes
>>>>>> are
>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
>> There
>>>> is
>>>>>> no
>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
>>>> volume
>>>>>> and a
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
>>>>>> supports
>>>>>>> true
>>>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
>> related
>>>>>> and
>>>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
>>>> VMs
>>>>>> with
>>>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
>>>> single
>>>>>> SAN
>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
>>>> multiple
>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
>>>> QoS.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
>>>> house
>>>>>> more
>>>>>>>>>> than
>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
>>>> without
>>>>>> any of
>>>>>>> my
>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
>>>> for
>>>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which concerns
>> me
>>>> --
>>>>>>> concepts
>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
>>>>> only
>>>>>>> used
>>>>>>>>>>>> (as
>>>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
>> not
>>>>>> PreSetup an
>>>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
>>>>>> VMware, it
>>>>>>>>>> uses
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
>>>> KVM,
>>>>>> it uses
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
>>>> here
>>>>>> was to
>>>>>>>>>> make a
>>>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
>>>> Edison
>>>>>> added
>>>>>>>>>> into
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
>>>> specificy,
>>>>>> though:
>>>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
>>>> Tutkowski
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
>>>> or
>>>>>> care
>>>>>>> that
>>>>>>>>>> it
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
>>>>>> questions
>>>>>>>>>> about
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
>>>>>> SolidFire
>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
>>>> hypervisor
>>>>>> logic
>>>>>>>>>>>> knows
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
>> constraint
>>>>>> and just
>>>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
>>>> the
>>>>>> hypervisor
>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
>>>> the
>>>>>> necessary
>>>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
>>>> Burwell <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able
>> to
>>>>>> manage a
>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>>>> CloudStack
>>>>>> (e.g. the
>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
>>>> chosen
>>>>>> to
>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
>>>> make
>>>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
>>>>>> volume via
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
>>>>>> work unless
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
>>>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
>>>>>> CloudStack, an
>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
>>>> exposes it
>>>>>>> through
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
>>>>>> CloudStack,
>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
>>>>>> changes, we
>>>>>>>>>> would
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
>>>>>> necessary
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
>> have
>>>> the
>>>>>> storage
>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
>>>> from
>>>>>> the
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
>>>> create
>>>>>> the
>>>>>>>>>> SR
>>>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
>> plugin
>>>>>> should
>>>>>>> have a
>>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
>>>> available
>>>>>> for
>>>>>>> volume
>>>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>>>> interrogation of
>>>>>> the
>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
>>>>>> shouldn't
>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
>> able
>>>> to
>>>>>>> answer
>>>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
>>>> driver
>>>>>> (which
>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
>>>> new
>>>>>> Primary
>>>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
>>>> that
>>>>>>> references
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
>>>>>> volume. This
>>>>>>>>>> leads
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
>> CloudStack
>>>>>> volume to a
>>>>>>>>>> VM
>>>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
>>>> the
>>>>>> plug-in
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
>>>> SAN).
>>>>>> The
>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
>> data
>>>>>> (like the
>>>>>>>>>> IQN of
>>>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
>>>>>> volume
>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
>> the
>>>>>> plug-in
>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
>> and
>>>>>> detects
>>>>>>> the
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
>>>> of a
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
>>>>>> terminology,
>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
>>>>>> allocation on
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
>>>> mean
>>>>>> 'SAN
>>>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
>>>> can
>>>>>> make in
>>>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
>> Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
>>>> make
>>>>>> sense, but
>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in
>> on
>>>>>> this
>>>>>>>>>> conversation
>>>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>>>> approach I
>>>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
>>>> John
>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
>>>>>> distinct
>>>>>>>>>> concepts
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
>> protocols.
>>>> In
>>>>>> both
>>>>>>>>>> cases, we
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system
>> to
>>>>>>> create/delete
>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
>> the
>>>>>> volume to
>>>>>>> be
>>>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
>>>> To
>>>>>> ensure
>>>>>>>>>> that
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
>>>>>> definitions,
>>>>>>>>>> refers to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
>>>>>> Minimally,
>>>>>>> we
>>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
>> device
>>>> to
>>>>>> move bits
>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>>>> Optionally, a
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
>> Therefore,
>>>>>> we can have
>>>>>>> a
>>>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
>>>> we
>>>>>> can have a
>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
>>>> capable
>>>>>> of
>>>>>>>>>> allocating
>>>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
>>>> may
>>>>>> be able
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
>>>>>> unmanaged by
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
>>>>>> operator has
>>>>>>>>>> chosen
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
>>>> my
>>>>>>> reasoning
>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
>>>>>> reasonable, it
>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
>>>> from
>>>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
>>>> driver
>>>>>> that
>>>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
>>>> establish
>>>>>> a
>>>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
>> that
>>>> is
>>>>>>>>>> associated
>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
>>>>>> drivers can
>>>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
>>>> a
>>>>>> device
>>>>>>> managed
>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
>>>> by
>>>>>> the driver to
>>>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
>>>> sense,
>>>>>> then we can
>>>>>>>>>> sit
>>>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
>>>> the
>>>>>> current
>>>>>>> iSCSI
>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
>>>> and
>>>>>> create
>>>>>>> a
>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
>> time,
>>>>>> the
>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
>> allocated).
>>>>>> Future
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
>> created
>>>>>> behind the
>>>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
>>>> go in
>>>>>> and
>>>>>>> create
>>>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
>> delete
>>>>>> volumes on
>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
>>>>>> storage
>>>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
>>>> was
>>>>>> created,
>>>>>>> the
>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
>>>> After
>>>>>> this is
>>>>>>>>>> done,
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
>> called.
>>>>>> No
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
>>>> was
>>>>>> just
>>>>>>>>>> created.
>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
>>>> Mike
>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
>> should
>>>>>> probably
>>>>>>> have
>>>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
>>>> pre-
>>>>>> allocated
>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
>>>> hypervisor
>>>>>> data
>>>>>>>>>> structure
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
>>>> then
>>>>>> that
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
>>>> (VDI)
>>>>>> are later
>>>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
>>>> these
>>>>>> cases, the
>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
>>>> already
>>>>>> in place
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
>> structure
>>>> to
>>>>>> the VM in
>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
>>>> not
>>>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
>>>> be a
>>>>>> SAN
>>>>>>> (not
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
>>>>>> structure
>>>>>>>>>> must be
>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
>>>> these
>>>>>> types of
>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
>>>>>> structure by an
>>>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
>> XenServer
>>>> for
>>>>>> this
>>>>>>>>>> volume,
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
>> VM
>>>> in
>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
>>>>>> John Burwell
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation
>> of
>>>> the
>>>>>> Dynamic
>>>>>>> type
>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
>>>> ask
>>>>>> about the
>>>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
>>>>>> definition
>>>>>>> of
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
>>>> Mike
>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
>> type
>>>> is
>>>>>>> uniquely
>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
>>>> Storage
>>>>>> that is
>>>>>>>>>> directly
>>>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
>>>> step of
>>>>>> going
>>>>>>> to
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
>>>> that
>>>>>> iSCSI
>>>>>>>>>> target as
>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
>>>>>> supported for
>>>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first
>> go
>>>> to
>>>>>> the
>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD
>> is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
>>>> PM,
>>>>>> John
>>>>>>> Burwell
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
>>>> what
>>>>>> is the
>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
>>>> RBD
>>>>>> considered
>>>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
>>>> Mike
>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
>>>>>> defined in
>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>>>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
>>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
>>>> //
>>>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
>>>> LUN,
>>>>>> with a
>>>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
>>>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
>>>> local
>>>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
>>>> VMFS
>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
>>>>>> XenServer, Storage
>>>>>>>>>> Pool
>>>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
>>>> local
>>>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
>>>>>> zone-wide
>>>>>>>>>> storage
>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
>>>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> 1:41
>>>> PM,
>>>>>> Mike
>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
>>>> another
>>>>>> storage
>>>>>>>>>>>> company
>>>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
>>>> Quality
>>>>>> of
>>>>>>> Service
>>>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
>> well.
>>>>>> They would
>>>>>>>>>> need
>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
>>>>>> necessary
>>>>>>>>>> changes to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>>> 1:39
>>>>>> PM, Mike
>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
>>>>>> XenServer code
>>>>>>> is
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
>>>> logic is
>>>>>> now aware
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
>>>>>> implementation of
>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
>>>> with
>>>>>> Edison's
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
>>>>>> framework that
>>>>>>>>>>>> supported
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
>>>>>> However, when I
>>>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
>>>> it
>>>>>> was not
>>>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
>> could
>>>> do
>>>>>> with these
>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
>>>> this
>>>>>> new type of
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
>> structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>>> 1:28
>>>>>> PM, John
>>>>>>>>>> Burwell
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to
>> me
>>>>>> that any
>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
>>>> about
>>>>>> details. I
>>>>>>>>>> expect
>>>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
>>>> operations
>>>>>> that can be
>>>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
>>>>>> allocate
>>>>>>> space/create
>>>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
>>>>>> dependency, we are
>>>>>>>>>> going
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
>> make
>>>> the
>>>>>> system
>>>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
>>>>>> understanding that the
>>>>>>>>>> Xen
>>>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>>>>>> CitrixResourceBase
>>>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>>> 3:12
>>>>>> PM, Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
>>>> bit
>>>>>> more
>>>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
>>>> from
>>>>>> one setup
>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
>>>> volume on
>>>>>> the
>>>>>>> storage
>>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
>>>> structure
>>>>>> to make
>>>>>>> use
>>>>>>>>>> of
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>>>> XenServer
>>>>>> or a
>>>>>>> datastore
>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
>>>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
>>>> storage
>>>>>> framework,
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
>>>> create
>>>>>> a 1:1
>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
>>>> data
>>>>>> disk
>>>>>>>>>> (necessary
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
>>>> plug-
>>>>>> in that is
>>>>>>>>>> called
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
>>>>>> storage
>>>>>>> framework
>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>>>> hypervisor-
>>>>>> specific data
>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
>>>> I've
>>>>>> been
>>>>>>>>>> enhancing
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
>>>>>> submitted. I've got
>>>>>>>>>> ESX(i)
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
>>>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
>> bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> at
>>>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in
>> -
>>>> by
>>>>>> itself - is
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
>>>>>> volume-attach
>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
>>>> the
>>>>>> plug-in to
>>>>>>>>>> have
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
>>>> comes to
>>>>>> attach the
>>>>>>>>>> volume
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
>>>> enough
>>>>>> to recognize
>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
>>>> has
>>>>>> just been
>>>>>>>>>> created)
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
>>>>>> XenServer) or a
>>>>>>>>>> datastore
>>>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
>> just
>>>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
>>>> most
>>>>>> of my time
>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>>>> at
>>>>>> 12:48 PM,
>>>>>>> John
>>>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
>>>> why
>>>>>> the the storage
>>>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>>>> 1:21
>>>>>> PM, Mike
>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
>>>>>> would like to
>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
>>>> I'm
>>>>>> just out of
>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
>>>> plug-in
>>>>>> supports
>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
>>>> feature
>>>>>> freeze, I
>>>>>>>>>> have
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
>>>> clear,
>>>>>> this is
>>>>>>> all
>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> Compute
>>>>>> Offerings post
>>>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013
>>>>>> at 11:14 AM,
>>>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
>> plan
>>>> on
>>>>>> supporting
>>>>>>>>>> KVM in
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
>>>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
>>>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>>>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
>>>> June
>>>>>> 3, 2013
>>>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
>>>> [MERGE]
>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
>>>> merging
>>>>>> Wei's feature
>>>>>>>>>>>> first,
>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
>>>> for
>>>>>> KVM only,
>>>>>>>>>> then
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>>>> 2013
>>>>>> at 8:55 AM,
>>>>>>> Wei
>>>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
>>>> as
>>>>>> no one works
>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
>>>>>> themselves. They
>>>>>>>>>> can
>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering
>> of
>>>> a
>>>>>> VMs and
>>>>>>> volumes
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
>>>>>> exact
>>>>>>> limitation
>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
>>>> for
>>>>>> nics as
>>>>>>> well.
>>>>>>>>>> I
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
>>>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
>>>>>> misunstand the second
>>>>>>> the
>>>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
>>>> what
>>>>>> you said about
>>>>>>> the
>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
>> 2013,
>>>> at
>>>>>> 2:13 AM, Wei
>>>>>>>>>> ZHOU
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
>> Mike's
>>>>>> aswer helps you.
>>>>>>>>>> I am
>>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
>>>> billing
>>>>>> should
>>>>>>>>>> depend
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
>>>> Please
>>>>>> review
>>>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
>>>>>> statistics
>>>>>>> including
>>>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
>>>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
>>>> AWS
>>>>>> model,
>>>>>>> customers
>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>>>> IOPS,
>>>>>> as well as,
>>>>>>> for
>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://aws.amazon.com/ebs/).
>>>>>>> I
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
>>>> employ
>>>>>> similar cost
>>>>>>>>>>>> models.
>>>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
>>>> in
>>>>>> the current
>>>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
>>>> mean
>>>>>> IOPS runtime
>>>>>>>>>>>> change?
>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
>> for
>>>> a
>>>>>> running
>>>>>>> virtual
>>>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>> does
>>>>>> not support
>>>>>>>>>>>> changing
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
>>>>>> offering or disk
>>>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
>> the
>>>>>> Java interface
>>>>>>>>>> level.
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
>>>>>> generalize allocation
>>>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
>>>>>> service guarantees
>>>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
>>>> good
>>>>>> question.
>>>>>>>>>> Maybe it
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> disk_io_throttling as Mike
>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
>>>> on
>>>>>> my thoughts
>>>>>>> in a
>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
>>>>>> between these two
>>>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to
>> be
>>>>>> merged before
>>>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
>>>>>> between the
>>>>>>> branches
>>>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> disk_io_throttling)
>>>>>> to
>>>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>>>> John
>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things
>> I
>>>>>> want to
>>>>>>> understand
>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
>>>>>> value in
>>>>>>> capturing
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>> model
>>>> (e.g.
>>>>>> for
>>>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
>>>> there
>>>>>> be a common
>>>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> provisioning
>>>> at
>>>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
>>>>>> conflicting
>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>> and
>>>>>> storage device
>>>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
>>>>>> believe (and
>>>>>>> billed)
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>> device
>>>>>> has been
>>>>>>>>>> configured
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
>>>>>> configuration
>>>>>>>>>> between a
>>>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
>>>>>> later
>>>>>>> modification
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
>>>> 2013,
>>>>>> at 8:38 PM,
>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
>>>> Wei's
>>>>>> feature deals
>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> hypervisor
>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> feature
>>>> is
>>>>>> focused on
>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
>>>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
>> Jun
>>>> 2,
>>>>>> 2013 at 6:35
>>>>>>> PM,
>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>>>> opinion is
>>>>>> that no
>>>>>>> features
>>>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
>>>> have
>>>>>> been resolved
>>>>>>> and
>>>>>>>>>> it
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
>>>> vs
>>>>>> discrete
>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
>>>> don't
>>>>>> think this
>>>>>>>>>> criteria
>>>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
>> how
>>>>>> does this work
>>>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>> involves
>>>>>> provisioned
>>>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>> where provisioned
>>>>>>>>>> IOPS
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
>>>> 2013,
>>>>>> at 6:47 AM,
>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
>> will
>>>>>> change it next
>>>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
>>>>>> Wido den Hollander
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
>>>> have
>>>>>> pushed the
>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
>>>>>> object thems for
>>>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
>>>> sake
>>>>>> of clarity I
>>>>>>> just
>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
>>>> IOps
>>>>>> to R/W IOps
>>>>>>>>>> asap
>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
>>>> the
>>>>>> groundwork for
>>>>>>>>>> the
>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
>>>>>> prevent that we
>>>>>>> create
>>>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 2013/5/31
>>>>>> Wido den
>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
>>>>>> enhancement must
>>>>>>> to
>>>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>> how
>>>>>> it could be
>>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
>>>> IOPS
>>>>>> into read
>>>>>>> and
>>>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>> unless/until we
>>>>>>> decided
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
>> Wei
>>>>>> meant was merging
>>>>>>> it
>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> branch
>>>>>> and add Read /
>>>>>>>>>> Write
>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
>>>>>> released with Read
>>>>>>> and
>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
>>>>>> make the May 31st
>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
>> other
>>>>>> threads) then it
>>>>>>>>>> won't
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
>>>>>> completely agree that
>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May
>> 31,
>>>>>> 2013, at 9:51 AM,
>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>> Good
>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
>>>>>> about at the
>>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> difference
>>>> of
>>>>>> read and write
>>>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
>>>>>> difference of sent
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>>>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O
>> is
>>>>>> much more
>>>>>>>>>> expensive
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> Implementing
>>>>>> it will be some
>>>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> implemented
>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
>>>> For
>>>>>> the deadline
>>>>>>> of
>>>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
>>>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think
>> it's
>>>> a
>>>>>> feature we
>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>> we
>>>>>> have to make sure
>>>>>>> we
>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
>>>> that
>>>>>> will confuse
>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> 2013/5/31
>>>>>> Wido den
>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
>>>> like to
>>>>>> merge
>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
>>>>>> object, I will
>>>>>>> merge
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>> purpose
>>>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
>>>>>> machines are running
>>>>>>>>>> on
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
>>>> strage).
>>>>>> Because of
>>>>>>> the
>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
>>>> has
>>>>>> large disk
>>>>>>>>>> operation,
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>> VMs
>>>>>> running on the
>>>>>>> same
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>> neccesary
>>>>>> to set the
>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking
>> at
>>>> the
>>>>>> code I see
>>>>>>> you
>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
>> and
>>>>>> libvirt support
>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>> which
>>>>>> could benefit a
>>>>>>>>>> lot of
>>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
>>>>>> strange, in the
>>>>>>>>>> polling
>>>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
>>>> on
>>>>>> the throttling
>>>>>>>>>> side
>>>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> IOps
>>>> are
>>>>>> usually much
>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
>> valid
>>>>>> use-case where
>>>>>>>>>> that
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
>>>> Read
>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
>>>> only
>>>>>> supports KVM
>>>>>>>>>> at
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to
>> at
>>>>>> least have the
>>>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
>>>>>> would be a lot of
>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
>>>> hypervisor
>>>>>> doesn't
>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
>>>> can
>>>>>> always sum
>>>>>>> both
>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
>>>>>> explain why you
>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>> feature
>>>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> the
>>>>>> maximum rate of
>>>>>>>>>> VMs
>>>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
>> change
>>>> the
>>>>>> maximum rate
>>>>>>> of
>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
>>>> the
>>>>>> disk rate
>>>>>>> (total
>>>>>>>>>>>> bps
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
>> ticket:
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
>>>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
>>>> check
>>>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
>>>>>> check the branch's
>>>>>>>>>> RAT
>>>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
>>>> there
>>>>>> new dependencies
>>>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
>>>>>> automated testing
>>>>>>> (unit
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
>>>> are
>>>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
>>>>>> testing has been done
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
>> the
>>>>>> bytes rate and
>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
>>>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
>>>> stop,
>>>>>> start, reboot,
>>>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>>>> Volume
>>>>>> operations,
>>>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
>>>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
>> review
>>>> the
>>>>>> code, you can
>>>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
>>>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>>>> 2071
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>>>> 2071
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
>>>> Disk
>>>>>> I/O
>>>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>>>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>>>> the
>>>>>> way the world
>>>>>>>>>> uses
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>> Developer,
>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>> way
>>>>>> the world uses
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack
>>>>>> Developer,
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>>>> way
>>>>>> the world uses
>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer,
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>>>> the
>>>>>> world uses the
>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer,
>>>>>>> SolidFire
>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>>>> the
>>>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>>>> Developer, SolidFire
>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>>>> world uses the
>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>> SolidFire
>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>> uses the
>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>> SolidFire
>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world
>>>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>>>> SolidFire
>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>>> uses
>>>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>>>> the
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *(tm)*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *(tm)*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *(tm)*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *(tm)*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*
>
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: John Burwell [mailto:jburwell@basho.com]
> Sent: Friday, June 07, 2013 3:26 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
>
> On Jun 7, 2013, at 6:13 PM, Edison Su <Ed...@citrix.com> wrote:
>
> >
> >
> >> -----Original Message-----
> >> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >> Sent: Friday, June 07, 2013 2:37 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>
> >> As we only have three weeks until feature freeze, we should come to a
> >> consensus on this design point as soon as possible.
> >>
> >> Right now, if the storage framework asks my driver it is is managed, it
> >> will say 'yes.' This means the framework will tell the driver to perform
> >> its management activities. This then means the driver will call into the
> >> host (it doesn't know which hypervisor, by the way) to perform the
> activity
> >> of, say, creating an SR on XenServer or a datastore on ESX.
> >>
> >> The driver doesn't know which hypervisor it's talking to, it just sends a
> >> message to the host to perform the necessary pre-attach work.
> >
> > Could we just expose a method like "attachVolume/dettachVolume" on
> the PrimaryDataStoreDriver driver?
> > For most of cases, the implementation of each driver would just send
> attachvolumecommand/dettachvolumecmd to hypervisor(we can put the
> implementation in a base class, so that can be shared by all of this drivers),
> and in each hypervisor resource code, which may just call hypervisor's api to
> attach the volume to VM, while for certain storage, like SolidFire, may need
> to create a SR first, and create volume on it, then call hypervisor's API to
> attach volume to VM.
> > While for some other storage vendor, may want to bypass hypervisor
> during attaching volume, so inside driver's attachvolume implementation,
> the driver can do some magic, such as, directly talk to an agent inside VM
> instance, then create a disk inside VM.
> >
> > How do you guys think?
>
> This behavior is the opposite of my expectations. I would expect the
> VirtualMachineManager to coordinate this process. Why would Storage
> need to know or care about a Hypervisor? My understanding is that Xen
> needs to create an SR in a particular manner on all ISCSI devices upon
> allocation. On attach, the Xen plugin would query the datastore to
> determine the presence of an SR, and create it as necessary. It would
> proceed with the creating the volume, etc. In the "real world", SANs have no
> knowledge about the concept of a hypervisor let alone specific
> implementations. I think we would be wise to replicate that model in
> CloudStack.
I think there are volumes managed/created by hypervisor(the normal case), and there are volumes managed by storage vendor(solidfire case). How to expose the volume to user/VM may be quite different in both cases.
If the volume is managed by storage vendor, then vendor wants the volume to be used by hypervisor, won't it be the storage vendor's responsibility to do whatever the hypervisor wants?
It's the same, if volume is managed by storage vendor, and vendor wants the volume to used by VM directly, won't it be the storage vendor's responsibility to do whatever magic they want?
If the responsibility is on the storage vendor, then why we can't let storage vendor to do whatever necessary steps to get the things done?
>
> >
> >
> >>
> >>
> >>
> >>
> >> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com> wrote:
> >>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> >>>> Sent: Friday, June 07, 2013 1:14 PM
> >>>> To: dev@cloudstack.apache.org
> >>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>
> >>>> Hi John,
> >>>>
> >>>> How's about this:
> >>>>
> >>>> The driver can implement an isManaged() method. The
> >> VolumeManagerImpl
> >>>> can
> >>>> call into the driver to see if its managed. If it is, the
> >>> VolumeManagerImpl
> >>>> (which is responsible for calling into the hypervisor to attach the disk)
> >>>> can call into the hypervisor to create the necessary hypervisor data
> >>>> structure (ex. for XenServer, a storage repository).
> >>>
> >>> The problem here is that storage vendor may work differently with
> >>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe
> other
> >>> vendor wants to totally bypass hypervisor, and assign the LUN directly to
> >>> VM instance, see the discuss(
> >>> http://mail-archives.apache.org/mod_mbox/cloudstack-
> >>
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> >> %3E
> >>> ).
> >>> So I would let storage provider to implement attach disk to VM, instead
> of
> >>> implemented by cloudstack itself.
> >>>
> >>>
> >>>>
> >>>> If that's what you're going for, that works for me. By the way, Edison's
> >>>> default storage plug-in (which handles the default storage behavior in
> >>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
> >>>> hypervisors. You might want to contact him and inform him of your
> >>> concerns
> >>>> or that logic (as is) will make it to production.
> >>>>
> >>>> Please let me know if what I wrote in above (for my solution) is OK with
> >>>> you. :)
> >>>>
> >>>> Thanks!
> >>>>
> >>>>
> >>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> Please see my responses in-line below.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hey John,
> >>>>>>
> >>>>>> I still have a bit more testing I'd like to do before I build up a
> >>> patch
> >>>>>> file, but this is the gist of what I've done:
> >>>>>>
> >>>>>> * During a volume-attach operation, after VolumeManagerImpl tells
> >>>>>> VolumeServiceImpl to have the driver create a volume, I have
> >>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> >>>> managed.
> >>>>>> If it is managed, VolumeServiceImpl has the driver perform
> whatever
> >>>>>> activity is required. In my case, this includes sending a message to
> >>> the
> >>>>>> host where the VM is running to have, say XenServer, add a storage
> >>>>>> repository (based on the IP address of the SAN, the IQN of the SAN
> >>>>> volume,
> >>>>>> etc.) and a single VDI (the VDI consumes all of the space it can on
> >>> the
> >>>>>> storage repository). After this, the normal attach-volume message is
> >>> sent
> >>>>>> to the host by VolumeManagerImpl.
> >>>>>
> >>>>> There should be **no** code from a storage driver to a hypervisor. I
> >>>>> apologize for the repetition, but we simply can not have hypervisor
> >>>>> specific code in the storage layer. The circular dependencies between
> >>> the
> >>>>> two layers are not sustainable in the long term. Either the
> >>> VirtualManager
> >>>>> or Xen hypervisor plugin needs to be refactored/modified to
> >> coordinate
> >>>>> volume creation and then populating the SR. Ideally, we can
> >>> generalize the
> >>>>> process flow for attaching volumes such that the Xen hypervisor
> plugin
> >>>>> would only implement callbacks to perform the attach action and
> create
> >>>> the
> >>>>> structure and SR. To my mind, the SolidFire driver should only be
> >>>>> allocating space and providing information about contents (e.g. space
> >>>>> available, space consumed, streams to a URI, file handle for a URI,
> >>> etc)
> >>>>> and capabilities.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> * The reverse is performed for a detach-volume command.
> >>>>>>
> >>>>>> * Right now I simply "return true;" for isManaged() in my driver.
> >>>>> Edison's
> >>>>>> default driver simply does a "return false;". We could add a new
> >>>>> parameter
> >>>>>> to the createStoragePool API command, if we want, to remove the
> >>>>> hard-coded
> >>>>>> return values in the drivers (although my driver will probably just
> >>>>> ignore
> >>>>>> this parameter and always return true since it wouldn't make sense
> >>> for it
> >>>>>> to ever return false). We'd need another column in the storage_pool
> >>>> table
> >>>>>> to store this value.
> >>>>>
> >>>>> Yes, I think we should have a parameter added to the
> >> createStoragePool
> >>>>> surfaced to the HTTP API that allows DataStores to be configured for
> >>>>> management when their underlying drivers support it. To simplify
> >>> things,
> >>>>> this flag should only be mutable when the DataStore is created. It
> >>> would be
> >>>>> a bit crazy to take a DataStore from managed to unmanaged after
> >>> creation.
> >>>>>
> >>>>>>
> >>>>>> Sound like I'm in sync with what you were thinking?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> I agree, John. Just wanted to point out that I have a working GUI
> >>> for
> >>>>> you
> >>>>>>> to review (in that document), if you'd like to check it out.
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> I would like the UIs of two features reviewed together to ensure
> >>>>>>>> consistency across the concepts of hypervisor throttled IOPs and
> >>>>>>>> storage device provisioned IOPs. I see the potential for
> >>> confusion,
> >>>>>>>> and I think a side-by-side Ui review of these features will help
> >>>>>>>> minimize any potential confusion.
> >>>>>>>>
> >>>>>>>> As I mentioned, the term reconciliation issue will work itself if
> >>> it
> >>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
> >>> throttled
> >>>>>>>> IOPs or storage provisioned IOPs.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>>>>>>> <mi...@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi John,
> >>>>>>>>>
> >>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent to
> >> you
> >>>> the
> >>>>>>>>> other day to see how the GUI looks for provisioned storage IOPS.
> >>>>>>>>>
> >>>>>>>>> Several months ago, I put this topic out on the e-mail list and we
> >>>>>>>> decided
> >>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
> >>> dialog.
> >>>>>>>>> Other storage vendors are coming out with QoS, so they should
> >> be
> >>>> able
> >>>>> to
> >>>>>>>>> leverage this GUI going forward (even if they, say, only use Max
> >>>>> IOPS).
> >>>>>>>>> These fields are optional and can be ignored for storage that
> >>> does not
> >>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
> >>> admin can
> >>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
> >>>>>>>>>
> >>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or mine,
> >>> as
> >>>>> is
> >>>>>>>>> decided by the admin).
> >>>>>>>>>
> >>>>>>>>> I'm not sure what we can do about these two features
> expressing
> >>> the
> >>>>>>>> speed
> >>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
> >>> QoS in
> >>>>>>>> any
> >>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
> >>> IOPS).
> >>>>>>>>>
> >>>>>>>>> Thanks
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> >> <jb...@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Wei,
> >>>>>>>>>>
> >>>>>>>>>> We have been down the rabbit hole a bit on the
> >> Storage/Hypervisor
> >>>>> layer
> >>>>>>>>>> separation, but we still need to reconcile the behavior of
> >>> hypervisor
> >>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the following
> >>>>> issues
> >>>>>>>>>> outstanding:
> >>>>>>>>>>
> >>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> >> read/write
> >>>>>>>> values
> >>>>>>>>>> where as storage provisioned IOPS are expressed as total IOPS.
> >>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
> >> storage
> >>>>> volumes
> >>>>>>>>>> with provisioned IOPS?
> >>>>>>>>>> 3. How should usage data be captured for throttled and
> >>> provisioned
> >>>>> IOPS
> >>>>>>>>>> that will permit providers to discriminate these guaranteed
> >>>>> operations
> >>>>>>>> in
> >>>>>>>>>> the event they want to bill for it differently?
> >>>>>>>>>> 4. What is the user experience for throttled and provisioned
> >> IOPS
> >>>>> that
> >>>>>>>>>> minimizes confusion of these concepts?
> >>>>>>>>>>
> >>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
> >>> throttled
> >>>>>>>> IOPS
> >>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
> >>> items 1
> >>>>>>>> and 2.
> >>>>>>>>>> Since the two facilities would not be permitted to operate
> >>> together,
> >>>>>>>> they
> >>>>>>>>>> do not need to be semantically compatible. I think item 3 can
> be
> >>>>>>>> resolved
> >>>>>>>>>> with an additional flag or two on the usage records. As for
> >>> Item 4,
> >>>>> I
> >>>>>>>> am
> >>>>>>>>>> not familiar with how these two enhancements are depicted in
> >> the
> >>>> user
> >>>>>>>>>> interface. I think we need to review the UI enhancements for
> >>> both
> >>>>>>>>>> enhancements and ensure they are consistent.
> >>>>>>>>>>
> >>>>>>>>>> Do these solutions make sense?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU
> <us...@gmail.com>
> >>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> John and Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> >>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> >> CLOUDSTACK-
> >>>> 2875 ) this
> >>>>>>>> week.
> >>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and ask
> >>> for
> >>>>>>>> second
> >>>>>>>>>>> merge request after testing.
> >>>>>>>>>>>
> >>>>>>>>>>> -Wei
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>>>>>>
> >>>>>>>>>>>> I believe I understand where you're going with this, John,
> and
> >>>> have
> >>>>>>>> been
> >>>>>>>>>>>> re-working this section of code today.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for the comments,
> >>>>>>>>>>>> Mike
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> >>>> <jb...@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> See my responses in-line below.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for the response.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> "I am fine with the VolumeManager determining whether
> or
> >>>> not a
> >>>>>>>> Volume
> >>>>>>>>>> is
> >>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> >> actual
> >>>>>>>> isManaged
> >>>>>>>>>>>>> method), and asking the device driver to allocate resources
> >>> for
> >>>>> the
> >>>>>>>>>>>> volume
> >>>>>>>>>>>>> if it is managed."
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Are you thinking you'd like to see an isManaged() method
> >>>> added to
> >>>>>>>> the
> >>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then
> the
> >>>>>>>> storage
> >>>>>>>>>>>>> framework could call the manage() (or whatever name)
> >> method
> >>>> (which
> >>>>>>>>>> would
> >>>>>>>>>>>> be
> >>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
> >> this
> >>>>> would
> >>>>>>>>>> call
> >>>>>>>>>>>>> into a new method in the hypervisor code to create, say on
> >>>>>>>> XenServer,
> >>>>>>>>>> an
> >>>>>>>>>>>> SR?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
> >>>> DataStoreDriver.
> >>>>>>>> Since
> >>>>>>>>>>>>> the notion of Volumes only pertains to primary storage, I
> >>> would
> >>>>> add
> >>>>>>>> a
> >>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
> >> man
> >>>> term
> >>>>>>>> --
> >>>>>>>>>>>>> something other than volume) methods to
> >>>>>>>>>> allocate/create/deallocate/delete
> >>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
> >> property
> >>>> of
> >>>>>>>>>>>> DataStore
> >>>>>>>>>>>>> which can be enabled if/when the underlying
> >> DataStoreDriver
> >>>> can be
> >>>>>>>>>>>> managed.
> >>>>>>>>>>>>> This approach allows operators to override manageability of
> >>>>> devices.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen
> plugin
> >>>>> would
> >>>>>>>> be
> >>>>>>>>>>>>> responsible for composing DataStore/Volume methods to
> >>>> create any
> >>>>>>>>>>>>> directories or files necessary for the SR. There should be
> no
> >>>>>>>>>>>> dependencies
> >>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said earlier,
> >>> such
> >>>>>>>>>>>> circular
> >>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks again, John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> >>>> <jb...@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> >>>>> supports n
> >>>>>>>>>>>>>> devices types with each specific behaviors of m
> hypervisors.
> >>>>> Such
> >>>>>>>> a
> >>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
> >> beast.
> >>>>>>>>>>>> Therefore, my
> >>>>>>>>>>>>>> thoughts and recommendations are driven to evolve the
> >>>> Storage
> >>>>> layer
> >>>>>>>>>>>> towards
> >>>>>>>>>>>>>> this separation.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I am fine with the VolumeManager determining whether
> >> or
> >>>> not a
> >>>>>>>> Volume
> >>>>>>>>>> is
> >>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> >> actual
> >>>>>>>>>> isManaged
> >>>>>>>>>>>>>> method), and asking the device driver to allocate
> resources
> >>> for
> >>>>> the
> >>>>>>>>>>>> volume
> >>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs to
> >>>>> indicate
> >>>>>>>>>>>> whether
> >>>>>>>>>>>>>> or not it supports management operations. Finally, I think
> >>> we
> >>>>>>>> need to
> >>>>>>>>>>>>>> provide the ability for an administrator to elect to have
> >>>>> something
> >>>>>>>>>>>> that is
> >>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> >> managing
> >>>> the
> >>>>>>>>>> device,
> >>>>>>>>>>>>>> but the administrator has elected to leave it unmanaged).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Creation of a structure on the volume should be done in
> >> the
> >>>> Xen
> >>>>>>>>>>>>>> hypervisor module using methods exposed by the
> Storage
> >>>> layer to
> >>>>>>>>>> perform
> >>>>>>>>>>>>>> low-level operations (e.g. make directories, create a file,
> >>> etc).
> >>>>>>>>>> This
> >>>>>>>>>>>>>> structure is specific to the operation of the Xen
> >>> hypervisor, as
> >>>>>>>> such,
> >>>>>>>>>>>>>> should be confined to its implementation. From my
> >>>> perspective,
> >>>>>>>>>> nothing
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>> the Storage layer should be concerned with content.
> From
> >> its
> >>>>>>>>>>>> perspective,
> >>>>>>>>>>>>>> structure and data are opaque. It provides the means to
> >>> query
> >>>>> the
> >>>>>>>>>> data
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> support the interpretation of the content by higher-level
> >>>> layers
> >>>>>>>> (e.g.
> >>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a composition
> >> of
> >>>>>>>>>> operations
> >>>>>>>>>>>>>> from the Storage layer that varies based on the Volume
> >>>> storage
> >>>>>>>>>> protocol
> >>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Alternatively to the way the attach logic is implemented in
> >>> my
> >>>>>>>> patch,
> >>>>>>>>>> we
> >>>>>>>>>>>>>> could do the following:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
> >>>>>>>> VolumeManagerImpl
> >>>>>>>>>> we
> >>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
> >>>> hypervisor.
> >>>>> Before
> >>>>>>>>>> this
> >>>>>>>>>>>>>> command is sent, we could check to see if we're dealing
> >> with
> >>>>>>>> Dynamic
> >>>>>>>>>> (or
> >>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
> >>> "Create
> >>>>>>>> SR"
> >>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we would
> >> then
> >>>>>>>> proceed
> >>>>>>>>>> to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This way the attach logic remains the same and we just
> add
> >>>>> another
> >>>>>>>>>>>>>> command to the agent code that is called for this
> particular
> >>>> type
> >>>>>>>> of
> >>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I created a document for a customer today that outlines
> >> how
> >>>> the
> >>>>>>>>>> plug-in
> >>>>>>>>>>>>>>> works from a user standpoint. This will probably be of
> use
> >>> to
> >>>>>>>> you, as
> >>>>>>>>>>>> well,
> >>>>>>>>>>>>>>> as you perform the code review.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I have shared this document with you (you should have
> >>>> received
> >>>>>>>> that
> >>>>>>>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Talk to you later!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> >>>>> jburwell@basho.com
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
> >> but
> >>>>> want
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> >>>>>>>>>>>> disk_io_throttle,
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
> >>>> coherent
> >>>>>>>> form
> >>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
> >>> notion
> >>>>> of
> >>>>>>>> a
> >>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the
> >>>> notion
> >>>>>>>> of
> >>>>>>>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in
> was
> >>>>>>>> incomplete,
> >>>>>>>>>> so
> >>>>>>>>>>>>>>>>> I had
> >>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on
> this.
> >>> :)
> >>>> Do
> >>>>>>>> you
> >>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
> >> what
> >>>> you're
> >>>>>>>> looking
> >>>>>>>>>>>>>>>>> for?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
> >>> notion
> >>>>> of
> >>>>>>>> a
> >>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I want to see a separation of the management
> >>>> capabilities
> >>>>>>>> (i.e.
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
> >> managed
> >>>> by
> >>>>>>>> CloudStack)
> >>>>>>>>>>>>>>>>> from the
> >>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
> >>>> semantic
> >>>>>>>> that
> >>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed. I
> >> also
> >>>> want
> >>>>>>>> to
> >>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from the
> >>>> hypervisor
> >>>>>>>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Could you please answer the following questions
> >> for
> >>>> me
> >>>>> with
> >>>>>>>>>>>>>>>>> regards to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
> >> SAN
> >>>>> volumes?
> >>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN
> (LUN
> >> 0).
> >>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes
> and
> >>>> CloudStack
> >>>>>>>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> >> server or
> >>>>>>>> externally
> >>>>>>>>>> by
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> >> volume
> >>>> (same
> >>>>> as a
> >>>>>>>>>> SAN
> >>>>>>>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>>>>>>> created by the management server (via the plug-
> in)
> >>>> the
> >>>>> first
> >>>>>>>>>> time
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
> >>> still
> >>>>>>>> want
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>> use a
> >>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> >> today
> >>>>> (prior
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
> >> this
> >>>> case -
> >>>>>>>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While you
> >> can do
> >>>> this
> >>>>>>>>>>>> today,
> >>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management server
> >> or
> >>>> externally
> >>>>>>>> by
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> >> volumes
> >>>> are
> >>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in).
> There
> >> is
> >>>> no
> >>>>> admin
> >>>>>>>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> >> volume
> >>>> and a
> >>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
> >>>> supports
> >>>>> true
> >>>>>>>>>> QoS.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are
> related
> >>>> and
> >>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
> >> VMs
> >>>> with
> >>>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> >> single
> >>>> SAN
> >>>>>>>> volume
> >>>>>>>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> >> multiple
> >>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
> >> QoS.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
> >> house
> >>>> more
> >>>>>>>> than
> >>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> >> without
> >>>> any of
> >>>>> my
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
> >> for
> >>>>>>>>>> CloudStack's
> >>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which concerns
> me
> >> --
> >>>>> concepts
> >>>>>>>>>> from
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
> >>> only
> >>>>> used
> >>>>>>>>>> (as
> >>>>>>>>>>>>>>>>> far as
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have
> not
> >>>> PreSetup an
> >>>>>>>> SR).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> >>>> VMware, it
> >>>>>>>> uses
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> >> KVM,
> >>>> it uses
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> >> here
> >>>> was to
> >>>>>>>> make a
> >>>>>>>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> >> Edison
> >>>> added
> >>>>>>>> into
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> >> specificy,
> >>>> though:
> >>>>>>>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> >> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
> >> or
> >>>> care
> >>>>> that
> >>>>>>>> it
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> >>>> questions
> >>>>>>>> about
> >>>>>>>>>> it
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> >>>> SolidFire
> >>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> >> hypervisor
> >>>> logic
> >>>>>>>>>> knows
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that
> constraint
> >>>> and just
> >>>>>>>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
> >> the
> >>>> hypervisor
> >>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
> >> the
> >>>> necessary
> >>>>>>>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> >> Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able
> to
> >>>> manage a
> >>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> >> CloudStack
> >>>> (e.g. the
> >>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> >> chosen
> >>>> to
> >>>>>>>> dedicate
> >>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> >> make
> >>>> sense?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
> >>>> volume via
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
> >>>> work unless
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
> >> SR.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> >>>> CloudStack, an
> >>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> >> exposes it
> >>>>> through
> >>>>>>>> a
> >>>>>>>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
> >>>> CloudStack,
> >>>>>>>> but
> >>>>>>>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> >>>> changes, we
> >>>>>>>> would
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> >>>> necessary
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe
> have
> >> the
> >>>> storage
> >>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
> >> from
> >>>> the
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
> >> create
> >>>> the
> >>>>>>>> SR
> >>>>>>>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management
> plugin
> >>>> should
> >>>>> have a
> >>>>>>>>>>>> rich
> >>>>>>>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> >> available
> >>>> for
> >>>>> volume
> >>>>>>>>>>>>>>>>> storage.
> >>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> >> interrogation of
> >>>> the
> >>>>>>>>>> device
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
> >>>> shouldn't
> >>>>>>>> know
> >>>>>>>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be
> able
> >> to
> >>>>> answer
> >>>>>>>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> >> driver
> >>>> (which
> >>>>>>>> is a
> >>>>>>>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
> >> new
> >>>> Primary
> >>>>>>>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> >> that
> >>>>> references
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
> >>>> volume. This
> >>>>>>>> leads
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the
> CloudStack
> >>>> volume to a
> >>>>>>>> VM
> >>>>>>>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
> >> the
> >>>> plug-in
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> >> SAN).
> >>>> The
> >>>>>>>> plug-in
> >>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable
> data
> >>>> (like the
> >>>>>>>> IQN of
> >>>>>>>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> >>>>> CloudStack
> >>>>>>>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
> >>>> volume
> >>>>>>>> will be
> >>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and
> the
> >>>> plug-in
> >>>>>>>> code
> >>>>>>>>>>>>>>>>> will not
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run
> and
> >>>> detects
> >>>>> the
> >>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
> >> of a
> >>>>>>>> hypervisor
> >>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> >>>> terminology,
> >>>>>>>> volume,
> >>>>>>>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> >>>> allocation on
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> >> mean
> >>>> 'SAN
> >>>>>>>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
> >> can
> >>>> make in
> >>>>>>>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM,
> Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
> >> make
> >>>> sense, but
> >>>>>>>> I'm
> >>>>>>>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in
> on
> >>>> this
> >>>>>>>> conversation
> >>>>>>>>>>>>>>>>> as he
> >>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> >> approach I
> >>>> took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> >> John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
> >>>> distinct
> >>>>>>>> concepts
> >>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage
> protocols.
> >> In
> >>>> both
> >>>>>>>> cases, we
> >>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system
> to
> >>>>> create/delete
> >>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires
> the
> >>>> volume to
> >>>>> be
> >>>>>>>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
> >> To
> >>>> ensure
> >>>>>>>> that
> >>>>>>>>>> we
> >>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> >>>> definitions,
> >>>>>>>> refers to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
> >>>> Minimally,
> >>>>> we
> >>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage
> device
> >> to
> >>>> move bits
> >>>>>>>> from
> >>>>>>>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> >> Optionally, a
> >>>>>>>> storage
> >>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack.
> Therefore,
> >>>> we can have
> >>>>> a
> >>>>>>>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
> >> we
> >>>> can have a
> >>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> >> capable
> >>>> of
> >>>>>>>> allocating
> >>>>>>>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
> >> may
> >>>> be able
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> >>>> unmanaged by
> >>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
> >>>> operator has
> >>>>>>>> chosen
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
> >> my
> >>>>> reasoning
> >>>>>>>>>> make
> >>>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
> >>>> reasonable, it
> >>>>>>>> seems
> >>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
> >> from
> >>>>>>>> StoragePoolType,
> >>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
> >> driver
> >>>> that
> >>>>>>>>>> indicates
> >>>>>>>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> >> establish
> >>>> a
> >>>>>>>> abstraction
> >>>>>>>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate
> that
> >> is
> >>>>>>>> associated
> >>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
> >>>> drivers can
> >>>>>>>>>>>> declare
> >>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
> >> a
> >>>> device
> >>>>> managed
> >>>>>>>> by
> >>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
> >> by
> >>>> the driver to
> >>>>>>>>>>>> automate
> >>>>>>>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> >> sense,
> >>>> then we can
> >>>>>>>> sit
> >>>>>>>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
> >> the
> >>>> current
> >>>>> iSCSI
> >>>>>>>>>>>> type
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
> >> and
> >>>> create
> >>>>> a
> >>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in
> time,
> >>>> the
> >>>>> iSCSI
> >>>>>>>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-
> allocated).
> >>>> Future
> >>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was
> created
> >>>> behind the
> >>>>>>>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
> >> go in
> >>>> and
> >>>>> create
> >>>>>>>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and
> delete
> >>>> volumes on
> >>>>>>>> its
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
> >>>> storage
> >>>>>>>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
> >> was
> >>>> created,
> >>>>> the
> >>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
> >> After
> >>>> this is
> >>>>>>>> done,
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is
> called.
> >>>> No
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
> >> was
> >>>> just
> >>>>>>>> created.
> >>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
> >> Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I
> should
> >>>> probably
> >>>>> have
> >>>>>>>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> >> pre-
> >>>> allocated
> >>>>>>>> (ex.
> >>>>>>>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> >> hypervisor
> >>>> data
> >>>>>>>> structure
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
> >> then
> >>>> that
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
> >> (VDI)
> >>>> are later
> >>>>>>>>>>>> placed
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> >> these
> >>>> cases, the
> >>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> >> already
> >>>> in place
> >>>>>>>> and
> >>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data
> structure
> >> to
> >>>> the VM in
> >>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
> >> not
> >>>>>>>>>> pre-allocated.
> >>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
> >> be a
> >>>> SAN
> >>>>> (not
> >>>>>>>> a
> >>>>>>>>>>>>>>>>> volume on
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
> >>>> structure
> >>>>>>>> must be
> >>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
> >> these
> >>>> types of
> >>>>>>>> volumes
> >>>>>>>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> >>>> structure by an
> >>>>>>>>>> admin.
> >>>>>>>>>>>>>>>>> Once
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on
> XenServer
> >> for
> >>>> this
> >>>>>>>> volume,
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the
> VM
> >> in
> >>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
> >>>> John Burwell
> >>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation
> of
> >> the
> >>>> Dynamic
> >>>>> type
> >>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
> >> ask
> >>>> about the
> >>>>>>>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
> >>>> definition
> >>>>> of
> >>>>>>>> a
> >>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> >> Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI
> type
> >> is
> >>>>> uniquely
> >>>>>>>>>> used
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> >> Storage
> >>>> that is
> >>>>>>>> directly
> >>>>>>>>>>>>>>>>> based on
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> >> step of
> >>>> going
> >>>>> to
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
> >> that
> >>>> iSCSI
> >>>>>>>> target as
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> >>>> supported for
> >>>>>>>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first
> go
> >> to
> >>>> the
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD
> is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
> >> PM,
> >>>> John
> >>>>> Burwell
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
> >> what
> >>>> is the
> >>>>>>>>>> difference
> >>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
> >> RBD
> >>>> considered
> >>>>>>>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
> >> Mike
> >>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
> >>>> defined in
> >>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> >>>> StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
> >>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
> >> //
> >>>> NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
> >> LUN,
> >>>> with a
> >>>>>>>>>> clusterfs
> >>>>>>>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
> >>>> Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
> >> local
> >>>> LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
> >> VMFS
> >>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> >>>> XenServer, Storage
> >>>>>>>> Pool
> >>>>>>>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
> >> local
> >>>> EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
> >>>> zone-wide
> >>>>>>>> storage
> >>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
> >>>> shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 1:41
> >> PM,
> >>>> Mike
> >>>>>>>> Tutkowski
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> >> another
> >>>> storage
> >>>>>>>>>> company
> >>>>>>>>>>>>>>>>> wants
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> >> Quality
> >>>> of
> >>>>> Service
> >>>>>>>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as
> well.
> >>>> They would
> >>>>>>>> need
> >>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
> >>>> necessary
> >>>>>>>> changes to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 1:39
> >>>> PM, Mike
> >>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
> >>>> XenServer code
> >>>>> is
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
> >> logic is
> >>>> now aware
> >>>>>>>> of
> >>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> >>>> implementation of
> >>>>>>>> this
> >>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> >> with
> >>>> Edison's
> >>>>>>>> storage
> >>>>>>>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
> >>>> framework that
> >>>>>>>>>> supported
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
> >>>> However, when I
> >>>>>>>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
> >> it
> >>>> was not
> >>>>>>>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack
> could
> >> do
> >>>> with these
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
> >> this
> >>>> new type of
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data
> structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 1:28
> >>>> PM, John
> >>>>>>>> Burwell
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to
> me
> >>>> that any
> >>>>>>>>>> operation
> >>>>>>>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> >> about
> >>>> details. I
> >>>>>>>> expect
> >>>>>>>>>>>>>>>>> to see
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> >> operations
> >>>> that can be
> >>>>>>>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> >>>> allocate
> >>>>> space/create
> >>>>>>>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> >>>> dependency, we are
> >>>>>>>> going
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will
> make
> >> the
> >>>> system
> >>>>>>>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> >>>> understanding that the
> >>>>>>>> Xen
> >>>>>>>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> >>>> CitrixResourceBase
> >>>>> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> >> 3:12
> >>>> PM, Mike
> >>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
> >> bit
> >>>> more
> >>>>>>>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
> >> from
> >>>> one setup
> >>>>>>>>>>>> method
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> >> volume on
> >>>> the
> >>>>> storage
> >>>>>>>>>>>>>>>>> system,
> >>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> >> structure
> >>>> to make
> >>>>> use
> >>>>>>>> of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> >> XenServer
> >>>> or a
> >>>>> datastore
> >>>>>>>> on
> >>>>>>>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
> >>>> system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> >> storage
> >>>> framework,
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> >> create
> >>>> a 1:1
> >>>>>>>>>>>> relationship
> >>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
> >> data
> >>>> disk
> >>>>>>>> (necessary
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
> >> plug-
> >>>> in that is
> >>>>>>>> called
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
> >>>> storage
> >>>>> framework
> >>>>>>>> did
> >>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> >> hypervisor-
> >>>> specific data
> >>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
> >> I've
> >>>> been
> >>>>>>>> enhancing
> >>>>>>>>>> it
> >>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> >>>> submitted. I've got
> >>>>>>>> ESX(i)
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
> >>>> freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a
> bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> >>>> 1:03 PM, Mike
> >>>>>>>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in
> -
> >> by
> >>>> itself - is
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
> >>>> volume-attach
> >>>>>>>> logic
> >>>>>>>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
> >> the
> >>>> plug-in to
> >>>>>>>> have
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> >> comes to
> >>>> attach the
> >>>>>>>> volume
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> >> enough
> >>>> to recognize
> >>>>>>>> it's
> >>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
> >> has
> >>>> just been
> >>>>>>>> created)
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> >>>> XenServer) or a
> >>>>>>>> datastore
> >>>>>>>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was
> just
> >>>> created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
> >> most
> >>>> of my time
> >>>>>>>>>>>> recently
> >>>>>>>>>>>>>>>>> making
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> >> at
> >>>> 12:48 PM,
> >>>>> John
> >>>>>>>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> >> why
> >>>> the the storage
> >>>>>>>>>>>> driver
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> >> 1:21
> >>>> PM, Mike
> >>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
> >>>> would like to
> >>>>>>>> support
> >>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
> >> I'm
> >>>> just out of
> >>>>>>>> time
> >>>>>>>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> >> plug-in
> >>>> supports
> >>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> >> feature
> >>>> freeze, I
> >>>>>>>> have
> >>>>>>>>>> it
> >>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
> >> clear,
> >>>> this is
> >>>>> all
> >>>>>>>> in
> >>>>>>>>>>>>>>>>> regards
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> Compute
> >>>> Offerings post
> >>>>> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>> at 11:14 AM,
> >>>>>>>> Kelcey
> >>>>>>>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any
> plan
> >> on
> >>>> supporting
> >>>>>>>> KVM in
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> >>>> Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> >>>> Tutkowski" <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> >>>> dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
> >> June
> >>>> 3, 2013
> >>>>>>>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> >> [MERGE]
> >>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> >> merging
> >>>> Wei's feature
> >>>>>>>>>> first,
> >>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
> >> for
> >>>> KVM only,
> >>>>>>>> then
> >>>>>>>>>> it
> >>>>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> >> 2013
> >>>> at 8:55 AM,
> >>>>> Wei
> >>>>>>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
> >> as
> >>>> no one works
> >>>>>>>> on
> >>>>>>>>>>>>>>>>> billing
> >>>>>>>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
> >>>> themselves. They
> >>>>>>>> can
> >>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering
> of
> >> a
> >>>> VMs and
> >>>>> volumes
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
> >>>> exact
> >>>>> limitation
> >>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
> >> for
> >>>> nics as
> >>>>> well.
> >>>>>>>> I
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
> >>>> throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> >>>> misunstand the second
> >>>>> the
> >>>>>>>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> >> what
> >>>> you said about
> >>>>> the
> >>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3,
> 2013,
> >> at
> >>>> 2:13 AM, Wei
> >>>>>>>> ZHOU
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope
> Mike's
> >>>> aswer helps you.
> >>>>>>>> I am
> >>>>>>>>>>>>>>>>> trying
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
> >> billing
> >>>> should
> >>>>>>>> depend
> >>>>>>>>>> on
> >>>>>>>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
> >> Please
> >>>> review
> >>>>>>>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
> >>>> statistics
> >>>>> including
> >>>>>>>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> >> machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
> >> AWS
> >>>> model,
> >>>>> customers
> >>>>>>>>>> are
> >>>>>>>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> >> IOPS,
> >>>> as well as,
> >>>>> for
> >>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://aws.amazon.com/ebs/).
> >>>>> I
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> >> employ
> >>>> similar cost
> >>>>>>>>>> models.
> >>>>>>>>>>>>>>>>> Could
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
> >> in
> >>>> the current
> >>>>>>>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
> >> mean
> >>>> IOPS runtime
> >>>>>>>>>> change?
> >>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation
> for
> >> a
> >>>> running
> >>>>> virtual
> >>>>>>>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >> does
> >>>> not support
> >>>>>>>>>> changing
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
> >>>> offering or disk
> >>>>>>>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at
> the
> >>>> Java interface
> >>>>>>>> level.
> >>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> >>>> generalize allocation
> >>>>>>>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
> >>>> service guarantees
> >>>>>>>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> >> good
> >>>> question.
> >>>>>>>> Maybe it
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> disk_io_throttling as Mike
> >>>>>>>> needs to
> >>>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
> >> on
> >>>> my thoughts
> >>>>> in a
> >>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
> >>>> between these two
> >>>>>>>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to
> be
> >>>> merged before
> >>>>>>>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
> >>>> between the
> >>>>> branches
> >>>>>>>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> disk_io_throttling)
> >>>> to
> >>>>>>>> coordinate on
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> >> John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things
> I
> >>>> want to
> >>>>> understand
> >>>>>>>>>> are
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
> >>>> value in
> >>>>> capturing
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
> model
> >> (e.g.
> >>>> for
> >>>>>>>> billing/usage
> >>>>>>>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
> >> there
> >>>> be a common
> >>>>>>>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> provisioning
> >> at
> >>>> runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
> >>>> conflicting
> >>>>>>>> provisioned
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >> and
> >>>> storage device
> >>>>>>>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
> >>>> believe (and
> >>>>> billed)
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >> device
> >>>> has been
> >>>>>>>> configured
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
> >>>> configuration
> >>>>>>>> between a
> >>>>>>>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
> >>>> later
> >>>>> modification
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
> >> 2013,
> >>>> at 8:38 PM,
> >>>>>>>> Mike
> >>>>>>>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
> >> Wei's
> >>>> feature deals
> >>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >> hypervisor
> >>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> feature
> >> is
> >>>> focused on
> >>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
> >>>> helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun,
> Jun
> >> 2,
> >>>> 2013 at 6:35
> >>>>> PM,
> >>>>>>>>>>>> John
> >>>>>>>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> >> opinion is
> >>>> that no
> >>>>> features
> >>>>>>>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> >> have
> >>>> been resolved
> >>>>> and
> >>>>>>>> it
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> ready
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
> >> vs
> >>>> discrete
> >>>>>>>> read/write
> >>>>>>>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
> >> don't
> >>>> think this
> >>>>>>>> criteria
> >>>>>>>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also,
> how
> >>>> does this work
> >>>>>>>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>>>>>>> )?
> >>>>>>>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> >> involves
> >>>> provisioned
> >>>>>>>> IOPS. I
> >>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>> where provisioned
> >>>>>>>> IOPS
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
> >> 2013,
> >>>> at 6:47 AM,
> >>>>>>>> Wei
> >>>>>>>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I
> will
> >>>> change it next
> >>>>>>>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
> >>>> Wido den Hollander
> >>>>> <
> >>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>> 06/01/2013 08:24 AM, Wei
> >>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
> >> have
> >>>> pushed the
> >>>>>>>>>>>> features
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
> >>>> object thems for
> >>>>>>>>>>>> technical
> >>>>>>>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> >> sake
> >>>> of clarity I
> >>>>> just
> >>>>>>>>>>>> want
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
> >> IOps
> >>>> to R/W IOps
> >>>>>>>> asap
> >>>>>>>>>> so
> >>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
> >> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
> >> the
> >>>> groundwork for
> >>>>>>>> the
> >>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
> >>>> prevent that we
> >>>>> create
> >>>>>>>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/5/31
> >>>> Wido den
> >>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>> 05/31/2013 03:59 PM, John
> >>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
> >>>> enhancement must
> >>>>> to
> >>>>>>>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >> how
> >>>> it could be
> >>>>>>>> fixed
> >>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
> >> IOPS
> >>>> into read
> >>>>> and
> >>>>>>>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>> unless/until we
> >>>>> decided
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What
> Wei
> >>>> meant was merging
> >>>>> it
> >>>>>>>>>> into
> >>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> branch
> >>>> and add Read /
> >>>>>>>> Write
> >>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> >>>> released with Read
> >>>>> and
> >>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
> >>>> make the May 31st
> >>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see
> other
> >>>> threads) then it
> >>>>>>>> won't
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> >>>> completely agree that
> >>>>>>>>>> there
> >>>>>>>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May
> 31,
> >>>> 2013, at 9:51 AM,
> >>>>>>>> Wido
> >>>>>>>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>> 05/31/2013 03:13 PM, Wei
> >>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> >> Good
> >>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
> >>>> about at the
> >>>>>>>>>> beginning.
> >>>>>>>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> difference
> >> of
> >>>> read and write
> >>>>>>>>>>>> mainly
> >>>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
> >>>> difference of sent
> >>>>>>>> and
> >>>>>>>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> >>>> reasoning seems odd.
> >>>>>>>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O
> is
> >>>> much more
> >>>>>>>> expensive
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> Implementing
> >>>> it will be some
> >>>>>>>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> implemented
> >>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
> >> For
> >>>> the deadline
> >>>>> of
> >>>>>>>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
> >>>> needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think
> it's
> >> a
> >>>> feature we
> >>>>>>>> can't
> >>>>>>>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >> we
> >>>> have to make sure
> >>>>> we
> >>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
> >> that
> >>>> will confuse
> >>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> 2013/5/31
> >>>> Wido den
> >>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> >>>> 05/30/2013 06:03 PM, Wei
> >>>>>>>> ZHOU
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
> >> like to
> >>>> merge
> >>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
> >>>> object, I will
> >>>>> merge
> >>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> purpose
> >>>> is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> >>>> machines are running
> >>>>>>>> on
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> >> strage).
> >>>> Because of
> >>>>> the
> >>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
> >> has
> >>>> large disk
> >>>>>>>> operation,
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> VMs
> >>>> running on the
> >>>>> same
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >> neccesary
> >>>> to set the
> >>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking
> at
> >> the
> >>>> code I see
> >>>>> you
> >>>>>>>>>> make
> >>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu
> and
> >>>> libvirt support
> >>>>>>>> setting
> >>>>>>>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >> which
> >>>> could benefit a
> >>>>>>>> lot of
> >>>>>>>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
> >>>> strange, in the
> >>>>>>>> polling
> >>>>>>>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
> >> on
> >>>> the throttling
> >>>>>>>> side
> >>>>>>>>>>>>>>>>> you only
> >>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> IOps
> >> are
> >>>> usually much
> >>>>>>>> more
> >>>>>>>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a
> valid
> >>>> use-case where
> >>>>>>>> that
> >>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
> >> Read
> >>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
> >> only
> >>>> supports KVM
> >>>>>>>> at
> >>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to
> at
> >>>> least have the
> >>>>>>>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
> >>>> would be a lot of
> >>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> >> hypervisor
> >>>> doesn't
> >>>>>>>> support
> >>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
> >> can
> >>>> always sum
> >>>>> both
> >>>>>>>> up
> >>>>>>>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> >>>> explain why you
> >>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >> feature
> >>>> includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> the
> >>>> maximum rate of
> >>>>>>>> VMs
> >>>>>>>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2)
> change
> >> the
> >>>> maximum rate
> >>>>> of
> >>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
> >> the
> >>>> disk rate
> >>>>> (total
> >>>>>>>>>> bps
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA
> ticket:
> >>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>> 1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>> 1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
> >>>> update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
> >> check
> >>>> list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
> >>>> check the branch's
> >>>>>>>> RAT
> >>>>>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> >> there
> >>>> new dependencies
> >>>>>>>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> >>>> automated testing
> >>>>> (unit
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
> >> are
> >>>> added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> >>>> testing has been done
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set
> the
> >>>> bytes rate and
> >>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
> >>>> operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
> >> stop,
> >>>> start, reboot,
> >>>>>>>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> >> Volume
> >>>> operations,
> >>>>>>>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> >>>> Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To
> review
> >> the
> >>>> code, you can
> >>>>>>>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> >> regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> VM+Disk+IO+Throttling<https://
> >>>>>>>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>
> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> >>>> rottling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>> 1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> >> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>> 1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> >> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>> 1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> >> 2071
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> >>>> 2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> >>>> 2071
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> >>>> 2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
> >> Disk
> >>>> I/O
> >>>>> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >>>> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >>>> CloudStack Developer,
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> >> 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> >> the
> >>>> way the world
> >>>>>>>> uses
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> >> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>> Developer,
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>> the world uses
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> >> CloudStack
> >>>> Developer,
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> >> way
> >>>> the world uses
> >>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >> the
> >>>> world uses the
> >>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer,
> >>>>> SolidFire
> >>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> >> the
> >>>> world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >>>> Developer, SolidFire
> >>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >>>> world uses the
> >>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>> SolidFire
> >>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>> uses the
> >>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> >> Developer,
> >>>> SolidFire
> >>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> >> world
> >>>> uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> >>>> SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >> uses
> >>>> the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>> SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >> the
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>
> >>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>>>>
> >>>>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *(tm)*
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *(tm)*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *(tm)*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *(tm)*
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *(tm)*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
On Jun 7, 2013, at 6:13 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> Sent: Friday, June 07, 2013 2:37 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>> As we only have three weeks until feature freeze, we should come to a
>> consensus on this design point as soon as possible.
>>
>> Right now, if the storage framework asks my driver it is is managed, it
>> will say 'yes.' This means the framework will tell the driver to perform
>> its management activities. This then means the driver will call into the
>> host (it doesn't know which hypervisor, by the way) to perform the activity
>> of, say, creating an SR on XenServer or a datastore on ESX.
>>
>> The driver doesn't know which hypervisor it's talking to, it just sends a
>> message to the host to perform the necessary pre-attach work.
>
> Could we just expose a method like "attachVolume/dettachVolume" on the PrimaryDataStoreDriver driver?
> For most of cases, the implementation of each driver would just send attachvolumecommand/dettachvolumecmd to hypervisor(we can put the implementation in a base class, so that can be shared by all of this drivers), and in each hypervisor resource code, which may just call hypervisor's api to attach the volume to VM, while for certain storage, like SolidFire, may need to create a SR first, and create volume on it, then call hypervisor's API to attach volume to VM.
> While for some other storage vendor, may want to bypass hypervisor during attaching volume, so inside driver's attachvolume implementation, the driver can do some magic, such as, directly talk to an agent inside VM instance, then create a disk inside VM.
>
> How do you guys think?
This behavior is the opposite of my expectations. I would expect the VirtualMachineManager to coordinate this process. Why would Storage need to know or care about a Hypervisor? My understanding is that Xen needs to create an SR in a particular manner on all ISCSI devices upon allocation. On attach, the Xen plugin would query the datastore to determine the presence of an SR, and create it as necessary. It would proceed with the creating the volume, etc. In the "real world", SANs have no knowledge about the concept of a hypervisor let alone specific implementations. I think we would be wise to replicate that model in CloudStack.
>
>
>>
>>
>>
>>
>> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>>>> Sent: Friday, June 07, 2013 1:14 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>
>>>> Hi John,
>>>>
>>>> How's about this:
>>>>
>>>> The driver can implement an isManaged() method. The
>> VolumeManagerImpl
>>>> can
>>>> call into the driver to see if its managed. If it is, the
>>> VolumeManagerImpl
>>>> (which is responsible for calling into the hypervisor to attach the disk)
>>>> can call into the hypervisor to create the necessary hypervisor data
>>>> structure (ex. for XenServer, a storage repository).
>>>
>>> The problem here is that storage vendor may work differently with
>>> hypervisor, for example, SolidFire wants a SR per LUN, while maybe other
>>> vendor wants to totally bypass hypervisor, and assign the LUN directly to
>>> VM instance, see the discuss(
>>> http://mail-archives.apache.org/mod_mbox/cloudstack-
>> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
>> %3E
>>> ).
>>> So I would let storage provider to implement attach disk to VM, instead of
>>> implemented by cloudstack itself.
>>>
>>>
>>>>
>>>> If that's what you're going for, that works for me. By the way, Edison's
>>>> default storage plug-in (which handles the default storage behavior in
>>>> CloudStack (ex. how pre 4.2 works)) does include code that talks to
>>>> hypervisors. You might want to contact him and inform him of your
>>> concerns
>>>> or that logic (as is) will make it to production.
>>>>
>>>> Please let me know if what I wrote in above (for my solution) is OK with
>>>> you. :)
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> Please see my responses in-line below.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>
>>>>>> Hey John,
>>>>>>
>>>>>> I still have a bit more testing I'd like to do before I build up a
>>> patch
>>>>>> file, but this is the gist of what I've done:
>>>>>>
>>>>>> * During a volume-attach operation, after VolumeManagerImpl tells
>>>>>> VolumeServiceImpl to have the driver create a volume, I have
>>>>>> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
>>>> managed.
>>>>>> If it is managed, VolumeServiceImpl has the driver perform whatever
>>>>>> activity is required. In my case, this includes sending a message to
>>> the
>>>>>> host where the VM is running to have, say XenServer, add a storage
>>>>>> repository (based on the IP address of the SAN, the IQN of the SAN
>>>>> volume,
>>>>>> etc.) and a single VDI (the VDI consumes all of the space it can on
>>> the
>>>>>> storage repository). After this, the normal attach-volume message is
>>> sent
>>>>>> to the host by VolumeManagerImpl.
>>>>>
>>>>> There should be **no** code from a storage driver to a hypervisor. I
>>>>> apologize for the repetition, but we simply can not have hypervisor
>>>>> specific code in the storage layer. The circular dependencies between
>>> the
>>>>> two layers are not sustainable in the long term. Either the
>>> VirtualManager
>>>>> or Xen hypervisor plugin needs to be refactored/modified to
>> coordinate
>>>>> volume creation and then populating the SR. Ideally, we can
>>> generalize the
>>>>> process flow for attaching volumes such that the Xen hypervisor plugin
>>>>> would only implement callbacks to perform the attach action and create
>>>> the
>>>>> structure and SR. To my mind, the SolidFire driver should only be
>>>>> allocating space and providing information about contents (e.g. space
>>>>> available, space consumed, streams to a URI, file handle for a URI,
>>> etc)
>>>>> and capabilities.
>>>>>
>>>>>
>>>>>>
>>>>>> * The reverse is performed for a detach-volume command.
>>>>>>
>>>>>> * Right now I simply "return true;" for isManaged() in my driver.
>>>>> Edison's
>>>>>> default driver simply does a "return false;". We could add a new
>>>>> parameter
>>>>>> to the createStoragePool API command, if we want, to remove the
>>>>> hard-coded
>>>>>> return values in the drivers (although my driver will probably just
>>>>> ignore
>>>>>> this parameter and always return true since it wouldn't make sense
>>> for it
>>>>>> to ever return false). We'd need another column in the storage_pool
>>>> table
>>>>>> to store this value.
>>>>>
>>>>> Yes, I think we should have a parameter added to the
>> createStoragePool
>>>>> surfaced to the HTTP API that allows DataStores to be configured for
>>>>> management when their underlying drivers support it. To simplify
>>> things,
>>>>> this flag should only be mutable when the DataStore is created. It
>>> would be
>>>>> a bit crazy to take a DataStore from managed to unmanaged after
>>> creation.
>>>>>
>>>>>>
>>>>>> Sound like I'm in sync with what you were thinking?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com
>>>>>>> wrote:
>>>>>>
>>>>>>> I agree, John. Just wanted to point out that I have a working GUI
>>> for
>>>>> you
>>>>>>> to review (in that document), if you'd like to check it out.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> I would like the UIs of two features reviewed together to ensure
>>>>>>>> consistency across the concepts of hypervisor throttled IOPs and
>>>>>>>> storage device provisioned IOPs. I see the potential for
>>> confusion,
>>>>>>>> and I think a side-by-side Ui review of these features will help
>>>>>>>> minimize any potential confusion.
>>>>>>>>
>>>>>>>> As I mentioned, the term reconciliation issue will work itself if
>>> it
>>>>>>>> is acceptable that a VM is only permitted utilize hypervisor
>>> throttled
>>>>>>>> IOPs or storage provisioned IOPs.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>>>>>>> <mi...@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Hi John,
>>>>>>>>>
>>>>>>>>> Yeah, when you get a chance, refer to the Google doc I sent to
>> you
>>>> the
>>>>>>>>> other day to see how the GUI looks for provisioned storage IOPS.
>>>>>>>>>
>>>>>>>>> Several months ago, I put this topic out on the e-mail list and we
>>>>>>>> decided
>>>>>>>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
>>> dialog.
>>>>>>>>> Other storage vendors are coming out with QoS, so they should
>> be
>>>> able
>>>>> to
>>>>>>>>> leverage this GUI going forward (even if they, say, only use Max
>>>>> IOPS).
>>>>>>>>> These fields are optional and can be ignored for storage that
>>> does not
>>>>>>>>> support provisioned IOPS. Just like the Disk Size field, the
>>> admin can
>>>>>>>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
>>>>>>>>>
>>>>>>>>> I'm OK if we do an either/or model (either Wei's feature or mine,
>>> as
>>>>> is
>>>>>>>>> decided by the admin).
>>>>>>>>>
>>>>>>>>> I'm not sure what we can do about these two features expressing
>>> the
>>>>>>>> speed
>>>>>>>>> in different terms. I've never seen a SAN express the IOPS for
>>> QoS in
>>>>>>>> any
>>>>>>>>> way other than total IOPS (i.e. not broken in into read/write
>>> IOPS).
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
>> <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Wei,
>>>>>>>>>>
>>>>>>>>>> We have been down the rabbit hole a bit on the
>> Storage/Hypervisor
>>>>> layer
>>>>>>>>>> separation, but we still need to reconcile the behavior of
>>> hypervisor
>>>>>>>>>> throttled I/O and storage provisioned IOPS. I see the following
>>>>> issues
>>>>>>>>>> outstanding:
>>>>>>>>>>
>>>>>>>>>> 1. Hypervisor throttled IOPS are expressed as discrete
>> read/write
>>>>>>>> values
>>>>>>>>>> where as storage provisioned IOPS are expressed as total IOPS.
>>>>>>>>>> 2. How do we handle VMs with throttled IOPS attached to
>> storage
>>>>> volumes
>>>>>>>>>> with provisioned IOPS?
>>>>>>>>>> 3. How should usage data be captured for throttled and
>>> provisioned
>>>>> IOPS
>>>>>>>>>> that will permit providers to discriminate these guaranteed
>>>>> operations
>>>>>>>> in
>>>>>>>>>> the event they want to bill for it differently?
>>>>>>>>>> 4. What is the user experience for throttled and provisioned
>> IOPS
>>>>> that
>>>>>>>>>> minimizes confusion of these concepts?
>>>>>>>>>>
>>>>>>>>>> My thinking is that a VM can have either utilize hypervisor
>>> throttled
>>>>>>>> IOPS
>>>>>>>>>> or storage provisioned IOPS. This policy would neatly solve
>>> items 1
>>>>>>>> and 2.
>>>>>>>>>> Since the two facilities would not be permitted to operate
>>> together,
>>>>>>>> they
>>>>>>>>>> do not need to be semantically compatible. I think item 3 can be
>>>>>>>> resolved
>>>>>>>>>> with an additional flag or two on the usage records. As for
>>> Item 4,
>>>>> I
>>>>>>>> am
>>>>>>>>>> not familiar with how these two enhancements are depicted in
>> the
>>>> user
>>>>>>>>>> interface. I think we need to review the UI enhancements for
>>> both
>>>>>>>>>> enhancements and ensure they are consistent.
>>>>>>>>>>
>>>>>>>>>> Do these solutions make sense?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com>
>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> John and Mike,
>>>>>>>>>>>
>>>>>>>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
>>>>>>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
>> CLOUDSTACK-
>>>> 2875 ) this
>>>>>>>> week.
>>>>>>>>>>> I will start to develop on iops/bps changes tomorrow, and ask
>>> for
>>>>>>>> second
>>>>>>>>>>> merge request after testing.
>>>>>>>>>>>
>>>>>>>>>>> -Wei
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>>>>>>>>>>>
>>>>>>>>>>>> I believe I understand where you're going with this, John, and
>>>> have
>>>>>>>> been
>>>>>>>>>>>> re-working this section of code today.
>>>>>>>>>>>>
>>>>>>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the comments,
>>>>>>>>>>>> Mike
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
>>>> <jb...@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> See my responses in-line below.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>
>>>>>>>>>>>>> "I am fine with the VolumeManager determining whether or
>>>> not a
>>>>>>>> Volume
>>>>>>>>>> is
>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>> actual
>>>>>>>> isManaged
>>>>>>>>>>>>> method), and asking the device driver to allocate resources
>>> for
>>>>> the
>>>>>>>>>>>> volume
>>>>>>>>>>>>> if it is managed."
>>>>>>>>>>>>>
>>>>>>>>>>>>> Are you thinking you'd like to see an isManaged() method
>>>> added to
>>>>>>>> the
>>>>>>>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then the
>>>>>>>> storage
>>>>>>>>>>>>> framework could call the manage() (or whatever name)
>> method
>>>> (which
>>>>>>>>>> would
>>>>>>>>>>>> be
>>>>>>>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
>> this
>>>>> would
>>>>>>>>>> call
>>>>>>>>>>>>> into a new method in the hypervisor code to create, say on
>>>>>>>> XenServer,
>>>>>>>>>> an
>>>>>>>>>>>> SR?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would like to see canBeManaged() : boolean on
>>>> DataStoreDriver.
>>>>>>>> Since
>>>>>>>>>>>>> the notion of Volumes only pertains to primary storage, I
>>> would
>>>>> add
>>>>>>>> a
>>>>>>>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
>> man
>>>> term
>>>>>>>> --
>>>>>>>>>>>>> something other than volume) methods to
>>>>>>>>>> allocate/create/deallocate/delete
>>>>>>>>>>>>> underlying storage. To my mind, managed is a mutable
>> property
>>>> of
>>>>>>>>>>>> DataStore
>>>>>>>>>>>>> which can be enabled if/when the underlying
>> DataStoreDriver
>>>> can be
>>>>>>>>>>>> managed.
>>>>>>>>>>>>> This approach allows operators to override manageability of
>>>>> devices.
>>>>>>>>>>>>>
>>>>>>>>>>>>> In terms of orchestration/process flow for SR, the Xen plugin
>>>>> would
>>>>>>>> be
>>>>>>>>>>>>> responsible for composing DataStore/Volume methods to
>>>> create any
>>>>>>>>>>>>> directories or files necessary for the SR. There should be no
>>>>>>>>>>>> dependencies
>>>>>>>>>>>>> from the Storage to the Hypervisor layer. As I said earlier,
>>> such
>>>>>>>>>>>> circular
>>>>>>>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Just want to make sure I'm on the same page with you.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks again, John
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
>>>> <jb...@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Fundamentally, we can't end up with a Storage layer that
>>>>> supports n
>>>>>>>>>>>>>> devices types with each specific behaviors of m hypervisors.
>>>>> Such
>>>>>>>> a
>>>>>>>>>>>>>> scenario will create an unmaintainable and untestable
>> beast.
>>>>>>>>>>>> Therefore, my
>>>>>>>>>>>>>> thoughts and recommendations are driven to evolve the
>>>> Storage
>>>>> layer
>>>>>>>>>>>> towards
>>>>>>>>>>>>>> this separation.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am fine with the VolumeManager determining whether
>> or
>>>> not a
>>>>>>>> Volume
>>>>>>>>>> is
>>>>>>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
>> actual
>>>>>>>>>> isManaged
>>>>>>>>>>>>>> method), and asking the device driver to allocate resources
>>> for
>>>>> the
>>>>>>>>>>>> volume
>>>>>>>>>>>>>> if it is managed. Furthermore, the device driver needs to
>>>>> indicate
>>>>>>>>>>>> whether
>>>>>>>>>>>>>> or not it supports management operations. Finally, I think
>>> we
>>>>>>>> need to
>>>>>>>>>>>>>> provide the ability for an administrator to elect to have
>>>>> something
>>>>>>>>>>>> that is
>>>>>>>>>>>>>> manageable be unmanaged (i.e. the driver is capable
>> managing
>>>> the
>>>>>>>>>> device,
>>>>>>>>>>>>>> but the administrator has elected to leave it unmanaged).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Creation of a structure on the volume should be done in
>> the
>>>> Xen
>>>>>>>>>>>>>> hypervisor module using methods exposed by the Storage
>>>> layer to
>>>>>>>>>> perform
>>>>>>>>>>>>>> low-level operations (e.g. make directories, create a file,
>>> etc).
>>>>>>>>>> This
>>>>>>>>>>>>>> structure is specific to the operation of the Xen
>>> hypervisor, as
>>>>>>>> such,
>>>>>>>>>>>>>> should be confined to its implementation. From my
>>>> perspective,
>>>>>>>>>> nothing
>>>>>>>>>>>> in
>>>>>>>>>>>>>> the Storage layer should be concerned with content. From
>> its
>>>>>>>>>>>> perspective,
>>>>>>>>>>>>>> structure and data are opaque. It provides the means to
>>> query
>>>>> the
>>>>>>>>>> data
>>>>>>>>>>>> to
>>>>>>>>>>>>>> support the interpretation of the content by higher-level
>>>> layers
>>>>>>>> (e.g.
>>>>>>>>>>>>>> Hypervisors). To my mind, attach should be a composition
>> of
>>>>>>>>>> operations
>>>>>>>>>>>>>> from the Storage layer that varies based on the Volume
>>>> storage
>>>>>>>>>> protocol
>>>>>>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alternatively to the way the attach logic is implemented in
>>> my
>>>>>>>> patch,
>>>>>>>>>> we
>>>>>>>>>>>>>> could do the following:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Leave the attach logic in the agent code alone. In
>>>>>>>> VolumeManagerImpl
>>>>>>>>>> we
>>>>>>>>>>>>>> create an AttachVolumeCommand and send it to the
>>>> hypervisor.
>>>>> Before
>>>>>>>>>> this
>>>>>>>>>>>>>> command is sent, we could check to see if we're dealing
>> with
>>>>>>>> Dynamic
>>>>>>>>>> (or
>>>>>>>>>>>>>> whatever we want to call it) storage and - if so - send a
>>> "Create
>>>>>>>> SR"
>>>>>>>>>>>>>> command to the hypervisor. If this returns OK, we would
>> then
>>>>>>>> proceed
>>>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This way the attach logic remains the same and we just add
>>>>> another
>>>>>>>>>>>>>> command to the agent code that is called for this particular
>>>> type
>>>>>>>> of
>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I created a document for a customer today that outlines
>> how
>>>> the
>>>>>>>>>> plug-in
>>>>>>>>>>>>>>> works from a user standpoint. This will probably be of use
>>> to
>>>>>>>> you, as
>>>>>>>>>>>> well,
>>>>>>>>>>>>>>> as you perform the code review.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have shared this document with you (you should have
>>>> received
>>>>>>>> that
>>>>>>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Talk to you later!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
>>>>> jburwell@basho.com
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
>> but
>>>>> want
>>>>>>>> to
>>>>>>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
>>>>>>>>>>>> disk_io_throttle,
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> object_store branches. I hope to collect them in a
>>>> coherent
>>>>>>>> form
>>>>>>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
>>> notion
>>>>> of
>>>>>>>> a
>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the
>>>> notion
>>>>>>>> of
>>>>>>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The storage framework that leverages the plug-in was
>>>>>>>> incomplete,
>>>>>>>>>> so
>>>>>>>>>>>>>>>>> I had
>>>>>>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I appreciate all the time you've been spending on this.
>>> :)
>>>> Do
>>>>>>>> you
>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>> recommendation as to how we should accomplish
>> what
>>>> you're
>>>>>>>> looking
>>>>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
>>> notion
>>>>> of
>>>>>>>> a
>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I want to see a separation of the management
>>>> capabilities
>>>>>>>> (i.e.
>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> device be managed/does an operator want it
>> managed
>>>> by
>>>>>>>> CloudStack)
>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
>>>> semantic
>>>>>>>> that
>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>> allow any type of storage device to be managed. I
>> also
>>>> want
>>>>>>>> to
>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> progress on decoupling the storage types from the
>>>> hypervisor
>>>>>>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Could you please answer the following questions
>> for
>>>> me
>>>>> with
>>>>>>>>>>>>>>>>> regards to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
>> SAN
>>>>> volumes?
>>>>>>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN
>> 0).
>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and
>>>> CloudStack
>>>>>>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Are the LUN(s) created by the management
>> server or
>>>>>>>> externally
>>>>>>>>>> by
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
>> volume
>>>> (same
>>>>> as a
>>>>>>>>>> SAN
>>>>>>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>>>>>>> created by the management server (via the plug-in)
>>>> the
>>>>> first
>>>>>>>>>> time
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
>>> still
>>>>>>>> want
>>>>>>>>>> to
>>>>>>>>>>>>>>>>> use a
>>>>>>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
>> today
>>>>> (prior
>>>>>>>> to
>>>>>>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
>> this
>>>> case -
>>>>>>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> data disks can share this SAN volume. While you
>> can do
>>>> this
>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Are the SAN volumes by the management server
>> or
>>>> externally
>>>>>>>> by
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
>> volumes
>>>> are
>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>> by the management server (via the plug-in). There
>> is
>>>> no
>>>>> admin
>>>>>>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
>> volume
>>>> and a
>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>> which is necessary for any storage vendor that
>>>> supports
>>>>> true
>>>>>>>>>> QoS.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I would like to clarify how these pieces are related
>>>> and
>>>>>>>>>>>> expected
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
>> VMs
>>>> with
>>>>>>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
>> single
>>>> SAN
>>>>>>>> volume
>>>>>>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
>> multiple
>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
>> QoS.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
>> house
>>>> more
>>>>>>>> than
>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
>> without
>>>> any of
>>>>> my
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
>> for
>>>>>>>>>> CloudStack's
>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me
>> --
>>>>> concepts
>>>>>>>>>> from
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
>>> only
>>>>> used
>>>>>>>>>> (as
>>>>>>>>>>>>>>>>> far as
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not
>>>> PreSetup an
>>>>>>>> SR).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
>>>> VMware, it
>>>>>>>> uses
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
>> KVM,
>>>> it uses
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
>> here
>>>> was to
>>>>>>>> make a
>>>>>>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
>> Edison
>>>> added
>>>>>>>> into
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
>> specificy,
>>>> though:
>>>>>>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
>> Tutkowski
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
>> or
>>>> care
>>>>> that
>>>>>>>> it
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
>>>> questions
>>>>>>>> about
>>>>>>>>>> it
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
>>>> SolidFire
>>>>>>>> plug-in
>>>>>>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
>> hypervisor
>>>> logic
>>>>>>>>>> knows
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint
>>>> and just
>>>>>>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
>> the
>>>> hypervisor
>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
>> the
>>>> necessary
>>>>>>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
>> Burwell <
>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to
>>>> manage a
>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
>> CloudStack
>>>> (e.g. the
>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
>> chosen
>>>> to
>>>>>>>> dedicate
>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
>> make
>>>> sense?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
>>>> volume via
>>>>>>>> the
>>>>>>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
>>>> work unless
>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
>> SR.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
>>>> CloudStack, an
>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
>> exposes it
>>>>> through
>>>>>>>> a
>>>>>>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
>>>> CloudStack,
>>>>>>>> but
>>>>>>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
>>>> changes, we
>>>>>>>> would
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
>>>> necessary
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have
>> the
>>>> storage
>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
>> from
>>>> the
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
>> create
>>>> the
>>>>>>>> SR
>>>>>>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin
>>>> should
>>>>> have a
>>>>>>>>>>>> rich
>>>>>>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
>> available
>>>> for
>>>>> volume
>>>>>>>>>>>>>>>>> storage.
>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>>>>>>> this interface would allow the
>> interrogation of
>>>> the
>>>>>>>>>> device
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
>>>> shouldn't
>>>>>>>> know
>>>>>>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able
>> to
>>>>> answer
>>>>>>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
>> driver
>>>> (which
>>>>>>>> is a
>>>>>>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
>> new
>>>> Primary
>>>>>>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
>> that
>>>>> references
>>>>>>>> the
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
>>>> volume. This
>>>>>>>> leads
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack
>>>> volume to a
>>>>>>>> VM
>>>>>>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
>> the
>>>> plug-in
>>>>>>>> to
>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
>> SAN).
>>>> The
>>>>>>>> plug-in
>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data
>>>> (like the
>>>>>>>> IQN of
>>>>>>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
>>>>> CloudStack
>>>>>>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
>>>> volume
>>>>>>>> will be
>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the
>>>> plug-in
>>>>>>>> code
>>>>>>>>>>>>>>>>> will not
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and
>>>> detects
>>>>> the
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
>> of a
>>>>>>>> hypervisor
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
>>>> terminology,
>>>>>>>> volume,
>>>>>>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
>>>> allocation on
>>>>>>>> the
>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
>> mean
>>>> 'SAN
>>>>>>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
>> can
>>>> make in
>>>>>>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
>> make
>>>> sense, but
>>>>>>>> I'm
>>>>>>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on
>>>> this
>>>>>>>> conversation
>>>>>>>>>>>>>>>>> as he
>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
>> approach I
>>>> took.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
>> John
>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
>>>> distinct
>>>>>>>> concepts
>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols.
>> In
>>>> both
>>>>>>>> cases, we
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
>>>>> create/delete
>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the
>>>> volume to
>>>>> be
>>>>>>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
>> To
>>>> ensure
>>>>>>>> that
>>>>>>>>>> we
>>>>>>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
>>>> definitions,
>>>>>>>> refers to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
>>>> Minimally,
>>>>> we
>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device
>> to
>>>> move bits
>>>>>>>> from
>>>>>>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
>> Optionally, a
>>>>>>>> storage
>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore,
>>>> we can have
>>>>> a
>>>>>>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
>> we
>>>> can have a
>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
>> capable
>>>> of
>>>>>>>> allocating
>>>>>>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
>> may
>>>> be able
>>>>>>>> to
>>>>>>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
>>>> unmanaged by
>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
>>>> operator has
>>>>>>>> chosen
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
>> my
>>>>> reasoning
>>>>>>>>>> make
>>>>>>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
>>>> reasonable, it
>>>>>>>> seems
>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
>> from
>>>>>>>> StoragePoolType,
>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
>> driver
>>>> that
>>>>>>>>>> indicates
>>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
>> establish
>>>> a
>>>>>>>> abstraction
>>>>>>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that
>> is
>>>>>>>> associated
>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
>>>> drivers can
>>>>>>>>>>>> declare
>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
>> a
>>>> device
>>>>> managed
>>>>>>>> by
>>>>>>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
>> by
>>>> the driver to
>>>>>>>>>>>> automate
>>>>>>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
>> sense,
>>>> then we can
>>>>>>>> sit
>>>>>>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
>> the
>>>> current
>>>>> iSCSI
>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
>> and
>>>> create
>>>>> a
>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time,
>>>> the
>>>>> iSCSI
>>>>>>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated).
>>>> Future
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created
>>>> behind the
>>>>>>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
>> go in
>>>> and
>>>>> create
>>>>>>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete
>>>> volumes on
>>>>>>>> its
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
>>>> storage
>>>>>>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
>> was
>>>> created,
>>>>> the
>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
>> After
>>>> this is
>>>>>>>> done,
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called.
>>>> No
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
>> was
>>>> just
>>>>>>>> created.
>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
>> Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should
>>>> probably
>>>>> have
>>>>>>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
>> pre-
>>>> allocated
>>>>>>>> (ex.
>>>>>>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
>> hypervisor
>>>> data
>>>>>>>> structure
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
>> then
>>>> that
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
>> (VDI)
>>>> are later
>>>>>>>>>>>> placed
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
>> these
>>>> cases, the
>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
>> already
>>>> in place
>>>>>>>> and
>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure
>> to
>>>> the VM in
>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
>> not
>>>>>>>>>> pre-allocated.
>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
>> be a
>>>> SAN
>>>>> (not
>>>>>>>> a
>>>>>>>>>>>>>>>>> volume on
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
>>>> structure
>>>>>>>> must be
>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
>> these
>>>> types of
>>>>>>>> volumes
>>>>>>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
>>>> structure by an
>>>>>>>>>> admin.
>>>>>>>>>>>>>>>>> Once
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer
>> for
>>>> this
>>>>>>>> volume,
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM
>> in
>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
>>>> John Burwell
>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of
>> the
>>>> Dynamic
>>>>> type
>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
>> ask
>>>> about the
>>>>>>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
>>>> definition
>>>>> of
>>>>>>>> a
>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
>> Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type
>> is
>>>>> uniquely
>>>>>>>>>> used
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
>> Storage
>>>> that is
>>>>>>>> directly
>>>>>>>>>>>>>>>>> based on
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
>> step of
>>>> going
>>>>> to
>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
>> that
>>>> iSCSI
>>>>>>>> target as
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
>>>> supported for
>>>>>>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go
>> to
>>>> the
>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
>> PM,
>>>> John
>>>>> Burwell
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
>> what
>>>> is the
>>>>>>>>>> difference
>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
>> RBD
>>>> considered
>>>>>>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
>> Mike
>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
>>>> defined in
>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
>>>> StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
>> //
>>>> NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
>> LUN,
>>>> with a
>>>>>>>>>> clusterfs
>>>>>>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
>>>> Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
>> local
>>>> LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
>> VMFS
>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
>>>> XenServer, Storage
>>>>>>>> Pool
>>>>>>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
>> local
>>>> EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
>>>> zone-wide
>>>>>>>> storage
>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
>>>> shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41
>> PM,
>>>> Mike
>>>>>>>> Tutkowski
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
>> another
>>>> storage
>>>>>>>>>> company
>>>>>>>>>>>>>>>>> wants
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
>> Quality
>>>> of
>>>>> Service
>>>>>>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well.
>>>> They would
>>>>>>>> need
>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
>>>> necessary
>>>>>>>> changes to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> 1:39
>>>> PM, Mike
>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
>>>> XenServer code
>>>>> is
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
>> logic is
>>>> now aware
>>>>>>>> of
>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
>>>> implementation of
>>>>>>>> this
>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
>> with
>>>> Edison's
>>>>>>>> storage
>>>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
>>>> framework that
>>>>>>>>>> supported
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
>>>> However, when I
>>>>>>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
>> it
>>>> was not
>>>>>>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could
>> do
>>>> with these
>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
>> this
>>>> new type of
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> 1:28
>>>> PM, John
>>>>>>>> Burwell
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me
>>>> that any
>>>>>>>>>> operation
>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
>> about
>>>> details. I
>>>>>>>> expect
>>>>>>>>>>>>>>>>> to see
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
>> operations
>>>> that can be
>>>>>>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
>>>> allocate
>>>>> space/create
>>>>>>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
>>>> dependency, we are
>>>>>>>> going
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make
>> the
>>>> system
>>>>>>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
>>>> understanding that the
>>>>>>>> Xen
>>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
>>>> CitrixResourceBase
>>>>> class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>> 3:12
>>>> PM, Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
>> bit
>>>> more
>>>>>>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
>> from
>>>> one setup
>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
>> volume on
>>>> the
>>>>> storage
>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
>> structure
>>>> to make
>>>>> use
>>>>>>>> of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
>> XenServer
>>>> or a
>>>>> datastore
>>>>>>>> on
>>>>>>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
>>>> system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
>> storage
>>>> framework,
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
>> create
>>>> a 1:1
>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
>> data
>>>> disk
>>>>>>>> (necessary
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
>> plug-
>>>> in that is
>>>>>>>> called
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
>>>> storage
>>>>> framework
>>>>>>>> did
>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
>> hypervisor-
>>>> specific data
>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
>> I've
>>>> been
>>>>>>>> enhancing
>>>>>>>>>> it
>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
>>>> submitted. I've got
>>>>>>>> ESX(i)
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
>>>> freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
>>>> 1:03 PM, Mike
>>>>>>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in -
>> by
>>>> itself - is
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
>>>> volume-attach
>>>>>>>> logic
>>>>>>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
>> the
>>>> plug-in to
>>>>>>>> have
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
>> comes to
>>>> attach the
>>>>>>>> volume
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
>> enough
>>>> to recognize
>>>>>>>> it's
>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
>> has
>>>> just been
>>>>>>>> created)
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
>>>> XenServer) or a
>>>>>>>> datastore
>>>>>>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just
>>>> created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
>> most
>>>> of my time
>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>> making
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
>> at
>>>> 12:48 PM,
>>>>> John
>>>>>>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
>> why
>>>> the the storage
>>>>>>>>>>>> driver
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
>> 1:21
>>>> PM, Mike
>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
>>>> would like to
>>>>>>>> support
>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
>> I'm
>>>> just out of
>>>>>>>> time
>>>>>>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
>> plug-in
>>>> supports
>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
>> feature
>>>> freeze, I
>>>>>>>> have
>>>>>>>>>> it
>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
>> clear,
>>>> this is
>>>>> all
>>>>>>>> in
>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute
>>>> Offerings post
>>>>> 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013
>>>> at 11:14 AM,
>>>>>>>> Kelcey
>>>>>>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan
>> on
>>>> supporting
>>>>>>>> KVM in
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
>>>> Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
>>>> Tutkowski" <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
>>>> dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
>> June
>>>> 3, 2013
>>>>>>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
>> [MERGE]
>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
>> merging
>>>> Wei's feature
>>>>>>>>>> first,
>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
>> for
>>>> KVM only,
>>>>>>>> then
>>>>>>>>>> it
>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
>> 2013
>>>> at 8:55 AM,
>>>>> Wei
>>>>>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
>> as
>>>> no one works
>>>>>>>> on
>>>>>>>>>>>>>>>>> billing
>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
>>>> themselves. They
>>>>>>>> can
>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of
>> a
>>>> VMs and
>>>>> volumes
>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
>>>> exact
>>>>> limitation
>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
>> for
>>>> nics as
>>>>> well.
>>>>>>>> I
>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
>>>> throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
>>>> misunstand the second
>>>>> the
>>>>>>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
>> what
>>>> you said about
>>>>> the
>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
>>>> Burwell <
>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
>> at
>>>> 2:13 AM, Wei
>>>>>>>> ZHOU
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's
>>>> aswer helps you.
>>>>>>>> I am
>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
>> billing
>>>> should
>>>>>>>> depend
>>>>>>>>>> on
>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
>> Please
>>>> review
>>>>>>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
>>>> statistics
>>>>> including
>>>>>>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
>> machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
>> AWS
>>>> model,
>>>>> customers
>>>>>>>>>> are
>>>>>>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
>> IOPS,
>>>> as well as,
>>>>> for
>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://aws.amazon.com/ebs/).
>>>>> I
>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
>> employ
>>>> similar cost
>>>>>>>>>> models.
>>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
>> in
>>>> the current
>>>>>>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
>> mean
>>>> IOPS runtime
>>>>>>>>>> change?
>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for
>> a
>>>> running
>>>>> virtual
>>>>>>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> does
>>>> not support
>>>>>>>>>> changing
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
>>>> offering or disk
>>>>>>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the
>>>> Java interface
>>>>>>>> level.
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
>>>> generalize allocation
>>>>>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
>>>> service guarantees
>>>>>>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
>> good
>>>> question.
>>>>>>>> Maybe it
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> disk_io_throttling as Mike
>>>>>>>> needs to
>>>>>>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
>> on
>>>> my thoughts
>>>>> in a
>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
>>>> between these two
>>>>>>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be
>>>> merged before
>>>>>>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
>>>> between the
>>>>> branches
>>>>>>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> disk_io_throttling)
>>>> to
>>>>>>>> coordinate on
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
>> John
>>>> Burwell <
>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I
>>>> want to
>>>>> understand
>>>>>>>>>> are
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
>>>> value in
>>>>> capturing
>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model
>> (e.g.
>>>> for
>>>>>>>> billing/usage
>>>>>>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
>> there
>>>> be a common
>>>>>>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning
>> at
>>>> runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
>>>> conflicting
>>>>>>>> provisioned
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> and
>>>> storage device
>>>>>>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
>>>> believe (and
>>>>> billed)
>>>>>>>>>> for
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> device
>>>> has been
>>>>>>>> configured
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
>>>> configuration
>>>>>>>> between a
>>>>>>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
>>>> later
>>>>> modification
>>>>>>>> to
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
>> 2013,
>>>> at 8:38 PM,
>>>>>>>> Mike
>>>>>>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
>> Wei's
>>>> feature deals
>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> hypervisor
>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature
>> is
>>>> focused on
>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
>>>> helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun
>> 2,
>>>> 2013 at 6:35
>>>>> PM,
>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
>> opinion is
>>>> that no
>>>>> features
>>>>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
>> have
>>>> been resolved
>>>>> and
>>>>>>>> it
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
>> vs
>>>> discrete
>>>>>>>> read/write
>>>>>>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
>> don't
>>>> think this
>>>>>>>> criteria
>>>>>>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how
>>>> does this work
>>>>>>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>> involves
>>>> provisioned
>>>>>>>> IOPS. I
>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>> where provisioned
>>>>>>>> IOPS
>>>>>>>>>> in
>>>>>>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
>> 2013,
>>>> at 6:47 AM,
>>>>>>>> Wei
>>>>>>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will
>>>> change it next
>>>>>>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
>>>> Wido den Hollander
>>>>> <
>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> 06/01/2013 08:24 AM, Wei
>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
>> have
>>>> pushed the
>>>>>>>>>>>> features
>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
>>>> object thems for
>>>>>>>>>>>> technical
>>>>>>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
>> sake
>>>> of clarity I
>>>>> just
>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
>> IOps
>>>> to R/W IOps
>>>>>>>> asap
>>>>>>>>>> so
>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
>> the
>>>> groundwork for
>>>>>>>> the
>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
>>>> prevent that we
>>>>> create
>>>>>>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
>>>> Wido den
>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> 05/31/2013 03:59 PM, John
>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
>>>> enhancement must
>>>>> to
>>>>>>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>> how
>>>> it could be
>>>>>>>> fixed
>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
>> IOPS
>>>> into read
>>>>> and
>>>>>>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>> unless/until we
>>>>> decided
>>>>>>>> to
>>>>>>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei
>>>> meant was merging
>>>>> it
>>>>>>>>>> into
>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch
>>>> and add Read /
>>>>>>>> Write
>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
>>>> released with Read
>>>>> and
>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
>>>> make the May 31st
>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other
>>>> threads) then it
>>>>>>>> won't
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
>>>> completely agree that
>>>>>>>>>> there
>>>>>>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31,
>>>> 2013, at 9:51 AM,
>>>>>>>> Wido
>>>>>>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> 05/31/2013 03:13 PM, Wei
>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>> Good
>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
>>>> about at the
>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
>> of
>>>> read and write
>>>>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
>>>> difference of sent
>>>>>>>> and
>>>>>>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
>>>> reasoning seems odd.
>>>>>>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is
>>>> much more
>>>>>>>> expensive
>>>>>>>>>> in
>>>>>>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> Implementing
>>>> it will be some
>>>>>>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> implemented
>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
>> For
>>>> the deadline
>>>>> of
>>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
>>>> needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's
>> a
>>>> feature we
>>>>>>>> can't
>>>>>>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>> we
>>>> have to make sure
>>>>> we
>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
>> that
>>>> will confuse
>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
>>>> Wido den
>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
>>>> 05/30/2013 06:03 PM, Wei
>>>>>>>> ZHOU
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
>> like to
>>>> merge
>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
>>>> object, I will
>>>>> merge
>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> purpose
>>>> is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
>>>> machines are running
>>>>>>>> on
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
>> strage).
>>>> Because of
>>>>> the
>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
>> has
>>>> large disk
>>>>>>>> operation,
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs
>>>> running on the
>>>>> same
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>> neccesary
>>>> to set the
>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at
>> the
>>>> code I see
>>>>> you
>>>>>>>>>> make
>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and
>>>> libvirt support
>>>>>>>> setting
>>>>>>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> which
>>>> could benefit a
>>>>>>>> lot of
>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
>>>> strange, in the
>>>>>>>> polling
>>>>>>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
>> on
>>>> the throttling
>>>>>>>> side
>>>>>>>>>>>>>>>>> you only
>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps
>> are
>>>> usually much
>>>>>>>> more
>>>>>>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid
>>>> use-case where
>>>>>>>> that
>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
>> Read
>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
>> only
>>>> supports KVM
>>>>>>>> at
>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at
>>>> least have the
>>>>>>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
>>>> would be a lot of
>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
>> hypervisor
>>>> doesn't
>>>>>>>> support
>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
>> can
>>>> always sum
>>>>> both
>>>>>>>> up
>>>>>>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
>>>> explain why you
>>>>>>>>>>>> implemented
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> feature
>>>> includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
>>>> maximum rate of
>>>>>>>> VMs
>>>>>>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change
>> the
>>>> maximum rate
>>>>> of
>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
>> the
>>>> disk rate
>>>>> (total
>>>>>>>>>> bps
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>> 1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>> 1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
>>>> update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
>> check
>>>> list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
>>>> check the branch's
>>>>>>>> RAT
>>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
>> there
>>>> new dependencies
>>>>>>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
>>>> automated testing
>>>>> (unit
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
>> are
>>>> added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
>>>> testing has been done
>>>>>>>> to
>>>>>>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
>>>> bytes rate and
>>>>>>>> IOPS
>>>>>>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
>>>> operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
>> stop,
>>>> start, reboot,
>>>>>>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
>> Volume
>>>> operations,
>>>>>>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
>>>> Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review
>> the
>>>> code, you can
>>>>>>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
>> regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> VM+Disk+IO+Throttling<https://
>>>>>>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
>>>> rottling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>> 1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>> 1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>> 1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
>> 2071
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
>>>> 2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
>>>> 2071
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
>>>> 2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
>> Disk
>>>> I/O
>>>>> Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>>>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>>>> CloudStack Developer,
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
>> 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
>> the
>>>> way the world
>>>>>>>> uses
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
>> Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>> Developer,
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way
>>>> the world uses
>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
>> CloudStack
>>>> Developer,
>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
>> way
>>>> the world uses
>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>> world uses the
>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer,
>>>>> SolidFire
>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
>> the
>>>> world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>>>> Developer, SolidFire
>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>>>> world uses the
>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>> SolidFire
>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>> uses the
>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
>> Developer,
>>>> SolidFire
>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
>> world
>>>> uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>>>> the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>
>>>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *(tm)*
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *(tm)*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *(tm)*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *(tm)*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *(tm)*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Friday, June 07, 2013 2:37 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> As we only have three weeks until feature freeze, we should come to a
> consensus on this design point as soon as possible.
>
> Right now, if the storage framework asks my driver it is is managed, it
> will say 'yes.' This means the framework will tell the driver to perform
> its management activities. This then means the driver will call into the
> host (it doesn't know which hypervisor, by the way) to perform the activity
> of, say, creating an SR on XenServer or a datastore on ESX.
>
> The driver doesn't know which hypervisor it's talking to, it just sends a
> message to the host to perform the necessary pre-attach work.
Could we just expose a method like "attachVolume/dettachVolume" on the PrimaryDataStoreDriver driver?
For most of cases, the implementation of each driver would just send attachvolumecommand/dettachvolumecmd to hypervisor(we can put the implementation in a base class, so that can be shared by all of this drivers), and in each hypervisor resource code, which may just call hypervisor's api to attach the volume to VM, while for certain storage, like SolidFire, may need to create a SR first, and create volume on it, then call hypervisor's API to attach volume to VM.
While for some other storage vendor, may want to bypass hypervisor during attaching volume, so inside driver's attachvolume implementation, the driver can do some magic, such as, directly talk to an agent inside VM instance, then create a disk inside VM.
How do you guys think?
>
>
>
>
> On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com> wrote:
>
> >
> >
> > > -----Original Message-----
> > > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > > Sent: Friday, June 07, 2013 1:14 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >
> > > Hi John,
> > >
> > > How's about this:
> > >
> > > The driver can implement an isManaged() method. The
> VolumeManagerImpl
> > > can
> > > call into the driver to see if its managed. If it is, the
> > VolumeManagerImpl
> > > (which is responsible for calling into the hypervisor to attach the disk)
> > > can call into the hypervisor to create the necessary hypervisor data
> > > structure (ex. for XenServer, a storage repository).
> >
> > The problem here is that storage vendor may work differently with
> > hypervisor, for example, SolidFire wants a SR per LUN, while maybe other
> > vendor wants to totally bypass hypervisor, and assign the LUN directly to
> > VM instance, see the discuss(
> > http://mail-archives.apache.org/mod_mbox/cloudstack-
> dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com
> %3E
> > ).
> > So I would let storage provider to implement attach disk to VM, instead of
> > implemented by cloudstack itself.
> >
> >
> > >
> > > If that's what you're going for, that works for me. By the way, Edison's
> > > default storage plug-in (which handles the default storage behavior in
> > > CloudStack (ex. how pre 4.2 works)) does include code that talks to
> > > hypervisors. You might want to contact him and inform him of your
> > concerns
> > > or that logic (as is) will make it to production.
> > >
> > > Please let me know if what I wrote in above (for my solution) is OK with
> > > you. :)
> > >
> > > Thanks!
> > >
> > >
> > > On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com>
> wrote:
> > >
> > > > Mike,
> > > >
> > > > Please see my responses in-line below.
> > > >
> > > > Thanks,
> > > > -John
> > > >
> > > > On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > > > wrote:
> > > >
> > > > > Hey John,
> > > > >
> > > > > I still have a bit more testing I'd like to do before I build up a
> > patch
> > > > > file, but this is the gist of what I've done:
> > > > >
> > > > > * During a volume-attach operation, after VolumeManagerImpl tells
> > > > > VolumeServiceImpl to have the driver create a volume, I have
> > > > > VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> > > managed.
> > > > > If it is managed, VolumeServiceImpl has the driver perform whatever
> > > > > activity is required. In my case, this includes sending a message to
> > the
> > > > > host where the VM is running to have, say XenServer, add a storage
> > > > > repository (based on the IP address of the SAN, the IQN of the SAN
> > > > volume,
> > > > > etc.) and a single VDI (the VDI consumes all of the space it can on
> > the
> > > > > storage repository). After this, the normal attach-volume message is
> > sent
> > > > > to the host by VolumeManagerImpl.
> > > >
> > > > There should be **no** code from a storage driver to a hypervisor. I
> > > > apologize for the repetition, but we simply can not have hypervisor
> > > > specific code in the storage layer. The circular dependencies between
> > the
> > > > two layers are not sustainable in the long term. Either the
> > VirtualManager
> > > > or Xen hypervisor plugin needs to be refactored/modified to
> coordinate
> > > > volume creation and then populating the SR. Ideally, we can
> > generalize the
> > > > process flow for attaching volumes such that the Xen hypervisor plugin
> > > > would only implement callbacks to perform the attach action and create
> > > the
> > > > structure and SR. To my mind, the SolidFire driver should only be
> > > > allocating space and providing information about contents (e.g. space
> > > > available, space consumed, streams to a URI, file handle for a URI,
> > etc)
> > > > and capabilities.
> > > >
> > > >
> > > > >
> > > > > * The reverse is performed for a detach-volume command.
> > > > >
> > > > > * Right now I simply "return true;" for isManaged() in my driver.
> > > > Edison's
> > > > > default driver simply does a "return false;". We could add a new
> > > > parameter
> > > > > to the createStoragePool API command, if we want, to remove the
> > > > hard-coded
> > > > > return values in the drivers (although my driver will probably just
> > > > ignore
> > > > > this parameter and always return true since it wouldn't make sense
> > for it
> > > > > to ever return false). We'd need another column in the storage_pool
> > > table
> > > > > to store this value.
> > > >
> > > > Yes, I think we should have a parameter added to the
> createStoragePool
> > > > surfaced to the HTTP API that allows DataStores to be configured for
> > > > management when their underlying drivers support it. To simplify
> > things,
> > > > this flag should only be mutable when the DataStore is created. It
> > would be
> > > > a bit crazy to take a DataStore from managed to unmanaged after
> > creation.
> > > >
> > > > >
> > > > > Sound like I'm in sync with what you were thinking?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > > > mike.tutkowski@solidfire.com
> > > > >> wrote:
> > > > >
> > > > >> I agree, John. Just wanted to point out that I have a working GUI
> > for
> > > > you
> > > > >> to review (in that document), if you'd like to check it out.
> > > > >>
> > > > >> Thanks!
> > > > >>
> > > > >>
> > > > >> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com>
> > > > wrote:
> > > > >>
> > > > >>> Mike,
> > > > >>>
> > > > >>> I would like the UIs of two features reviewed together to ensure
> > > > >>> consistency across the concepts of hypervisor throttled IOPs and
> > > > >>> storage device provisioned IOPs. I see the potential for
> > confusion,
> > > > >>> and I think a side-by-side Ui review of these features will help
> > > > >>> minimize any potential confusion.
> > > > >>>
> > > > >>> As I mentioned, the term reconciliation issue will work itself if
> > it
> > > > >>> is acceptable that a VM is only permitted utilize hypervisor
> > throttled
> > > > >>> IOPs or storage provisioned IOPs.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> -John
> > > > >>>
> > > > >>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > > > >>> <mi...@solidfire.com> wrote:
> > > > >>>
> > > > >>>> Hi John,
> > > > >>>>
> > > > >>>> Yeah, when you get a chance, refer to the Google doc I sent to
> you
> > > the
> > > > >>>> other day to see how the GUI looks for provisioned storage IOPS.
> > > > >>>>
> > > > >>>> Several months ago, I put this topic out on the e-mail list and we
> > > > >>> decided
> > > > >>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
> > dialog.
> > > > >>>> Other storage vendors are coming out with QoS, so they should
> be
> > > able
> > > > to
> > > > >>>> leverage this GUI going forward (even if they, say, only use Max
> > > > IOPS).
> > > > >>>> These fields are optional and can be ignored for storage that
> > does not
> > > > >>>> support provisioned IOPS. Just like the Disk Size field, the
> > admin can
> > > > >>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
> > > > >>>>
> > > > >>>> I'm OK if we do an either/or model (either Wei's feature or mine,
> > as
> > > > is
> > > > >>>> decided by the admin).
> > > > >>>>
> > > > >>>> I'm not sure what we can do about these two features expressing
> > the
> > > > >>> speed
> > > > >>>> in different terms. I've never seen a SAN express the IOPS for
> > QoS in
> > > > >>> any
> > > > >>>> way other than total IOPS (i.e. not broken in into read/write
> > IOPS).
> > > > >>>>
> > > > >>>> Thanks
> > > > >>>>
> > > > >>>>
> > > > >>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell
> <jb...@basho.com>
> > > > >>> wrote:
> > > > >>>>
> > > > >>>>> Wei,
> > > > >>>>>
> > > > >>>>> We have been down the rabbit hole a bit on the
> Storage/Hypervisor
> > > > layer
> > > > >>>>> separation, but we still need to reconcile the behavior of
> > hypervisor
> > > > >>>>> throttled I/O and storage provisioned IOPS. I see the following
> > > > issues
> > > > >>>>> outstanding:
> > > > >>>>>
> > > > >>>>> 1. Hypervisor throttled IOPS are expressed as discrete
> read/write
> > > > >>> values
> > > > >>>>> where as storage provisioned IOPS are expressed as total IOPS.
> > > > >>>>> 2. How do we handle VMs with throttled IOPS attached to
> storage
> > > > volumes
> > > > >>>>> with provisioned IOPS?
> > > > >>>>> 3. How should usage data be captured for throttled and
> > provisioned
> > > > IOPS
> > > > >>>>> that will permit providers to discriminate these guaranteed
> > > > operations
> > > > >>> in
> > > > >>>>> the event they want to bill for it differently?
> > > > >>>>> 4. What is the user experience for throttled and provisioned
> IOPS
> > > > that
> > > > >>>>> minimizes confusion of these concepts?
> > > > >>>>>
> > > > >>>>> My thinking is that a VM can have either utilize hypervisor
> > throttled
> > > > >>> IOPS
> > > > >>>>> or storage provisioned IOPS. This policy would neatly solve
> > items 1
> > > > >>> and 2.
> > > > >>>>> Since the two facilities would not be permitted to operate
> > together,
> > > > >>> they
> > > > >>>>> do not need to be semantically compatible. I think item 3 can be
> > > > >>> resolved
> > > > >>>>> with an additional flag or two on the usage records. As for
> > Item 4,
> > > > I
> > > > >>> am
> > > > >>>>> not familiar with how these two enhancements are depicted in
> the
> > > user
> > > > >>>>> interface. I think we need to review the UI enhancements for
> > both
> > > > >>>>> enhancements and ensure they are consistent.
> > > > >>>>>
> > > > >>>>> Do these solutions make sense?
> > > > >>>>>
> > > > >>>>> Thanks,
> > > > >>>>> -John
> > > > >>>>>
> > > > >>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com>
> > > wrote:
> > > > >>>>>
> > > > >>>>>> John and Mike,
> > > > >>>>>>
> > > > >>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> > > > >>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 ,
> CLOUDSTACK-
> > > 2875 ) this
> > > > >>> week.
> > > > >>>>>> I will start to develop on iops/bps changes tomorrow, and ask
> > for
> > > > >>> second
> > > > >>>>>> merge request after testing.
> > > > >>>>>>
> > > > >>>>>> -Wei
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > > > >>>>>>
> > > > >>>>>>> I believe I understand where you're going with this, John, and
> > > have
> > > > >>> been
> > > > >>>>>>> re-working this section of code today.
> > > > >>>>>>>
> > > > >>>>>>> I should be able to run it by you tomorrow.
> > > > >>>>>>>
> > > > >>>>>>> Thanks for the comments,
> > > > >>>>>>> Mike
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > > <jb...@basho.com>
> > > > >>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Mike,
> > > > >>>>>>>>
> > > > >>>>>>>> See my responses in-line below.
> > > > >>>>>>>>
> > > > >>>>>>>> Thanks,
> > > > >>>>>>>> -John
> > > > >>>>>>>>
> > > > >>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > > > >>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> Hi John,
> > > > >>>>>>>>
> > > > >>>>>>>> Thanks for the response.
> > > > >>>>>>>>
> > > > >>>>>>>> "I am fine with the VolumeManager determining whether or
> > > not a
> > > > >>> Volume
> > > > >>>>> is
> > > > >>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> actual
> > > > >>> isManaged
> > > > >>>>>>>> method), and asking the device driver to allocate resources
> > for
> > > > the
> > > > >>>>>>> volume
> > > > >>>>>>>> if it is managed."
> > > > >>>>>>>>
> > > > >>>>>>>> Are you thinking you'd like to see an isManaged() method
> > > added to
> > > > >>> the
> > > > >>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then the
> > > > >>> storage
> > > > >>>>>>>> framework could call the manage() (or whatever name)
> method
> > > (which
> > > > >>>>> would
> > > > >>>>>>> be
> > > > >>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and
> this
> > > > would
> > > > >>>>> call
> > > > >>>>>>>> into a new method in the hypervisor code to create, say on
> > > > >>> XenServer,
> > > > >>>>> an
> > > > >>>>>>> SR?
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> I would like to see canBeManaged() : boolean on
> > > DataStoreDriver.
> > > > >>> Since
> > > > >>>>>>>> the notion of Volumes only pertains to primary storage, I
> > would
> > > > add
> > > > >>> a
> > > > >>>>>>>> allocateStorage and deallocateStorage (Storage is a straw
> man
> > > term
> > > > >>> --
> > > > >>>>>>>> something other than volume) methods to
> > > > >>>>> allocate/create/deallocate/delete
> > > > >>>>>>>> underlying storage. To my mind, managed is a mutable
> property
> > > of
> > > > >>>>>>> DataStore
> > > > >>>>>>>> which can be enabled if/when the underlying
> DataStoreDriver
> > > can be
> > > > >>>>>>> managed.
> > > > >>>>>>>> This approach allows operators to override manageability of
> > > > devices.
> > > > >>>>>>>>
> > > > >>>>>>>> In terms of orchestration/process flow for SR, the Xen plugin
> > > > would
> > > > >>> be
> > > > >>>>>>>> responsible for composing DataStore/Volume methods to
> > > create any
> > > > >>>>>>>> directories or files necessary for the SR. There should be no
> > > > >>>>>>> dependencies
> > > > >>>>>>>> from the Storage to the Hypervisor layer. As I said earlier,
> > such
> > > > >>>>>>> circular
> > > > >>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> Just want to make sure I'm on the same page with you.
> > > > >>>>>>>>
> > > > >>>>>>>> Thanks again, John
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > > <jb...@basho.com>
> > > > >>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> Mike,
> > > > >>>>>>>>>
> > > > >>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> > > > supports n
> > > > >>>>>>>>> devices types with each specific behaviors of m hypervisors.
> > > > Such
> > > > >>> a
> > > > >>>>>>>>> scenario will create an unmaintainable and untestable
> beast.
> > > > >>>>>>> Therefore, my
> > > > >>>>>>>>> thoughts and recommendations are driven to evolve the
> > > Storage
> > > > layer
> > > > >>>>>>> towards
> > > > >>>>>>>>> this separation.
> > > > >>>>>>>>>
> > > > >>>>>>>>> I am fine with the VolumeManager determining whether
> or
> > > not a
> > > > >>> Volume
> > > > >>>>> is
> > > > >>>>>>>>> managed (i.e. not based on the StoragePoolType, but an
> actual
> > > > >>>>> isManaged
> > > > >>>>>>>>> method), and asking the device driver to allocate resources
> > for
> > > > the
> > > > >>>>>>> volume
> > > > >>>>>>>>> if it is managed. Furthermore, the device driver needs to
> > > > indicate
> > > > >>>>>>> whether
> > > > >>>>>>>>> or not it supports management operations. Finally, I think
> > we
> > > > >>> need to
> > > > >>>>>>>>> provide the ability for an administrator to elect to have
> > > > something
> > > > >>>>>>> that is
> > > > >>>>>>>>> manageable be unmanaged (i.e. the driver is capable
> managing
> > > the
> > > > >>>>> device,
> > > > >>>>>>>>> but the administrator has elected to leave it unmanaged).
> > > > >>>>>>>>>
> > > > >>>>>>>>> Creation of a structure on the volume should be done in
> the
> > > Xen
> > > > >>>>>>>>> hypervisor module using methods exposed by the Storage
> > > layer to
> > > > >>>>> perform
> > > > >>>>>>>>> low-level operations (e.g. make directories, create a file,
> > etc).
> > > > >>>>> This
> > > > >>>>>>>>> structure is specific to the operation of the Xen
> > hypervisor, as
> > > > >>> such,
> > > > >>>>>>>>> should be confined to its implementation. From my
> > > perspective,
> > > > >>>>> nothing
> > > > >>>>>>> in
> > > > >>>>>>>>> the Storage layer should be concerned with content. From
> its
> > > > >>>>>>> perspective,
> > > > >>>>>>>>> structure and data are opaque. It provides the means to
> > query
> > > > the
> > > > >>>>> data
> > > > >>>>>>> to
> > > > >>>>>>>>> support the interpretation of the content by higher-level
> > > layers
> > > > >>> (e.g.
> > > > >>>>>>>>> Hypervisors). To my mind, attach should be a composition
> of
> > > > >>>>> operations
> > > > >>>>>>>>> from the Storage layer that varies based on the Volume
> > > storage
> > > > >>>>> protocol
> > > > >>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > > > >>>>>>>>>
> > > > >>>>>>>>> Thanks,
> > > > >>>>>>>>> -John
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > > > >>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>> Hi John,
> > > > >>>>>>>>>
> > > > >>>>>>>>> Alternatively to the way the attach logic is implemented in
> > my
> > > > >>> patch,
> > > > >>>>> we
> > > > >>>>>>>>> could do the following:
> > > > >>>>>>>>>
> > > > >>>>>>>>> Leave the attach logic in the agent code alone. In
> > > > >>> VolumeManagerImpl
> > > > >>>>> we
> > > > >>>>>>>>> create an AttachVolumeCommand and send it to the
> > > hypervisor.
> > > > Before
> > > > >>>>> this
> > > > >>>>>>>>> command is sent, we could check to see if we're dealing
> with
> > > > >>> Dynamic
> > > > >>>>> (or
> > > > >>>>>>>>> whatever we want to call it) storage and - if so - send a
> > "Create
> > > > >>> SR"
> > > > >>>>>>>>> command to the hypervisor. If this returns OK, we would
> then
> > > > >>> proceed
> > > > >>>>> to
> > > > >>>>>>> the
> > > > >>>>>>>>> AttachVolumeCommand, as usual.
> > > > >>>>>>>>>
> > > > >>>>>>>>> This way the attach logic remains the same and we just add
> > > > another
> > > > >>>>>>>>> command to the agent code that is called for this particular
> > > type
> > > > >>> of
> > > > >>>>>>>>> storage.
> > > > >>>>>>>>>
> > > > >>>>>>>>> What do you think?
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > > > >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>> Hey John,
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I created a document for a customer today that outlines
> how
> > > the
> > > > >>>>> plug-in
> > > > >>>>>>>>>> works from a user standpoint. This will probably be of use
> > to
> > > > >>> you, as
> > > > >>>>>>> well,
> > > > >>>>>>>>>> as you perform the code review.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I have shared this document with you (you should have
> > > received
> > > > >>> that
> > > > >>>>>>>>>> information in a separate e-mail).
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Talk to you later!
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > > > >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> Oh, OK, that sounds really good, John.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > > > jburwell@basho.com
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts,
> but
> > > > want
> > > > >>> to
> > > > >>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > > > >>>>>>> disk_io_throttle,
> > > > >>>>>>>>>>>> and
> > > > >>>>>>>>>>>> object_store branches. I hope to collect them in a
> > > coherent
> > > > >>> form
> > > > >>>>>>>>>>>> tomorrow (5 June 2013).
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > > > >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
> > notion
> > > > of
> > > > >>> a
> > > > >>>>>>>>>>>> managed
> > > > >>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the
> > > notion
> > > > >>> of
> > > > >>>>>>>>>>>>> provisioned storage IOPS.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> The storage framework that leverages the plug-in was
> > > > >>> incomplete,
> > > > >>>>> so
> > > > >>>>>>>>>>>> I had
> > > > >>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> I appreciate all the time you've been spending on this.
> > :)
> > > Do
> > > > >>> you
> > > > >>>>>>>>>>>> have a
> > > > >>>>>>>>>>>>> recommendation as to how we should accomplish
> what
> > > you're
> > > > >>> looking
> > > > >>>>>>>>>>>> for?
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Thanks!
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > > > >>> jburwell@basho.com>
> > > > >>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
> > notion
> > > > of
> > > > >>> a
> > > > >>>>>>>>>>>> managed
> > > > >>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> I want to see a separation of the management
> > > capabilities
> > > > >>> (i.e.
> > > > >>>>>>> can
> > > > >>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>> device be managed/does an operator want it
> managed
> > > by
> > > > >>> CloudStack)
> > > > >>>>>>>>>>>> from the
> > > > >>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
> > > semantic
> > > > >>> that
> > > > >>>>>>>>>>>> will
> > > > >>>>>>>>>>>>>> allow any type of storage device to be managed. I
> also
> > > want
> > > > >>> to
> > > > >>>>>>> make
> > > > >>>>>>>>>>>>>> progress on decoupling the storage types from the
> > > hypervisor
> > > > >>>>>>>>>>>> definitions.
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > > > >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Hi John,
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> No problem. Answers are below in red.
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Thanks!
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > > > >>>>> jburwell@basho.com
> > > > >>>>>>>>
> > > > >>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Could you please answer the following questions
> for
> > > me
> > > > with
> > > > >>>>>>>>>>>> regards to
> > > > >>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and
> SAN
> > > > volumes?
> > > > >>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN
> 0).
> > > > >>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and
> > > CloudStack
> > > > >>>>>>>>>>>> Volumes?
> > > > >>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Are the LUN(s) created by the management
> server or
> > > > >>> externally
> > > > >>>>> by
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>> operator?
> > > > >>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN
> volume
> > > (same
> > > > as a
> > > > >>>>> SAN
> > > > >>>>>>>>>>>> LUN) is
> > > > >>>>>>>>>>>>>>> created by the management server (via the plug-in)
> > > the
> > > > first
> > > > >>>>> time
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
> > still
> > > > >>> want
> > > > >>>>> to
> > > > >>>>>>>>>>>> use a
> > > > >>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already
> today
> > > > (prior
> > > > >>> to
> > > > >>>>>>>>>>>> 4.2). The
> > > > >>>>>>>>>>>>>>> admin manually creates the SAN volume and - in
> this
> > > case -
> > > > >>>>>>>>>>>> multiple VMs
> > > > >>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>> data disks can share this SAN volume. While you
> can do
> > > this
> > > > >>>>>>> today,
> > > > >>>>>>>>>>>> it is
> > > > >>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Are the SAN volumes by the management server
> or
> > > externally
> > > > >>> by
> > > > >>>>>>> the
> > > > >>>>>>>>>>>>>> operator?
> > > > >>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN
> volumes
> > > are
> > > > >>>>>>> completely
> > > > >>>>>>>>>>>>>> managed
> > > > >>>>>>>>>>>>>>> by the management server (via the plug-in). There
> is
> > > no
> > > > admin
> > > > >>>>>>>>>>>>>> interaction.
> > > > >>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN
> volume
> > > and a
> > > > >>>>>>> CloudStack
> > > > >>>>>>>>>>>>>> volume,
> > > > >>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > > supports
> > > > true
> > > > >>>>> QoS.
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> I would like to clarify how these pieces are related
> > > and
> > > > >>>>>>> expected
> > > > >>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>> operate.
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > > > >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple
> VMs
> > > with
> > > > >>>>>>>>>>>> provisioned
> > > > >>>>>>>>>>>>>> IOPS
> > > > >>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a
> single
> > > SAN
> > > > >>> volume
> > > > >>>>>>>>>>>> (LUN) to a
> > > > >>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have
> multiple
> > > > >>> CloudStack
> > > > >>>>>>>>>>>> volumes
> > > > >>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee
> QoS.
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume
> house
> > > more
> > > > >>> than
> > > > >>>>>>> one
> > > > >>>>>>>>>>>>>>>>> CloudStack volume, then can do that today
> without
> > > any of
> > > > my
> > > > >>>>>>>>>>>> plug-in
> > > > >>>>>>>>>>>>>> code.
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> > > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume
> for
> > > > >>>>> CloudStack's
> > > > >>>>>>>>>>>> use
> > > > >>>>>>>>>>>>>> onto
> > > > >>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me
> --
> > > > concepts
> > > > >>>>> from
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> > > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
> > only
> > > > used
> > > > >>>>> (as
> > > > >>>>>>>>>>>> far as
> > > > >>>>>>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>>> know)
> > > > >>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not
> > > PreSetup an
> > > > >>> SR).
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> > > VMware, it
> > > > >>> uses
> > > > >>>>>>> the
> > > > >>>>>>>>>>>> vmfs
> > > > >>>>>>>>>>>>>>>> type.
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> KVM,
> > > it uses
> > > > >>> the
> > > > >>>>>>>>>>>>>>>>>>> SharedMountPoint
> > > > >>>>>>>>>>>>>>>>>>>> type.
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking
> here
> > > was to
> > > > >>> make a
> > > > >>>>>>>>>>>> new type
> > > > >>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability
> Edison
> > > added
> > > > >>> into
> > > > >>>>>>> the
> > > > >>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more
> specificy,
> > > though:
> > > > >>>>>>>>>>>> Dynamic_iSCSI
> > > > >>>>>>>>>>>>>>>>>>> versus,
> > > > >>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike
> Tutkowski
> > > <
> > > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know
> or
> > > care
> > > > that
> > > > >>> it
> > > > >>>>>>> is
> > > > >>>>>>>>>>>> being
> > > > >>>>>>>>>>>>>>>> used
> > > > >>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > > questions
> > > > >>> about
> > > > >>>>> it
> > > > >>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>> storing."
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > > SolidFire
> > > > >>> plug-in
> > > > >>>>>>>>>>>> needs to
> > > > >>>>>>>>>>>>>>>>>>> call
> > > > >>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the
> hypervisor
> > > logic
> > > > >>>>> knows
> > > > >>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>> treat
> > > > >>>>>>>>>>>>>>>>>>> it as
> > > > >>>>>>>>>>>>>>>>>>>>> such.
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint
> > > and just
> > > > >>>>>>>>>>>> calling it
> > > > >>>>>>>>>>>>>>>> iSCSI
> > > > >>>>>>>>>>>>>>>>>>> or
> > > > >>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for
> the
> > > hypervisor
> > > > >>>>>>> attach
> > > > >>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform
> the
> > > necessary
> > > > >>>>>>>>>>>> activities.
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John
> Burwell <
> > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike
> Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to
> > > manage a
> > > > >>>>>>> device,
> > > > >>>>>>>>>>>> an
> > > > >>>>>>>>>>>>>>>>>>> operator
> > > > >>>>>>>>>>>>>>>>>>>>>> may
> > > > >>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by
> CloudStack
> > > (e.g. the
> > > > >>>>>>> device
> > > > >>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>> shared
> > > > >>>>>>>>>>>>>>>>>>> by
> > > > >>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has
> chosen
> > > to
> > > > >>> dedicate
> > > > >>>>>>>>>>>> only a
> > > > >>>>>>>>>>>>>>>> portion
> > > > >>>>>>>>>>>>>>>>>>>>>> of it
> > > > >>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning
> make
> > > sense?"
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
> > > volume via
> > > > >>> the
> > > > >>>>>>>>>>>> plug-in
> > > > >>>>>>>>>>>>>>>>>>> (before
> > > > >>>>>>>>>>>>>>>>>>>>>> an
> > > > >>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
> > > work unless
> > > > >>> the
> > > > >>>>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>>>>> consumes
> > > > >>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an
> SR.
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > > CloudStack, an
> > > > >>>>>>> operator
> > > > >>>>>>>>>>>>>>>> allocates
> > > > >>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and
> exposes it
> > > > through
> > > > >>> a
> > > > >>>>>>>>>>>> LUN. They
> > > > >>>>>>>>>>>>>>>>>>> simply
> > > > >>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
> > > CloudStack,
> > > > >>> but
> > > > >>>>>>>>>>>> not allow
> > > > >>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > > changes, we
> > > > >>> would
> > > > >>>>>>> be
> > > > >>>>>>>>>>>>>> passing
> > > > >>>>>>>>>>>>>>>>>>> in a
> > > > >>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > > necessary
> > > > >>> hypervisor
> > > > >>>>>>>>>>>> support
> > > > >>>>>>>>>>>>>>>> (like
> > > > >>>>>>>>>>>>>>>>>>> an
> > > > >>>>>>>>>>>>>>>>>>>>>> SR)
> > > > >>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have
> the
> > > storage
> > > > >>>>>>> framework
> > > > >>>>>>>>>>>>>> itself
> > > > >>>>>>>>>>>>>>>>>>>>>> detect
> > > > >>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support
> from
> > > the
> > > > >>> hypervisor
> > > > >>>>>>>>>>>> side and
> > > > >>>>>>>>>>>>>>>>>>> have
> > > > >>>>>>>>>>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to
> create
> > > the
> > > > >>> SR
> > > > >>>>>>>>>>>> before the
> > > > >>>>>>>>>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin
> > > should
> > > > have a
> > > > >>>>>>> rich
> > > > >>>>>>>>>>>> enough
> > > > >>>>>>>>>>>>>>>>>>>>>> interface to storage to determine
> available
> > > for
> > > > volume
> > > > >>>>>>>>>>>> storage.
> > > > >>>>>>>>>>>>>> For
> > > > >>>>>>>>>>>>>>>>>>> Xen,
> > > > >>>>>>>>>>>>>>>>>>>>>> this interface would allow the
> interrogation of
> > > the
> > > > >>>>> device
> > > > >>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>> determine the
> > > > >>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
> > > shouldn't
> > > > >>> know
> > > > >>>>>>>>>>>> or care
> > > > >>>>>>>>>>>>>>>>>>> that it
> > > > >>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able
> to
> > > > answer
> > > > >>>>>>>>>>>> questions
> > > > >>>>>>>>>>>>>>>>>>> about it
> > > > >>>>>>>>>>>>>>>>>>>>>> is storing.
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > > Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire
> driver
> > > (which
> > > > >>> is a
> > > > >>>>>>>>>>>> type of
> > > > >>>>>>>>>>>>>>>>>>>>>> so-called
> > > > >>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a
> new
> > > Primary
> > > > >>>>>>>>>>>> Storage shows
> > > > >>>>>>>>>>>>>>>> up
> > > > >>>>>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering
> that
> > > > references
> > > > >>> the
> > > > >>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>> tag
> > > > >>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
> > > volume. This
> > > > >>> leads
> > > > >>>>>>> to
> > > > >>>>>>>>>>>> a new
> > > > >>>>>>>>>>>>>>>> row
> > > > >>>>>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack
> > > volume to a
> > > > >>> VM
> > > > >>>>>>>>>>>> (attach
> > > > >>>>>>>>>>>>>>>> disk).
> > > > >>>>>>>>>>>>>>>>>>>>>> This
> > > > >>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling
> the
> > > plug-in
> > > > >>> to
> > > > >>>>>>>>>>>> create a
> > > > >>>>>>>>>>>>>> new
> > > > >>>>>>>>>>>>>>>>>>>>>> volume
> > > > >>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a
> SAN).
> > > The
> > > > >>> plug-in
> > > > >>>>>>>>>>>> also
> > > > >>>>>>>>>>>>>>>> updates
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data
> > > (like the
> > > > >>> IQN of
> > > > >>>>>>>>>>>> the SAN
> > > > >>>>>>>>>>>>>>>>>>>>>> volume).
> > > > >>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> > > > CloudStack
> > > > >>>>>>>>>>>> volume is in
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
> > > volume
> > > > >>> will be
> > > > >>>>>>>>>>>> in the
> > > > >>>>>>>>>>>>>>>>>>> 'Ready'
> > > > >>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the
> > > plug-in
> > > > >>> code
> > > > >>>>>>>>>>>> will not
> > > > >>>>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>>>>>>>> called
> > > > >>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and
> > > detects
> > > > the
> > > > >>>>>>>>>>>> CloudStack
> > > > >>>>>>>>>>>>>>>>>>> volume
> > > > >>>>>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form
> of a
> > > > >>> hypervisor
> > > > >>>>>>> data
> > > > >>>>>>>>>>>>>>>> structure
> > > > >>>>>>>>>>>>>>>>>>>>>> (ex.
> > > > >>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
> > > Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > > terminology,
> > > > >>> volume,
> > > > >>>>>>>>>>>> in these
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > > allocation on
> > > > >>> the
> > > > >>>>>>>>>>>> device,
> > > > >>>>>>>>>>>>>>>>>>>>>> correct?"
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to
> mean
> > > 'SAN
> > > > >>> volume'.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user
> can
> > > make in
> > > > >>>>>>>>>>>> CloudStack, I
> > > > >>>>>>>>>>>>>>>>>>> try to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike
> > > Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well
> make
> > > sense, but
> > > > >>> I'm
> > > > >>>>>>>>>>>> having a
> > > > >>>>>>>>>>>>>>>> hard
> > > > >>>>>>>>>>>>>>>>>>>>>> time
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on
> > > this
> > > > >>> conversation
> > > > >>>>>>>>>>>> as he
> > > > >>>>>>>>>>>>>> was
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the
> approach I
> > > took.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM,
> John
> > > Burwell <
> > > > >>>>>>>>>>>>>>>> jburwell@basho.com
> > > > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
> > > distinct
> > > > >>> concepts
> > > > >>>>>>> --
> > > > >>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>> device
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols.
> In
> > > both
> > > > >>> cases, we
> > > > >>>>>>>>>>>> are
> > > > >>>>>>>>>>>>>>>>>>>>>> communicating with
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
> > > > create/delete
> > > > >>>>>>>>>>>> volumes
> > > > >>>>>>>>>>>>>>>>>>> (Dynamic)
> > > > >>>>>>>>>>>>>>>>>>>>>> on the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the
> > > volume to
> > > > be
> > > > >>>>>>>>>>>> volume to be
> > > > >>>>>>>>>>>>>>>>>>>>>> managed
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context.
> To
> > > ensure
> > > > >>> that
> > > > >>>>> we
> > > > >>>>>>>>>>>> are in
> > > > >>>>>>>>>>>>>>>>>>> sync on
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > > definitions,
> > > > >>> refers to
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>> physical
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
> > > Minimally,
> > > > we
> > > > >>>>>>> must
> > > > >>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>> able
> > > > >>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device
> to
> > > move bits
> > > > >>> from
> > > > >>>>>>>>>>>> one place
> > > > >>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>> another,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc.
> Optionally, a
> > > > >>> storage
> > > > >>>>>>>>>>>> device
> > > > >>>>>>>>>>>>>> may
> > > > >>>>>>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>>>>>>>> able to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore,
> > > we can have
> > > > a
> > > > >>>>>>>>>>>> unmanaged
> > > > >>>>>>>>>>>>>>>> iSCSI
> > > > >>>>>>>>>>>>>>>>>>>>>> device
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and
> we
> > > can have a
> > > > >>>>>>> managed
> > > > >>>>>>>>>>>>>>>> SolidFire
> > > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is
> capable
> > > of
> > > > >>> allocating
> > > > >>>>>>>>>>>> LUNs and
> > > > >>>>>>>>>>>>>>>>>>> storing
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack
> may
> > > be able
> > > > >>> to
> > > > >>>>>>>>>>>> manage a
> > > > >>>>>>>>>>>>>>>>>>> device,
> > > > >>>>>>>>>>>>>>>>>>>>>> an
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > > unmanaged by
> > > > >>>>>>> CloudStack
> > > > >>>>>>>>>>>> (e.g.
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>> device is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
> > > operator has
> > > > >>> chosen
> > > > >>>>>>> to
> > > > >>>>>>>>>>>>>>>> dedicate
> > > > >>>>>>>>>>>>>>>>>>>>>> only a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does
> my
> > > > reasoning
> > > > >>>>> make
> > > > >>>>>>>>>>>> sense?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
> > > reasonable, it
> > > > >>> seems
> > > > >>>>>>>>>>>>>> appropriate
> > > > >>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns
> from
> > > > >>> StoragePoolType,
> > > > >>>>>>>>>>>> add the
> > > > >>>>>>>>>>>>>>>>>>> notion
> > > > >>>>>>>>>>>>>>>>>>>>>> of a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached
> driver
> > > that
> > > > >>>>> indicates
> > > > >>>>>>>>>>>> whether
> > > > >>>>>>>>>>>>>>>> or
> > > > >>>>>>>>>>>>>>>>>>>>>> not is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and
> establish
> > > a
> > > > >>> abstraction
> > > > >>>>>>>>>>>>>>>> representing a
> > > > >>>>>>>>>>>>>>>>>>>>>> physical
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that
> is
> > > > >>> associated
> > > > >>>>>>>>>>>> with a
> > > > >>>>>>>>>>>>>>>> volume.
> > > > >>>>>>>>>>>>>>>>>>>>>> With
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
> > > drivers can
> > > > >>>>>>> declare
> > > > >>>>>>>>>>>> which
> > > > >>>>>>>>>>>>>>>>>>>>>> protocols they
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter
> a
> > > device
> > > > managed
> > > > >>> by
> > > > >>>>>>>>>>>>>> CloudStack,
> > > > >>>>>>>>>>>>>>>>>>>>>> utilize the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed
> by
> > > the driver to
> > > > >>>>>>> automate
> > > > >>>>>>>>>>>>>>>>>>> allocation.
> > > > >>>>>>>>>>>>>>>>>>>>>> If
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make
> sense,
> > > then we can
> > > > >>> sit
> > > > >>>>>>>>>>>> down and
> > > > >>>>>>>>>>>>>>>>>>> drill
> > > > >>>>>>>>>>>>>>>>>>>>>> down to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
> > > Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between
> the
> > > current
> > > > iSCSI
> > > > >>>>>>> type
> > > > >>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>> Dynamic
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in
> and
> > > create
> > > > a
> > > > >>>>>>>>>>>> Primary
> > > > >>>>>>>>>>>>>>>> Storage
> > > > >>>>>>>>>>>>>>>>>>>>>> based
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time,
> > > the
> > > > iSCSI
> > > > >>>>>>>>>>>> volume must
> > > > >>>>>>>>>>>>>>>>>>> exist
> > > > >>>>>>>>>>>>>>>>>>>>>> on
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated).
> > > Future
> > > > >>>>>>>>>>>> CloudStack
> > > > >>>>>>>>>>>>>>>> volumes
> > > > >>>>>>>>>>>>>>>>>>> are
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created
> > > behind the
> > > > >>>>>>> scenes.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to
> go in
> > > and
> > > > create
> > > > >>>>>>>>>>>> Primary
> > > > >>>>>>>>>>>>>>>> Storage
> > > > >>>>>>>>>>>>>>>>>>>>>> based
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete
> > > volumes on
> > > > >>> its
> > > > >>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>> system
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
> > > storage
> > > > >>>>>>>>>>>> framework). When
> > > > >>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>> user
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that
> was
> > > created,
> > > > the
> > > > >>>>>>>>>>>> framework
> > > > >>>>>>>>>>>>>>>> tells
> > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume.
> After
> > > this is
> > > > >>> done,
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called.
> > > No
> > > > >>> hypervisor
> > > > >>>>>>>>>>>> data
> > > > >>>>>>>>>>>>>>>>>>> structure
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume
> was
> > > just
> > > > >>> created.
> > > > >>>>>>> The
> > > > >>>>>>>>>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>>>>> data
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM,
> Mike
> > > > Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should
> > > probably
> > > > have
> > > > >>>>>>>>>>>> defined them
> > > > >>>>>>>>>>>>>>>> up
> > > > >>>>>>>>>>>>>>>>>>>>>> front
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is
> pre-
> > > allocated
> > > > >>> (ex.
> > > > >>>>>>>>>>>> an admin
> > > > >>>>>>>>>>>>>>>>>>>>>> creates a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a
> hypervisor
> > > data
> > > > >>> structure
> > > > >>>>>>> is
> > > > >>>>>>>>>>>>>> created
> > > > >>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR),
> then
> > > that
> > > > >>>>>>> hypervisor
> > > > >>>>>>>>>>>> data
> > > > >>>>>>>>>>>>>>>>>>>>>> structure
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks
> (VDI)
> > > are later
> > > > >>>>>>> placed
> > > > >>>>>>>>>>>> on
> > > > >>>>>>>>>>>>>> this
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In
> these
> > > cases, the
> > > > >>>>>>> attach
> > > > >>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>>>>>>> assumes
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is
> already
> > > in place
> > > > >>> and
> > > > >>>>>>>>>>>> simply
> > > > >>>>>>>>>>>>>>>>>>> attaches
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure
> to
> > > the VM in
> > > > >>>>>>>>>>>> question.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is
> not
> > > > >>>>> pre-allocated.
> > > > >>>>>>>>>>>> Instead
> > > > >>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could
> be a
> > > SAN
> > > > (not
> > > > >>> a
> > > > >>>>>>>>>>>> volume on
> > > > >>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>> SAN,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
> > > structure
> > > > >>> must be
> > > > >>>>>>>>>>>> created
> > > > >>>>>>>>>>>>>>>>>>> when an
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because
> these
> > > types of
> > > > >>> volumes
> > > > >>>>>>>>>>>> have not
> > > > >>>>>>>>>>>>>>>>>>> been
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > > structure by an
> > > > >>>>> admin.
> > > > >>>>>>>>>>>> Once
> > > > >>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer
> for
> > > this
> > > > >>> volume,
> > > > >>>>>>> it
> > > > >>>>>>>>>>>>>> attaches
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM
> in
> > > question.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
> > > John Burwell
> > > > <
> > > > >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of
> the
> > > Dynamic
> > > > type
> > > > >>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>> behavior
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I
> ask
> > > about the
> > > > >>>>>>>>>>>> difference.
> > > > >>>>>>>>>>>>>>>>>>> Another
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
> > > definition
> > > > of
> > > > >>> a
> > > > >>>>>>>>>>>> Dynamic
> > > > >>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM,
> Mike
> > > Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type
> is
> > > > uniquely
> > > > >>>>> used
> > > > >>>>>>>>>>>> by
> > > > >>>>>>>>>>>>>>>>>>> XenServer
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary
> Storage
> > > that is
> > > > >>> directly
> > > > >>>>>>>>>>>> based on
> > > > >>>>>>>>>>>>>> an
> > > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the
> step of
> > > going
> > > > to
> > > > >>> the
> > > > >>>>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on
> that
> > > iSCSI
> > > > >>> target as
> > > > >>>>>>>>>>>>>> CloudStack
> > > > >>>>>>>>>>>>>>>>>>> does
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > > supported for
> > > > >>>>>>>>>>>> XenServer.
> > > > >>>>>>>>>>>>>> For
> > > > >>>>>>>>>>>>>>>>>>> all
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go
> to
> > > the
> > > > >>> hypervisor
> > > > >>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>> perform
> > > > >>>>>>>>>>>>>>>>>>>>>> this
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13
> PM,
> > > John
> > > > Burwell
> > > > >>> <
> > > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code,
> what
> > > is the
> > > > >>>>> difference
> > > > >>>>>>>>>>>> between
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't
> RBD
> > > considered
> > > > >>>>>>> Dynamic?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM,
> Mike
> > > > Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
> > > defined in
> > > > the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > > StoragePoolType {
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
> > > directory
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true),
> //
> > > NFS or CIFS
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared
> LUN,
> > > with a
> > > > >>>>> clusterfs
> > > > >>>>>>>>>>>>>> overlay
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
> > > Comstar
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer
> local
> > > LVM SR
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware
> VMFS
> > > storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > > XenServer, Storage
> > > > >>> Pool
> > > > >>>>>>>>>>>> is set
> > > > >>>>>>>>>>>>>>>> up
> > > > >>>>>>>>>>>>>>>>>>>>>> by
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer
> local
> > > EXT SR
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
> > > zone-wide
> > > > >>> storage
> > > > >>>>>>> (ex.
> > > > >>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
> > > shared) {
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41
> PM,
> > > Mike
> > > > >>> Tutkowski
> > > > >>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say
> another
> > > storage
> > > > >>>>> company
> > > > >>>>>>>>>>>> wants
> > > > >>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its
> Quality
> > > of
> > > > Service
> > > > >>>>>>>>>>>> feature. It
> > > > >>>>>>>>>>>>>>>>>>> would
> > > > >>>>>>>>>>>>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well.
> > > They would
> > > > >>> need
> > > > >>>>>>> only
> > > > >>>>>>>>>>>>>>>>>>> implement a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
> > > necessary
> > > > >>> changes to
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 1:39
> > > PM, Mike
> > > > >>>>>>> Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > mike.tutkowski@solidfire.com> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
> > > XenServer code
> > > > is
> > > > >>>>>>>>>>>> SolidFire
> > > > >>>>>>>>>>>>>>>>>>> specific.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach
> logic is
> > > now aware
> > > > >>> of
> > > > >>>>>>>>>>>> dynamic,
> > > > >>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > > implementation of
> > > > >>> this
> > > > >>>>>>>>>>>> kind of
> > > > >>>>>>>>>>>>>>>>>>>>>> storage).
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2
> with
> > > Edison's
> > > > >>> storage
> > > > >>>>>>>>>>>> framework
> > > > >>>>>>>>>>>>>>>>>>>>>> changes.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
> > > framework that
> > > > >>>>> supported
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>> creation
> > > > >>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
> > > However, when I
> > > > >>>>>>>>>>>> visited with
> > > > >>>>>>>>>>>>>>>> him
> > > > >>>>>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that
> it
> > > was not
> > > > >>>>>>>>>>>> complete. We
> > > > >>>>>>>>>>>>>>>>>>> realized
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could
> do
> > > with these
> > > > >>>>>>> volumes
> > > > >>>>>>>>>>>> unless
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize
> this
> > > new type of
> > > > >>>>> storage
> > > > >>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>> create
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 1:28
> > > PM, John
> > > > >>> Burwell
> > > > >>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me
> > > that any
> > > > >>>>> operation
> > > > >>>>>>>>>>>> in the
> > > > >>>>>>>>>>>>>>>>>>> Storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care
> about
> > > details. I
> > > > >>> expect
> > > > >>>>>>>>>>>> to see
> > > > >>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of
> operations
> > > that can be
> > > > >>>>>>>>>>>>>> composed/driven
> > > > >>>>>>>>>>>>>>>>>>> by
> > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > > allocate
> > > > space/create
> > > > >>>>>>>>>>>> structures
> > > > >>>>>>>>>>>>>>>> per
> > > > >>>>>>>>>>>>>>>>>>>>>> their
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > > dependency, we are
> > > > >>> going
> > > > >>>>>>> to
> > > > >>>>>>>>>>>> end
> > > > >>>>>>>>>>>>>>>> with a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make
> the
> > > system
> > > > >>>>>>> increasingly
> > > > >>>>>>>>>>>>>>>>>>> difficult to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > > understanding that the
> > > > >>> Xen
> > > > >>>>>>>>>>>> specific
> > > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > > CitrixResourceBase
> > > > class?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 3:12
> > > PM, Mike
> > > > >>>>>>>>>>>> Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a
> bit
> > > more
> > > > >>> detail:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside
> from
> > > one setup
> > > > >>>>>>> method
> > > > >>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>> XenServer,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a
> volume on
> > > the
> > > > storage
> > > > >>>>>>>>>>>> system,
> > > > >>>>>>>>>>>>>> then
> > > > >>>>>>>>>>>>>>>> go
> > > > >>>>>>>>>>>>>>>>>>>>>> into
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data
> structure
> > > to make
> > > > use
> > > > >>> of
> > > > >>>>>>> the
> > > > >>>>>>>>>>>>>> volume
> > > > >>>>>>>>>>>>>>>>>>> (ex.
> > > > >>>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on
> XenServer
> > > or a
> > > > datastore
> > > > >>> on
> > > > >>>>>>>>>>>> ESX(i)).
> > > > >>>>>>>>>>>>>>>> VMs
> > > > >>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
> > > system's volume.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new
> storage
> > > framework,
> > > > >>>>>>> storage
> > > > >>>>>>>>>>>> need
> > > > >>>>>>>>>>>>>> no
> > > > >>>>>>>>>>>>>>>>>>>>>> longer
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily
> create
> > > a 1:1
> > > > >>>>>>> relationship
> > > > >>>>>>>>>>>>>> between
> > > > >>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's
> data
> > > disk
> > > > >>> (necessary
> > > > >>>>>>> for
> > > > >>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>> Quality
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a
> plug-
> > > in that is
> > > > >>> called
> > > > >>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>> dynamically
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
> > > storage
> > > > framework
> > > > >>> did
> > > > >>>>>>>>>>>> not
> > > > >>>>>>>>>>>>>>>> address
> > > > >>>>>>>>>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the
> hypervisor-
> > > specific data
> > > > >>>>>>>>>>>> structure
> > > > >>>>>>>>>>>>>> when
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case,
> I've
> > > been
> > > > >>> enhancing
> > > > >>>>> it
> > > > >>>>>>>>>>>> to do
> > > > >>>>>>>>>>>>>> so.
> > > > >>>>>>>>>>>>>>>>>>> I've
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > > submitted. I've got
> > > > >>> ESX(i)
> > > > >>>>>>> in
> > > > >>>>>>>>>>>> my
> > > > >>>>>>>>>>>>>>>> sandbox
> > > > >>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
> > > freeze date.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > > 1:03 PM, Mike
> > > > >>>>>>>>>>>> Tutkowski <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in -
> by
> > > itself - is
> > > > >>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>> agnostic.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
> > > volume-attach
> > > > >>> logic
> > > > >>>>>>>>>>>> (in the
> > > > >>>>>>>>>>>>>>>>>>> agent
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into
> the
> > > plug-in to
> > > > >>> have
> > > > >>>>>>> it
> > > > >>>>>>>>>>>>>> create a
> > > > >>>>>>>>>>>>>>>>>>>>>> volume
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time
> comes to
> > > attach the
> > > > >>> volume
> > > > >>>>>>> to
> > > > >>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>> hypervisor,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart
> enough
> > > to recognize
> > > > >>> it's
> > > > >>>>>>>>>>>> being
> > > > >>>>>>>>>>>>>>>>>>> invoked on
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume
> has
> > > just been
> > > > >>> created)
> > > > >>>>>>> and
> > > > >>>>>>>>>>>>>> create,
> > > > >>>>>>>>>>>>>>>>>>>>>> say, a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > > XenServer) or a
> > > > >>> datastore
> > > > >>>>>>>>>>>> (for
> > > > >>>>>>>>>>>>>>>>>>> VMware) to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just
> > > created.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending
> most
> > > of my time
> > > > >>>>>>> recently
> > > > >>>>>>>>>>>> making
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at
> > > 12:48 PM,
> > > > John
> > > > >>>>>>>>>>>> Burwell <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain
> why
> > > the the storage
> > > > >>>>>>> driver
> > > > >>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> 1:21
> > > PM, Mike
> > > > >>>>>>> Tutkowski
> > > > >>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
> > > would like to
> > > > >>> support
> > > > >>>>>>>>>>>> all
> > > > >>>>>>>>>>>>>>>>>>> hypervisors
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think
> I'm
> > > just out of
> > > > >>> time
> > > > >>>>>>>>>>>> for 4.2
> > > > >>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>> get
> > > > >>>>>>>>>>>>>>>>>>>>>> KVM
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this
> plug-in
> > > supports
> > > > >>>>>>> XenServer.
> > > > >>>>>>>>>>>>>>>>>>> Depending on
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2
> feature
> > > freeze, I
> > > > >>> have
> > > > >>>>> it
> > > > >>>>>>>>>>>> working
> > > > >>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be
> clear,
> > > this is
> > > > all
> > > > >>> in
> > > > >>>>>>>>>>>> regards
> > > > >>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>> Disk
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute
> > > Offerings post
> > > > 4.2.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> > > at 11:14 AM,
> > > > >>> Kelcey
> > > > >>>>>>>>>>>> Jamison
> > > > >>>>>>>>>>>>>>>>>>> Damage
> > > > >>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan
> on
> > > supporting
> > > > >>> KVM in
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>> patch
> > > > >>>>>>>>>>>>>>>>>>> cycle
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > > Message -----
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > > Tutkowski" <
> > > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > > dev@cloudstack.apache.org
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday,
> June
> > > 3, 2013
> > > > >>> 10:12:32 AM
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re:
> [MERGE]
> > > > >>>>> disk_io_throttling
> > > > >>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>> MASTER
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on
> merging
> > > Wei's feature
> > > > >>>>> first,
> > > > >>>>>>>>>>>> then
> > > > >>>>>>>>>>>>>>>> mine.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is
> for
> > > KVM only,
> > > > >>> then
> > > > >>>>> it
> > > > >>>>>>>>>>>> is a
> > > > >>>>>>>>>>>>>> non
> > > > >>>>>>>>>>>>>>>>>>> issue
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3,
> 2013
> > > at 8:55 AM,
> > > > Wei
> > > > >>>>>>> ZHOU
> > > > >>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing,
> as
> > > no one works
> > > > >>> on
> > > > >>>>>>>>>>>> billing
> > > > >>>>>>>>>>>>>> now,
> > > > >>>>>>>>>>>>>>>>>>>>>> users
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
> > > themselves. They
> > > > >>> can
> > > > >>>>>>> get
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of
> a
> > > VMs and
> > > > volumes
> > > > >>>>> for
> > > > >>>>>>>>>>>>>>>>>>> calculation.
> > > > >>>>>>>>>>>>>>>>>>>>>> Of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
> > > exact
> > > > limitation
> > > > >>>>>>> value
> > > > >>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>> individual
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation
> for
> > > nics as
> > > > well.
> > > > >>> I
> > > > >>>>>>> can
> > > > >>>>>>>>>>>> work
> > > > >>>>>>>>>>>>>> on
> > > > >>>>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
> > > throttling?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > > misunstand the second
> > > > the
> > > > >>>>>>>>>>>> question.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with
> what
> > > you said about
> > > > the
> > > > >>>>> two
> > > > >>>>>>>>>>>>>> features.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> > > Burwell <
> > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013,
> at
> > > 2:13 AM, Wei
> > > > >>> ZHOU
> > > > >>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's
> > > aswer helps you.
> > > > >>> I am
> > > > >>>>>>>>>>>> trying
> > > > >>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>> adding
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think
> billing
> > > should
> > > > >>> depend
> > > > >>>>> on
> > > > >>>>>>>>>>>> IO
> > > > >>>>>>>>>>>>>>>>>>> statistics
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation.
> Please
> > > review
> > > > >>>>>>>>>>>> disk_io_stat if
> > > > >>>>>>>>>>>>>> you
> > > > >>>>>>>>>>>>>>>>>>> have
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
> > > statistics
> > > > including
> > > > >>>>>>>>>>>> bytes/iops
> > > > >>>>>>>>>>>>>>>>>>>>>> read/write
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual
> machine.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the
> AWS
> > > model,
> > > > customers
> > > > >>>>> are
> > > > >>>>>>>>>>>> billed
> > > > >>>>>>>>>>>>>>>> more
> > > > >>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned
> IOPS,
> > > as well as,
> > > > for
> > > > >>>>>>> those
> > > > >>>>>>>>>>>>>>>>>>> operations (
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > http://aws.amazon.com/ebs/).
> > > > I
> > > > >>>>>>> would
> > > > >>>>>>>>>>>>>> imagine
> > > > >>>>>>>>>>>>>>>>>>> our
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to
> employ
> > > similar cost
> > > > >>>>> models.
> > > > >>>>>>>>>>>> Could
> > > > >>>>>>>>>>>>>> an
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model
> in
> > > the current
> > > > >>> patch?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you
> mean
> > > IOPS runtime
> > > > >>>>> change?
> > > > >>>>>>>>>>>> KVM
> > > > >>>>>>>>>>>>>>>>>>> supports
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for
> a
> > > running
> > > > virtual
> > > > >>>>>>>>>>>> machine
> > > > >>>>>>>>>>>>>>>> through
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> does
> > > not support
> > > > >>>>> changing
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>> parameters
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
> > > offering or disk
> > > > >>>>>>> offering).
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the
> > > Java interface
> > > > >>> level.
> > > > >>>>>>> I
> > > > >>>>>>>>>>>>>>>> apologize
> > > > >>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > > generalize allocation
> > > > >>>>>>>>>>>> algorithms
> > > > >>>>>>>>>>>>>> with a
> > > > >>>>>>>>>>>>>>>>>>> set
> > > > >>>>>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
> > > service guarantees
> > > > >>>>>>>>>>>> provided by a
> > > > >>>>>>>>>>>>>>>>>>>>>> resource?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a
> good
> > > question.
> > > > >>> Maybe it
> > > > >>>>>>> is
> > > > >>>>>>>>>>>>>> better
> > > > >>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > disk_io_throttling as Mike
> > > > >>> needs to
> > > > >>>>>>>>>>>> consider
> > > > >>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand
> on
> > > my thoughts
> > > > in a
> > > > >>>>>>> later
> > > > >>>>>>>>>>>>>>>> response
> > > > >>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
> > > between these two
> > > > >>>>>>>>>>>> features. I
> > > > >>>>>>>>>>>>>>>> think
> > > > >>>>>>>>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk_io_throttling
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be
> > > merged before
> > > > >>>>>>>>>>>> SolidFire, but
> > > > >>>>>>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>>> think
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
> > > between the
> > > > branches
> > > > >>>>>>>>>>>> (possibly
> > > > >>>>>>>>>>>>>>>> have
> > > > >>>>>>>>>>>>>>>>>>>>>> have
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk_io_throttling)
> > > to
> > > > >>> coordinate on
> > > > >>>>>>>>>>>> this
> > > > >>>>>>>>>>>>>>>> issue.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3
> John
> > > Burwell <
> > > > >>>>>>>>>>>> jburwell@basho.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I
> > > want to
> > > > understand
> > > > >>>>> are
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>> following:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
> > > value in
> > > > capturing
> > > > >>>>> IOPS
> > > > >>>>>>>>>>>>>> policies
> > > > >>>>>>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model
> (e.g.
> > > for
> > > > >>> billing/usage
> > > > >>>>>>>>>>>>>> purposes,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should
> there
> > > be a common
> > > > >>>>>>>>>>>> interface model
> > > > >>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning
> at
> > > runtime?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
> > > conflicting
> > > > >>> provisioned
> > > > >>>>>>>>>>>> IOPS
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> and
> > > storage device
> > > > >>>>>>>>>>>> reconciled?
> > > > >>>>>>>>>>>>>> In
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
> > > believe (and
> > > > billed)
> > > > >>>>> for
> > > > >>>>>>>>>>>> more
> > > > >>>>>>>>>>>>>> IOPS
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> device
> > > has been
> > > > >>> configured
> > > > >>>>>>> to
> > > > >>>>>>>>>>>>>>>> deliver.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
> > > configuration
> > > > >>> between a
> > > > >>>>>>>>>>>> VM and a
> > > > >>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
> > > later
> > > > modification
> > > > >>> to
> > > > >>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>> device
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> inconsistency.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2,
> 2013,
> > > at 8:38 PM,
> > > > >>> Mike
> > > > >>>>>>>>>>>> Tutkowski
> > > > >>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > mike.tutkowski@solidfire.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe
> Wei's
> > > feature deals
> > > > >>> with
> > > > >>>>>>>>>>>>>>>> controlling
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> hypervisor
> > > side.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature
> is
> > > focused on
> > > > >>>>>>> controlling
> > > > >>>>>>>>>>>> IOPS
> > > > >>>>>>>>>>>>>>>> from
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
> > > helps. :)
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun
> 2,
> > > 2013 at 6:35
> > > > PM,
> > > > >>>>>>> John
> > > > >>>>>>>>>>>>>> Burwell
> > > > >>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My
> opinion is
> > > that no
> > > > features
> > > > >>>>>>>>>>>> should be
> > > > >>>>>>>>>>>>>>>>>>> merged
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
> have
> > > been resolved
> > > > and
> > > > >>> it
> > > > >>>>>>> is
> > > > >>>>>>>>>>>> ready
> > > > >>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>> turn
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops
> vs
> > > discrete
> > > > >>> read/write
> > > > >>>>>>>>>>>> ops issue
> > > > >>>>>>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I
> don't
> > > think this
> > > > >>> criteria
> > > > >>>>>>>>>>>> has been
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how
> > > does this work
> > > > >>>>>>>>>>>>>>>> intersect/compliment
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>> https://reviews.apache.org/r/11479/
> > > > >>>>>>>>>>>> )?
> > > > >>>>>>>>>>>>>> As I
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
> involves
> > > provisioned
> > > > >>> IOPS. I
> > > > >>>>>>>>>>>> would
> > > > >>>>>>>>>>>>>>>> like
> > > > >>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > where provisioned
> > > > >>> IOPS
> > > > >>>>> in
> > > > >>>>>>>>>>>> KVM and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> incompatible.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1,
> 2013,
> > > at 6:47 AM,
> > > > >>> Wei
> > > > >>>>>>>>>>>> ZHOU <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will
> > > change it next
> > > > >>> week.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
> > > Wido den Hollander
> > > > <
> > > > >>>>>>>>>>>>>> wido@widodh.nl
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > 06/01/2013 08:24 AM, Wei
> > > > >>> ZHOU
> > > > >>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I
> have
> > > pushed the
> > > > >>>>>>> features
> > > > >>>>>>>>>>>> into
> > > > >>>>>>>>>>>>>>>>>>> master.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
> > > object thems for
> > > > >>>>>>> technical
> > > > >>>>>>>>>>>>>> reason
> > > > >>>>>>>>>>>>>>>>>>> till
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the
> sake
> > > of clarity I
> > > > just
> > > > >>>>>>> want
> > > > >>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>> mention
> > > > >>>>>>>>>>>>>>>>>>>>>> again
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total
> IOps
> > > to R/W IOps
> > > > >>> asap
> > > > >>>>> so
> > > > >>>>>>>>>>>> that we
> > > > >>>>>>>>>>>>>>>>>>> never
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total
> IOps.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid
> the
> > > groundwork for
> > > > >>> the
> > > > >>>>>>> I/O
> > > > >>>>>>>>>>>>>>>> throttling
> > > > >>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
> > > prevent that we
> > > > create
> > > > >>>>>>>>>>>> legacy from
> > > > >>>>>>>>>>>>>>>> day
> > > > >>>>>>>>>>>>>>>>>>>>>> #1.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> > > Wido den
> > > > Hollander <
> > > > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > 05/31/2013 03:59 PM, John
> > > > >>>>>>> Burwell
> > > > >>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
> > > enhancement must
> > > > to
> > > > >>>>>>>>>>>> discretely
> > > > >>>>>>>>>>>>>>>>>>> support
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> how
> > > it could be
> > > > >>> fixed
> > > > >>>>>>>>>>>> later
> > > > >>>>>>>>>>>>>>>> because
> > > > >>>>>>>>>>>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total
> IOPS
> > > into read
> > > > and
> > > > >>>>>>>>>>>> write.
> > > > >>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > unless/until we
> > > > decided
> > > > >>> to
> > > > >>>>>>>>>>>> break
> > > > >>>>>>>>>>>>>>>>>>> backwards
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei
> > > meant was merging
> > > > it
> > > > >>>>> into
> > > > >>>>>>>>>>>> master
> > > > >>>>>>>>>>>>>>>> now
> > > > >>>>>>>>>>>>>>>>>>> so
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch
> > > and add Read /
> > > > >>> Write
> > > > >>>>>>> IOps
> > > > >>>>>>>>>>>>>> before
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>> 4.2
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > > released with Read
> > > > and
> > > > >>>>>>> Write
> > > > >>>>>>>>>>>>>> instead
> > > > >>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
> > > make the May 31st
> > > > >>>>>>> feature
> > > > >>>>>>>>>>>>>> freeze
> > > > >>>>>>>>>>>>>>>>>>> date.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other
> > > threads) then it
> > > > >>> won't
> > > > >>>>>>> be
> > > > >>>>>>>>>>>>>>>> necessary
> > > > >>>>>>>>>>>>>>>>>>>>>> to do
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > > completely agree that
> > > > >>>>> there
> > > > >>>>>>>>>>>> is no
> > > > >>>>>>>>>>>>>>>>>>>>>> association
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31,
> > > 2013, at 9:51 AM,
> > > > >>> Wido
> > > > >>>>>>>>>>>> den
> > > > >>>>>>>>>>>>>>>>>>> Hollander <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > 05/31/2013 03:13 PM, Wei
> > > > >>> ZHOU
> > > > >>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> Good
> > > question.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
> > > about at the
> > > > >>>>> beginning.
> > > > >>>>>>>>>>>>>> Finally I
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference
> of
> > > read and write
> > > > >>>>>>> mainly
> > > > >>>>>>>>>>>>>> because
> > > > >>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
> > > difference of sent
> > > > >>> and
> > > > >>>>>>>>>>>> received
> > > > >>>>>>>>>>>>>>>>>>> bytes as
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > > reasoning seems odd.
> > > > >>>>>>>>>>>> Networking and
> > > > >>>>>>>>>>>>>>>> disk
> > > > >>>>>>>>>>>>>>>>>>> I/O
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is
> > > much more
> > > > >>> expensive
> > > > >>>>> in
> > > > >>>>>>>>>>>> most
> > > > >>>>>>>>>>>>>>>>>>>>>> situations
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Implementing
> > > it will be some
> > > > >>>>>>>>>>>> copy-paste
> > > > >>>>>>>>>>>>>>>> work.
> > > > >>>>>>>>>>>>>>>>>>> It
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> implemented
> > > in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days.
> For
> > > the deadline
> > > > of
> > > > >>>>>>>>>>>> feature
> > > > >>>>>>>>>>>>>>>> freeze,
> > > > >>>>>>>>>>>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>>>>>> will
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
> > > needed.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's
> a
> > > feature we
> > > > >>> can't
> > > > >>>>>>>>>>>> miss. But
> > > > >>>>>>>>>>>>>>>> if
> > > > >>>>>>>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> we
> > > have to make sure
> > > > we
> > > > >>>>>>> don't
> > > > >>>>>>>>>>>>>> release
> > > > >>>>>>>>>>>>>>>>>>> with
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3,
> that
> > > will confuse
> > > > >>>>>>> users.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> > > Wido den
> > > > Hollander <
> > > > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > > 05/30/2013 06:03 PM, Wei
> > > > >>> ZHOU
> > > > >>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would
> like to
> > > merge
> > > > >>>>>>>>>>>> disk_io_throttling
> > > > >>>>>>>>>>>>>>>>>>> branch
> > > > >>>>>>>>>>>>>>>>>>>>>> into
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
> > > object, I will
> > > > merge
> > > > >>>>>>> into
> > > > >>>>>>>>>>>> master
> > > > >>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>> 48
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> purpose
> > > is :
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > > machines are running
> > > > >>> on
> > > > >>>>>>> the
> > > > >>>>>>>>>>>> same
> > > > >>>>>>>>>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share
> strage).
> > > Because of
> > > > the
> > > > >>>>> rate
> > > > >>>>>>>>>>>>>>>> limitation
> > > > >>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM
> has
> > > large disk
> > > > >>> operation,
> > > > >>>>>>> it
> > > > >>>>>>>>>>>> may
> > > > >>>>>>>>>>>>>>>> affect
> > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs
> > > running on the
> > > > same
> > > > >>>>>>>>>>>> storage
> > > > >>>>>>>>>>>>>>>> device.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> neccesary
> > > to set the
> > > > >>>>> maximum
> > > > >>>>>>>>>>>> rate
> > > > >>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>> limit
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at
> the
> > > code I see
> > > > you
> > > > >>>>> make
> > > > >>>>>>>>>>>> no
> > > > >>>>>>>>>>>>>>>>>>> difference
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and
> > > libvirt support
> > > > >>> setting
> > > > >>>>>>>>>>>> both a
> > > > >>>>>>>>>>>>>>>>>>> different
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> which
> > > could benefit a
> > > > >>> lot of
> > > > >>>>>>>>>>>> users.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
> > > strange, in the
> > > > >>> polling
> > > > >>>>>>>>>>>> side you
> > > > >>>>>>>>>>>>>>>>>>>>>> collect
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but
> on
> > > the throttling
> > > > >>> side
> > > > >>>>>>>>>>>> you only
> > > > >>>>>>>>>>>>>> go
> > > > >>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps
> are
> > > usually much
> > > > >>> more
> > > > >>>>>>>>>>>> expensive
> > > > >>>>>>>>>>>>>>>>>>> then
> > > > >>>>>>>>>>>>>>>>>>>>>> Read
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid
> > > use-case where
> > > > >>> that
> > > > >>>>>>> an
> > > > >>>>>>>>>>>> admin
> > > > >>>>>>>>>>>>>>>>>>> would
> > > > >>>>>>>>>>>>>>>>>>>>>> set
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs
> Read
> > > IOps.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this
> only
> > > supports KVM
> > > > >>> at
> > > > >>>>>>> this
> > > > >>>>>>>>>>>>>> point I
> > > > >>>>>>>>>>>>>>>>>>>>>> think
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at
> > > least have the
> > > > >>>>>>>>>>>> mechanism in
> > > > >>>>>>>>>>>>>>>> place
> > > > >>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> implementing
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
> > > would be a lot of
> > > > >>>>> work.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a
> hypervisor
> > > doesn't
> > > > >>> support
> > > > >>>>>>>>>>>> setting
> > > > >>>>>>>>>>>>>>>>>>> different
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you
> can
> > > always sum
> > > > both
> > > > >>> up
> > > > >>>>>>>>>>>> and set
> > > > >>>>>>>>>>>>>>>> that
> > > > >>>>>>>>>>>>>>>>>>> as
> > > > >>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> > > explain why you
> > > > >>>>>>> implemented
> > > > >>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>> this
> > > > >>>>>>>>>>>>>>>>>>> way?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> feature
> > > includes:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> > > maximum rate of
> > > > >>> VMs
> > > > >>>>>>> (in
> > > > >>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> configuration)
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change
> the
> > > maximum rate
> > > > of
> > > > >>>>> VMs
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit
> the
> > > disk rate
> > > > (total
> > > > >>>>> bps
> > > > >>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>> iops)
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> > > > >>>>>>>>>>>>>> https://issues.apache.org/****
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> issues.apache.org/****
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > jira/browse/CLOUDSTACK-1192<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > 1192>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <ht**tps://
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> 1192<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > 1192<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
> > > update later) :
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>
> > > >
> > >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>
> > > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > > >>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>
> > > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > VM+Disk+IO+Throttling<https://
> > > > >>>>>>> ****
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > cwiki.apache.org/confluence/****
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>
> > >
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>
> > > > >>>>>
> > > > >>>
> > > >
> > >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > rottling
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge
> check
> > > list :-
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
> > > check the branch's
> > > > >>> RAT
> > > > >>>>>>>>>>>> execution
> > > > >>>>>>>>>>>>>>>>>>>>>> success?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are
> there
> > > new dependencies
> > > > >>>>>>>>>>>> introduced?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> > > automated testing
> > > > (unit
> > > > >>>>> and
> > > > >>>>>>>>>>>>>>>>>>> integration)
> > > > >>>>>>>>>>>>>>>>>>>>>> is
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests
> are
> > > added.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> > > testing has been done
> > > > >>> to
> > > > >>>>>>>>>>>> check for
> > > > >>>>>>>>>>>>>>>>>>>>>> potential
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> > > bytes rate and
> > > > >>> IOPS
> > > > >>>>>>>>>>>> rate on
> > > > >>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
> > > operations, including
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy,
> stop,
> > > start, reboot,
> > > > >>>>>>>>>>>> destroy,
> > > > >>>>>>>>>>>>>>>> expunge.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3)
> Volume
> > > operations,
> > > > >>> including
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > > Detach
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review
> the
> > > code, you can
> > > > >>> try
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > c30057635d04a2396f84c588127d7e******be42e503a7
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> f2e5591b710d04cc86815044f5823e******73a4a58944
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best
> regards,
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>
> > > >
> > >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>
> > > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > > >>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>
> > > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > VM+Disk+IO+Throttling<https://
> > > > >>>>>>> ****
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > cwiki.apache.org/confluence/****
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>
> > >
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>
> > > > >>>>>
> > > > >>>
> > > >
> > >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > > rottling
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > > > >>> refs/heads/disk_io_throttling
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > 1301
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <ht**tps://
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-
> 1301<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > 1301>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <ht**tps://
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-
> 1301<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > 1301<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <ht**tps://
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-
> 2071
> > > > >>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > 2071
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > > 2071
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > > 2071
> > > > >>>>>>>>>>>>>> <
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > > 2071>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> <h**ttps://
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > (**CLOUDSTACK-1301
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM
> Disk
> > > I/O
> > > > Throttling)
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > > Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > > CloudStack Developer,
> > > > >>>>>>>>>>>> SolidFire
> > > > >>>>>>>>>>>>>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > > >>> mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o:
> 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing
> the
> > > way the world
> > > > >>> uses
> > > > >>>>>>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> > > Developer,
> > > > >>>>> SolidFire
> > > > >>>>>>>>>>>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> > > the world uses
> > > > >>> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > http://solidfire.com/solution/overview/?video=play
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack
> > > Developer,
> > > > >>> SolidFire
> > > > >>>>>>>>>>>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way
> > > the world uses
> > > > the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > Developer,
> > > > >>> SolidFire
> > > > >>>>>>>>>>>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> > > world uses the
> > > > >>>>>>> cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > Developer,
> > > > SolidFire
> > > > >>>>>>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the
> > > world uses the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>>>>>>>>>>>>>>>>
> > > http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > > Developer, SolidFire
> > > > >>>>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > > world uses the
> > > > >>> cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > > SolidFire
> > > > >>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> > > uses the
> > > > >>> cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > > SolidFire
> > > > >>> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world
> > > uses the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > SolidFire
> > > > Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> > > the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > SolidFire Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses
> > > the cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > > SolidFire Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > > Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > > cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > > Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > > Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > cloud<
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > > Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > >>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>>>> http://solidfire.com/solution/overview/?video=play
> > > > >>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> > > > >>>>>>>>>>>>>>>>>>>
> > > http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > > > >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > >>>>>>>>>>>>>>>>>>>> cloud<
> > > > >>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > > >>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > >>>>>>>>>>>>>>>>>
> > > cloud<http://solidfire.com/solution/overview/?video=play
> > > > >
> > > > >>>>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>>>> Advancing the way the world uses the
> > > > >>>>>>>>>>>>>>>
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>>>> Advancing the way the world uses the
> > > > >>>>>>>>>>>>>
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> --
> > > > >>>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>>> Advancing the way the world uses the cloud<
> > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> --
> > > > >>>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>>> Advancing the way the world uses the cloud<
> > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>>> *(tm)*
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> --
> > > > >>>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>>> o: 303.746.7302
> > > > >>>>>>>>> Advancing the way the world uses the cloud<
> > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>>> *(tm)*
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> *Mike Tutkowski*
> > > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>>> o: 303.746.7302
> > > > >>>>>>>> Advancing the way the world uses the cloud<
> > > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>>> *(tm)*
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> --
> > > > >>>>>>> *Mike Tutkowski*
> > > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>>>>> e: mike.tutkowski@solidfire.com
> > > > >>>>>>> o: 303.746.7302
> > > > >>>>>>> Advancing the way the world uses the
> > > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > > >>>>>>> *(tm)*
> > > > >>>>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> --
> > > > >>>> *Mike Tutkowski*
> > > > >>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >>>> e: mike.tutkowski@solidfire.com
> > > > >>>> o: 303.746.7302
> > > > >>>> Advancing the way the world uses the
> > > > >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > > >>>> *(tm)*
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> *Mike Tutkowski*
> > > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >> e: mike.tutkowski@solidfire.com
> > > > >> o: 303.746.7302
> > > > >> Advancing the way the world uses the cloud<
> > > > http://solidfire.com/solution/overview/?video=play>
> > > > >> *(tm)*
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *Mike Tutkowski*
> > > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > > e: mike.tutkowski@solidfire.com
> > > > > o: 303.746.7302
> > > > > Advancing the way the world uses the
> > > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > > *(tm)*
> > > >
> > > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *(tm)*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
As we only have three weeks until feature freeze, we should come to a
consensus on this design point as soon as possible.
Right now, if the storage framework asks my driver it is is managed, it
will say 'yes.' This means the framework will tell the driver to perform
its management activities. This then means the driver will call into the
host (it doesn't know which hypervisor, by the way) to perform the activity
of, say, creating an SR on XenServer or a datastore on ESX.
The driver doesn't know which hypervisor it's talking to, it just sends a
message to the host to perform the necessary pre-attach work.
On Fri, Jun 7, 2013 at 3:14 PM, Edison Su <Ed...@citrix.com> wrote:
>
>
> > -----Original Message-----
> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> > Sent: Friday, June 07, 2013 1:14 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [MERGE] disk_io_throttling to MASTER
> >
> > Hi John,
> >
> > How's about this:
> >
> > The driver can implement an isManaged() method. The VolumeManagerImpl
> > can
> > call into the driver to see if its managed. If it is, the
> VolumeManagerImpl
> > (which is responsible for calling into the hypervisor to attach the disk)
> > can call into the hypervisor to create the necessary hypervisor data
> > structure (ex. for XenServer, a storage repository).
>
> The problem here is that storage vendor may work differently with
> hypervisor, for example, SolidFire wants a SR per LUN, while maybe other
> vendor wants to totally bypass hypervisor, and assign the LUN directly to
> VM instance, see the discuss(
> http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com%3E
> ).
> So I would let storage provider to implement attach disk to VM, instead of
> implemented by cloudstack itself.
>
>
> >
> > If that's what you're going for, that works for me. By the way, Edison's
> > default storage plug-in (which handles the default storage behavior in
> > CloudStack (ex. how pre 4.2 works)) does include code that talks to
> > hypervisors. You might want to contact him and inform him of your
> concerns
> > or that logic (as is) will make it to production.
> >
> > Please let me know if what I wrote in above (for my solution) is OK with
> > you. :)
> >
> > Thanks!
> >
> >
> > On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com> wrote:
> >
> > > Mike,
> > >
> > > Please see my responses in-line below.
> > >
> > > Thanks,
> > > -John
> > >
> > > On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> > > wrote:
> > >
> > > > Hey John,
> > > >
> > > > I still have a bit more testing I'd like to do before I build up a
> patch
> > > > file, but this is the gist of what I've done:
> > > >
> > > > * During a volume-attach operation, after VolumeManagerImpl tells
> > > > VolumeServiceImpl to have the driver create a volume, I have
> > > > VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> > managed.
> > > > If it is managed, VolumeServiceImpl has the driver perform whatever
> > > > activity is required. In my case, this includes sending a message to
> the
> > > > host where the VM is running to have, say XenServer, add a storage
> > > > repository (based on the IP address of the SAN, the IQN of the SAN
> > > volume,
> > > > etc.) and a single VDI (the VDI consumes all of the space it can on
> the
> > > > storage repository). After this, the normal attach-volume message is
> sent
> > > > to the host by VolumeManagerImpl.
> > >
> > > There should be **no** code from a storage driver to a hypervisor. I
> > > apologize for the repetition, but we simply can not have hypervisor
> > > specific code in the storage layer. The circular dependencies between
> the
> > > two layers are not sustainable in the long term. Either the
> VirtualManager
> > > or Xen hypervisor plugin needs to be refactored/modified to coordinate
> > > volume creation and then populating the SR. Ideally, we can
> generalize the
> > > process flow for attaching volumes such that the Xen hypervisor plugin
> > > would only implement callbacks to perform the attach action and create
> > the
> > > structure and SR. To my mind, the SolidFire driver should only be
> > > allocating space and providing information about contents (e.g. space
> > > available, space consumed, streams to a URI, file handle for a URI,
> etc)
> > > and capabilities.
> > >
> > >
> > > >
> > > > * The reverse is performed for a detach-volume command.
> > > >
> > > > * Right now I simply "return true;" for isManaged() in my driver.
> > > Edison's
> > > > default driver simply does a "return false;". We could add a new
> > > parameter
> > > > to the createStoragePool API command, if we want, to remove the
> > > hard-coded
> > > > return values in the drivers (although my driver will probably just
> > > ignore
> > > > this parameter and always return true since it wouldn't make sense
> for it
> > > > to ever return false). We'd need another column in the storage_pool
> > table
> > > > to store this value.
> > >
> > > Yes, I think we should have a parameter added to the createStoragePool
> > > surfaced to the HTTP API that allows DataStores to be configured for
> > > management when their underlying drivers support it. To simplify
> things,
> > > this flag should only be mutable when the DataStore is created. It
> would be
> > > a bit crazy to take a DataStore from managed to unmanaged after
> creation.
> > >
> > > >
> > > > Sound like I'm in sync with what you were thinking?
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > > mike.tutkowski@solidfire.com
> > > >> wrote:
> > > >
> > > >> I agree, John. Just wanted to point out that I have a working GUI
> for
> > > you
> > > >> to review (in that document), if you'd like to check it out.
> > > >>
> > > >> Thanks!
> > > >>
> > > >>
> > > >> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com>
> > > wrote:
> > > >>
> > > >>> Mike,
> > > >>>
> > > >>> I would like the UIs of two features reviewed together to ensure
> > > >>> consistency across the concepts of hypervisor throttled IOPs and
> > > >>> storage device provisioned IOPs. I see the potential for
> confusion,
> > > >>> and I think a side-by-side Ui review of these features will help
> > > >>> minimize any potential confusion.
> > > >>>
> > > >>> As I mentioned, the term reconciliation issue will work itself if
> it
> > > >>> is acceptable that a VM is only permitted utilize hypervisor
> throttled
> > > >>> IOPs or storage provisioned IOPs.
> > > >>>
> > > >>> Thanks,
> > > >>> -John
> > > >>>
> > > >>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > > >>> <mi...@solidfire.com> wrote:
> > > >>>
> > > >>>> Hi John,
> > > >>>>
> > > >>>> Yeah, when you get a chance, refer to the Google doc I sent to you
> > the
> > > >>>> other day to see how the GUI looks for provisioned storage IOPS.
> > > >>>>
> > > >>>> Several months ago, I put this topic out on the e-mail list and we
> > > >>> decided
> > > >>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering
> dialog.
> > > >>>> Other storage vendors are coming out with QoS, so they should be
> > able
> > > to
> > > >>>> leverage this GUI going forward (even if they, say, only use Max
> > > IOPS).
> > > >>>> These fields are optional and can be ignored for storage that
> does not
> > > >>>> support provisioned IOPS. Just like the Disk Size field, the
> admin can
> > > >>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
> > > >>>>
> > > >>>> I'm OK if we do an either/or model (either Wei's feature or mine,
> as
> > > is
> > > >>>> decided by the admin).
> > > >>>>
> > > >>>> I'm not sure what we can do about these two features expressing
> the
> > > >>> speed
> > > >>>> in different terms. I've never seen a SAN express the IOPS for
> QoS in
> > > >>> any
> > > >>>> way other than total IOPS (i.e. not broken in into read/write
> IOPS).
> > > >>>>
> > > >>>> Thanks
> > > >>>>
> > > >>>>
> > > >>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com>
> > > >>> wrote:
> > > >>>>
> > > >>>>> Wei,
> > > >>>>>
> > > >>>>> We have been down the rabbit hole a bit on the Storage/Hypervisor
> > > layer
> > > >>>>> separation, but we still need to reconcile the behavior of
> hypervisor
> > > >>>>> throttled I/O and storage provisioned IOPS. I see the following
> > > issues
> > > >>>>> outstanding:
> > > >>>>>
> > > >>>>> 1. Hypervisor throttled IOPS are expressed as discrete read/write
> > > >>> values
> > > >>>>> where as storage provisioned IOPS are expressed as total IOPS.
> > > >>>>> 2. How do we handle VMs with throttled IOPS attached to storage
> > > volumes
> > > >>>>> with provisioned IOPS?
> > > >>>>> 3. How should usage data be captured for throttled and
> provisioned
> > > IOPS
> > > >>>>> that will permit providers to discriminate these guaranteed
> > > operations
> > > >>> in
> > > >>>>> the event they want to bill for it differently?
> > > >>>>> 4. What is the user experience for throttled and provisioned IOPS
> > > that
> > > >>>>> minimizes confusion of these concepts?
> > > >>>>>
> > > >>>>> My thinking is that a VM can have either utilize hypervisor
> throttled
> > > >>> IOPS
> > > >>>>> or storage provisioned IOPS. This policy would neatly solve
> items 1
> > > >>> and 2.
> > > >>>>> Since the two facilities would not be permitted to operate
> together,
> > > >>> they
> > > >>>>> do not need to be semantically compatible. I think item 3 can be
> > > >>> resolved
> > > >>>>> with an additional flag or two on the usage records. As for
> Item 4,
> > > I
> > > >>> am
> > > >>>>> not familiar with how these two enhancements are depicted in the
> > user
> > > >>>>> interface. I think we need to review the UI enhancements for
> both
> > > >>>>> enhancements and ensure they are consistent.
> > > >>>>>
> > > >>>>> Do these solutions make sense?
> > > >>>>>
> > > >>>>> Thanks,
> > > >>>>> -John
> > > >>>>>
> > > >>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com>
> > wrote:
> > > >>>>>
> > > >>>>>> John and Mike,
> > > >>>>>>
> > > >>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> > > >>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-
> > 2875 ) this
> > > >>> week.
> > > >>>>>> I will start to develop on iops/bps changes tomorrow, and ask
> for
> > > >>> second
> > > >>>>>> merge request after testing.
> > > >>>>>>
> > > >>>>>> -Wei
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > > >>>>>>
> > > >>>>>>> I believe I understand where you're going with this, John, and
> > have
> > > >>> been
> > > >>>>>>> re-working this section of code today.
> > > >>>>>>>
> > > >>>>>>> I should be able to run it by you tomorrow.
> > > >>>>>>>
> > > >>>>>>> Thanks for the comments,
> > > >>>>>>> Mike
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> > <jb...@basho.com>
> > > >>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Mike,
> > > >>>>>>>>
> > > >>>>>>>> See my responses in-line below.
> > > >>>>>>>>
> > > >>>>>>>> Thanks,
> > > >>>>>>>> -John
> > > >>>>>>>>
> > > >>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > > >>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>> Hi John,
> > > >>>>>>>>
> > > >>>>>>>> Thanks for the response.
> > > >>>>>>>>
> > > >>>>>>>> "I am fine with the VolumeManager determining whether or
> > not a
> > > >>> Volume
> > > >>>>> is
> > > >>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
> > > >>> isManaged
> > > >>>>>>>> method), and asking the device driver to allocate resources
> for
> > > the
> > > >>>>>>> volume
> > > >>>>>>>> if it is managed."
> > > >>>>>>>>
> > > >>>>>>>> Are you thinking you'd like to see an isManaged() method
> > added to
> > > >>> the
> > > >>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then the
> > > >>> storage
> > > >>>>>>>> framework could call the manage() (or whatever name) method
> > (which
> > > >>>>> would
> > > >>>>>>> be
> > > >>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and this
> > > would
> > > >>>>> call
> > > >>>>>>>> into a new method in the hypervisor code to create, say on
> > > >>> XenServer,
> > > >>>>> an
> > > >>>>>>> SR?
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> I would like to see canBeManaged() : boolean on
> > DataStoreDriver.
> > > >>> Since
> > > >>>>>>>> the notion of Volumes only pertains to primary storage, I
> would
> > > add
> > > >>> a
> > > >>>>>>>> allocateStorage and deallocateStorage (Storage is a straw man
> > term
> > > >>> --
> > > >>>>>>>> something other than volume) methods to
> > > >>>>> allocate/create/deallocate/delete
> > > >>>>>>>> underlying storage. To my mind, managed is a mutable property
> > of
> > > >>>>>>> DataStore
> > > >>>>>>>> which can be enabled if/when the underlying DataStoreDriver
> > can be
> > > >>>>>>> managed.
> > > >>>>>>>> This approach allows operators to override manageability of
> > > devices.
> > > >>>>>>>>
> > > >>>>>>>> In terms of orchestration/process flow for SR, the Xen plugin
> > > would
> > > >>> be
> > > >>>>>>>> responsible for composing DataStore/Volume methods to
> > create any
> > > >>>>>>>> directories or files necessary for the SR. There should be no
> > > >>>>>>> dependencies
> > > >>>>>>>> from the Storage to the Hypervisor layer. As I said earlier,
> such
> > > >>>>>>> circular
> > > >>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Just want to make sure I'm on the same page with you.
> > > >>>>>>>>
> > > >>>>>>>> Thanks again, John
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> > <jb...@basho.com>
> > > >>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Mike,
> > > >>>>>>>>>
> > > >>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> > > supports n
> > > >>>>>>>>> devices types with each specific behaviors of m hypervisors.
> > > Such
> > > >>> a
> > > >>>>>>>>> scenario will create an unmaintainable and untestable beast.
> > > >>>>>>> Therefore, my
> > > >>>>>>>>> thoughts and recommendations are driven to evolve the
> > Storage
> > > layer
> > > >>>>>>> towards
> > > >>>>>>>>> this separation.
> > > >>>>>>>>>
> > > >>>>>>>>> I am fine with the VolumeManager determining whether or
> > not a
> > > >>> Volume
> > > >>>>> is
> > > >>>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
> > > >>>>> isManaged
> > > >>>>>>>>> method), and asking the device driver to allocate resources
> for
> > > the
> > > >>>>>>> volume
> > > >>>>>>>>> if it is managed. Furthermore, the device driver needs to
> > > indicate
> > > >>>>>>> whether
> > > >>>>>>>>> or not it supports management operations. Finally, I think
> we
> > > >>> need to
> > > >>>>>>>>> provide the ability for an administrator to elect to have
> > > something
> > > >>>>>>> that is
> > > >>>>>>>>> manageable be unmanaged (i.e. the driver is capable managing
> > the
> > > >>>>> device,
> > > >>>>>>>>> but the administrator has elected to leave it unmanaged).
> > > >>>>>>>>>
> > > >>>>>>>>> Creation of a structure on the volume should be done in the
> > Xen
> > > >>>>>>>>> hypervisor module using methods exposed by the Storage
> > layer to
> > > >>>>> perform
> > > >>>>>>>>> low-level operations (e.g. make directories, create a file,
> etc).
> > > >>>>> This
> > > >>>>>>>>> structure is specific to the operation of the Xen
> hypervisor, as
> > > >>> such,
> > > >>>>>>>>> should be confined to its implementation. From my
> > perspective,
> > > >>>>> nothing
> > > >>>>>>> in
> > > >>>>>>>>> the Storage layer should be concerned with content. From its
> > > >>>>>>> perspective,
> > > >>>>>>>>> structure and data are opaque. It provides the means to
> query
> > > the
> > > >>>>> data
> > > >>>>>>> to
> > > >>>>>>>>> support the interpretation of the content by higher-level
> > layers
> > > >>> (e.g.
> > > >>>>>>>>> Hypervisors). To my mind, attach should be a composition of
> > > >>>>> operations
> > > >>>>>>>>> from the Storage layer that varies based on the Volume
> > storage
> > > >>>>> protocol
> > > >>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > > >>>>>>>>>
> > > >>>>>>>>> Thanks,
> > > >>>>>>>>> -John
> > > >>>>>>>>>
> > > >>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > > >>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>> Hi John,
> > > >>>>>>>>>
> > > >>>>>>>>> Alternatively to the way the attach logic is implemented in
> my
> > > >>> patch,
> > > >>>>> we
> > > >>>>>>>>> could do the following:
> > > >>>>>>>>>
> > > >>>>>>>>> Leave the attach logic in the agent code alone. In
> > > >>> VolumeManagerImpl
> > > >>>>> we
> > > >>>>>>>>> create an AttachVolumeCommand and send it to the
> > hypervisor.
> > > Before
> > > >>>>> this
> > > >>>>>>>>> command is sent, we could check to see if we're dealing with
> > > >>> Dynamic
> > > >>>>> (or
> > > >>>>>>>>> whatever we want to call it) storage and - if so - send a
> "Create
> > > >>> SR"
> > > >>>>>>>>> command to the hypervisor. If this returns OK, we would then
> > > >>> proceed
> > > >>>>> to
> > > >>>>>>> the
> > > >>>>>>>>> AttachVolumeCommand, as usual.
> > > >>>>>>>>>
> > > >>>>>>>>> This way the attach logic remains the same and we just add
> > > another
> > > >>>>>>>>> command to the agent code that is called for this particular
> > type
> > > >>> of
> > > >>>>>>>>> storage.
> > > >>>>>>>>>
> > > >>>>>>>>> What do you think?
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > > >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hey John,
> > > >>>>>>>>>>
> > > >>>>>>>>>> I created a document for a customer today that outlines how
> > the
> > > >>>>> plug-in
> > > >>>>>>>>>> works from a user standpoint. This will probably be of use
> to
> > > >>> you, as
> > > >>>>>>> well,
> > > >>>>>>>>>> as you perform the code review.
> > > >>>>>>>>>>
> > > >>>>>>>>>> I have shared this document with you (you should have
> > received
> > > >>> that
> > > >>>>>>>>>> information in a separate e-mail).
> > > >>>>>>>>>>
> > > >>>>>>>>>> Talk to you later!
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > > >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Oh, OK, that sounds really good, John.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > > jburwell@basho.com
> > > >>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts, but
> > > want
> > > >>> to
> > > >>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > > >>>>>>> disk_io_throttle,
> > > >>>>>>>>>>>> and
> > > >>>>>>>>>>>> object_store branches. I hope to collect them in a
> > coherent
> > > >>> form
> > > >>>>>>>>>>>> tomorrow (5 June 2013).
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>> -John
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > > >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the
> notion
> > > of
> > > >>> a
> > > >>>>>>>>>>>> managed
> > > >>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the
> > notion
> > > >>> of
> > > >>>>>>>>>>>>> provisioned storage IOPS.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> The storage framework that leverages the plug-in was
> > > >>> incomplete,
> > > >>>>> so
> > > >>>>>>>>>>>> I had
> > > >>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I appreciate all the time you've been spending on this.
> :)
> > Do
> > > >>> you
> > > >>>>>>>>>>>> have a
> > > >>>>>>>>>>>>> recommendation as to how we should accomplish what
> > you're
> > > >>> looking
> > > >>>>>>>>>>>> for?
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > > >>> jburwell@basho.com>
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the
> notion
> > > of
> > > >>> a
> > > >>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> I want to see a separation of the management
> > capabilities
> > > >>> (i.e.
> > > >>>>>>> can
> > > >>>>>>>>>>>> a
> > > >>>>>>>>>>>>>> device be managed/does an operator want it managed
> > by
> > > >>> CloudStack)
> > > >>>>>>>>>>>> from the
> > > >>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
> > semantic
> > > >>> that
> > > >>>>>>>>>>>> will
> > > >>>>>>>>>>>>>> allow any type of storage device to be managed. I also
> > want
> > > >>> to
> > > >>>>>>> make
> > > >>>>>>>>>>>>>> progress on decoupling the storage types from the
> > hypervisor
> > > >>>>>>>>>>>> definitions.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > > >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> No problem. Answers are below in red.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > > >>>>> jburwell@basho.com
> > > >>>>>>>>
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Could you please answer the following questions for
> > me
> > > with
> > > >>>>>>>>>>>> regards to
> > > >>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN
> > > volumes?
> > > >>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
> > > >>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and
> > CloudStack
> > > >>>>>>>>>>>> Volumes?
> > > >>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Are the LUN(s) created by the management server or
> > > >>> externally
> > > >>>>> by
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume
> > (same
> > > as a
> > > >>>>> SAN
> > > >>>>>>>>>>>> LUN) is
> > > >>>>>>>>>>>>>>> created by the management server (via the plug-in)
> > the
> > > first
> > > >>>>> time
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but
> still
> > > >>> want
> > > >>>>> to
> > > >>>>>>>>>>>> use a
> > > >>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already today
> > > (prior
> > > >>> to
> > > >>>>>>>>>>>> 4.2). The
> > > >>>>>>>>>>>>>>> admin manually creates the SAN volume and - in this
> > case -
> > > >>>>>>>>>>>> multiple VMs
> > > >>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>> data disks can share this SAN volume. While you can do
> > this
> > > >>>>>>> today,
> > > >>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Are the SAN volumes by the management server or
> > externally
> > > >>> by
> > > >>>>>>> the
> > > >>>>>>>>>>>>>> operator?
> > > >>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes
> > are
> > > >>>>>>> completely
> > > >>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>> by the management server (via the plug-in). There is
> > no
> > > admin
> > > >>>>>>>>>>>>>> interaction.
> > > >>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume
> > and a
> > > >>>>>>> CloudStack
> > > >>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>> which is necessary for any storage vendor that
> > supports
> > > true
> > > >>>>> QoS.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> I would like to clarify how these pieces are related
> > and
> > > >>>>>>> expected
> > > >>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>> operate.
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs
> > with
> > > >>>>>>>>>>>> provisioned
> > > >>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a single
> > SAN
> > > >>> volume
> > > >>>>>>>>>>>> (LUN) to a
> > > >>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have multiple
> > > >>> CloudStack
> > > >>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume house
> > more
> > > >>> than
> > > >>>>>>> one
> > > >>>>>>>>>>>>>>>>> CloudStack volume, then can do that today without
> > any of
> > > my
> > > >>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>> code.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume for
> > > >>>>> CloudStack's
> > > >>>>>>>>>>>> use
> > > >>>>>>>>>>>>>> onto
> > > >>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> > > >>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me --
> > > concepts
> > > >>>>> from
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today
> only
> > > used
> > > >>>>> (as
> > > >>>>>>>>>>>> far as
> > > >>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>> know)
> > > >>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not
> > PreSetup an
> > > >>> SR).
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> > VMware, it
> > > >>> uses
> > > >>>>>>> the
> > > >>>>>>>>>>>> vmfs
> > > >>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM,
> > it uses
> > > >>> the
> > > >>>>>>>>>>>>>>>>>>> SharedMountPoint
> > > >>>>>>>>>>>>>>>>>>>> type.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here
> > was to
> > > >>> make a
> > > >>>>>>>>>>>> new type
> > > >>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison
> > added
> > > >>> into
> > > >>>>>>> the
> > > >>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more specificy,
> > though:
> > > >>>>>>>>>>>> Dynamic_iSCSI
> > > >>>>>>>>>>>>>>>>>>> versus,
> > > >>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski
> > <
> > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or
> > care
> > > that
> > > >>> it
> > > >>>>>>> is
> > > >>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>> used
> > > >>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> > questions
> > > >>> about
> > > >>>>> it
> > > >>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>> storing."
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> > SolidFire
> > > >>> plug-in
> > > >>>>>>>>>>>> needs to
> > > >>>>>>>>>>>>>>>>>>> call
> > > >>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor
> > logic
> > > >>>>> knows
> > > >>>>>>>>>>>> to
> > > >>>>>>>>>>>>>> treat
> > > >>>>>>>>>>>>>>>>>>> it as
> > > >>>>>>>>>>>>>>>>>>>>> such.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint
> > and just
> > > >>>>>>>>>>>> calling it
> > > >>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for the
> > hypervisor
> > > >>>>>>> attach
> > > >>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform the
> > necessary
> > > >>>>>>>>>>>> activities.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
> > > >>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to
> > manage a
> > > >>>>>>> device,
> > > >>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack
> > (e.g. the
> > > >>>>>>> device
> > > >>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>> shared
> > > >>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has chosen
> > to
> > > >>> dedicate
> > > >>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>> portion
> > > >>>>>>>>>>>>>>>>>>>>>> of it
> > > >>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make
> > sense?"
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
> > volume via
> > > >>> the
> > > >>>>>>>>>>>> plug-in
> > > >>>>>>>>>>>>>>>>>>> (before
> > > >>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
> > work unless
> > > >>> the
> > > >>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>> consumes
> > > >>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> > CloudStack, an
> > > >>>>>>> operator
> > > >>>>>>>>>>>>>>>> allocates
> > > >>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it
> > > through
> > > >>> a
> > > >>>>>>>>>>>> LUN. They
> > > >>>>>>>>>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
> > CloudStack,
> > > >>> but
> > > >>>>>>>>>>>> not allow
> > > >>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> > changes, we
> > > >>> would
> > > >>>>>>> be
> > > >>>>>>>>>>>>>> passing
> > > >>>>>>>>>>>>>>>>>>> in a
> > > >>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> > necessary
> > > >>> hypervisor
> > > >>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>> (like
> > > >>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>> SR)
> > > >>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the
> > storage
> > > >>>>>>> framework
> > > >>>>>>>>>>>>>> itself
> > > >>>>>>>>>>>>>>>>>>>>>> detect
> > > >>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from
> > the
> > > >>> hypervisor
> > > >>>>>>>>>>>> side and
> > > >>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to create
> > the
> > > >>> SR
> > > >>>>>>>>>>>> before the
> > > >>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin
> > should
> > > have a
> > > >>>>>>> rich
> > > >>>>>>>>>>>> enough
> > > >>>>>>>>>>>>>>>>>>>>>> interface to storage to determine available
> > for
> > > volume
> > > >>>>>>>>>>>> storage.
> > > >>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>> Xen,
> > > >>>>>>>>>>>>>>>>>>>>>> this interface would allow the interrogation of
> > the
> > > >>>>> device
> > > >>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>> determine the
> > > >>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
> > shouldn't
> > > >>> know
> > > >>>>>>>>>>>> or care
> > > >>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to
> > > answer
> > > >>>>>>>>>>>> questions
> > > >>>>>>>>>>>>>>>>>>> about it
> > > >>>>>>>>>>>>>>>>>>>>>> is storing.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver
> > (which
> > > >>> is a
> > > >>>>>>>>>>>> type of
> > > >>>>>>>>>>>>>>>>>>>>>> so-called
> > > >>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new
> > Primary
> > > >>>>>>>>>>>> Storage shows
> > > >>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that
> > > references
> > > >>> the
> > > >>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>> tag
> > > >>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
> > volume. This
> > > >>> leads
> > > >>>>>>> to
> > > >>>>>>>>>>>> a new
> > > >>>>>>>>>>>>>>>> row
> > > >>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack
> > volume to a
> > > >>> VM
> > > >>>>>>>>>>>> (attach
> > > >>>>>>>>>>>>>>>> disk).
> > > >>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the
> > plug-in
> > > >>> to
> > > >>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN).
> > The
> > > >>> plug-in
> > > >>>>>>>>>>>> also
> > > >>>>>>>>>>>>>>>> updates
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data
> > (like the
> > > >>> IQN of
> > > >>>>>>>>>>>> the SAN
> > > >>>>>>>>>>>>>>>>>>>>>> volume).
> > > >>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> > > CloudStack
> > > >>>>>>>>>>>> volume is in
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
> > volume
> > > >>> will be
> > > >>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>> 'Ready'
> > > >>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the
> > plug-in
> > > >>> code
> > > >>>>>>>>>>>> will not
> > > >>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>> called
> > > >>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and
> > detects
> > > the
> > > >>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a
> > > >>> hypervisor
> > > >>>>>>> data
> > > >>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> > terminology,
> > > >>> volume,
> > > >>>>>>>>>>>> in these
> > > >>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> > allocation on
> > > >>> the
> > > >>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>> correct?"
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean
> > 'SAN
> > > >>> volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can
> > make in
> > > >>>>>>>>>>>> CloudStack, I
> > > >>>>>>>>>>>>>>>>>>> try to
> > > >>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well make
> > sense, but
> > > >>> I'm
> > > >>>>>>>>>>>> having a
> > > >>>>>>>>>>>>>>>> hard
> > > >>>>>>>>>>>>>>>>>>>>>> time
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on
> > this
> > > >>> conversation
> > > >>>>>>>>>>>> as he
> > > >>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I
> > took.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John
> > Burwell <
> > > >>>>>>>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
> > distinct
> > > >>> concepts
> > > >>>>>>> --
> > > >>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In
> > both
> > > >>> cases, we
> > > >>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>> communicating with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
> > > create/delete
> > > >>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>> (Dynamic)
> > > >>>>>>>>>>>>>>>>>>>>>> on the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the
> > volume to
> > > be
> > > >>>>>>>>>>>> volume to be
> > > >>>>>>>>>>>>>>>>>>>>>> managed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To
> > ensure
> > > >>> that
> > > >>>>> we
> > > >>>>>>>>>>>> are in
> > > >>>>>>>>>>>>>>>>>>> sync on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> > definitions,
> > > >>> refers to
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
> > Minimally,
> > > we
> > > >>>>>>> must
> > > >>>>>>>>>>>> be
> > > >>>>>>>>>>>>>> able
> > > >>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to
> > move bits
> > > >>> from
> > > >>>>>>>>>>>> one place
> > > >>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> another,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a
> > > >>> storage
> > > >>>>>>>>>>>> device
> > > >>>>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>> able to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore,
> > we can have
> > > a
> > > >>>>>>>>>>>> unmanaged
> > > >>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we
> > can have a
> > > >>>>>>> managed
> > > >>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable
> > of
> > > >>> allocating
> > > >>>>>>>>>>>> LUNs and
> > > >>>>>>>>>>>>>>>>>>> storing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may
> > be able
> > > >>> to
> > > >>>>>>>>>>>> manage a
> > > >>>>>>>>>>>>>>>>>>> device,
> > > >>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> > unmanaged by
> > > >>>>>>> CloudStack
> > > >>>>>>>>>>>> (e.g.
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> device is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
> > operator has
> > > >>> chosen
> > > >>>>>>> to
> > > >>>>>>>>>>>>>>>> dedicate
> > > >>>>>>>>>>>>>>>>>>>>>> only a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my
> > > reasoning
> > > >>>>> make
> > > >>>>>>>>>>>> sense?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
> > reasonable, it
> > > >>> seems
> > > >>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from
> > > >>> StoragePoolType,
> > > >>>>>>>>>>>> add the
> > > >>>>>>>>>>>>>>>>>>> notion
> > > >>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver
> > that
> > > >>>>> indicates
> > > >>>>>>>>>>>> whether
> > > >>>>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>>>>>> not is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish
> > a
> > > >>> abstraction
> > > >>>>>>>>>>>>>>>> representing a
> > > >>>>>>>>>>>>>>>>>>>>>> physical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is
> > > >>> associated
> > > >>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>> volume.
> > > >>>>>>>>>>>>>>>>>>>>>> With
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
> > drivers can
> > > >>>>>>> declare
> > > >>>>>>>>>>>> which
> > > >>>>>>>>>>>>>>>>>>>>>> protocols they
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a
> > device
> > > managed
> > > >>> by
> > > >>>>>>>>>>>>>> CloudStack,
> > > >>>>>>>>>>>>>>>>>>>>>> utilize the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed by
> > the driver to
> > > >>>>>>> automate
> > > >>>>>>>>>>>>>>>>>>> allocation.
> > > >>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense,
> > then we can
> > > >>> sit
> > > >>>>>>>>>>>> down and
> > > >>>>>>>>>>>>>>>>>>> drill
> > > >>>>>>>>>>>>>>>>>>>>>> down to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the
> > current
> > > iSCSI
> > > >>>>>>> type
> > > >>>>>>>>>>>> and
> > > >>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and
> > create
> > > a
> > > >>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time,
> > the
> > > iSCSI
> > > >>>>>>>>>>>> volume must
> > > >>>>>>>>>>>>>>>>>>> exist
> > > >>>>>>>>>>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated).
> > Future
> > > >>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>> are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created
> > behind the
> > > >>>>>>> scenes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in
> > and
> > > create
> > > >>>>>>>>>>>> Primary
> > > >>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>> based
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete
> > volumes on
> > > >>> its
> > > >>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
> > storage
> > > >>>>>>>>>>>> framework). When
> > > >>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>> user
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was
> > created,
> > > the
> > > >>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>> tells
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After
> > this is
> > > >>> done,
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called.
> > No
> > > >>> hypervisor
> > > >>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was
> > just
> > > >>> created.
> > > >>>>>>> The
> > > >>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike
> > > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should
> > probably
> > > have
> > > >>>>>>>>>>>> defined them
> > > >>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>> front
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-
> > allocated
> > > >>> (ex.
> > > >>>>>>>>>>>> an admin
> > > >>>>>>>>>>>>>>>>>>>>>> creates a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor
> > data
> > > >>> structure
> > > >>>>>>> is
> > > >>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then
> > that
> > > >>>>>>> hypervisor
> > > >>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI)
> > are later
> > > >>>>>>> placed
> > > >>>>>>>>>>>> on
> > > >>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these
> > cases, the
> > > >>>>>>> attach
> > > >>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>> assumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already
> > in place
> > > >>> and
> > > >>>>>>>>>>>> simply
> > > >>>>>>>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to
> > the VM in
> > > >>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
> > > >>>>> pre-allocated.
> > > >>>>>>>>>>>> Instead
> > > >>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a
> > SAN
> > > (not
> > > >>> a
> > > >>>>>>>>>>>> volume on
> > > >>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>> SAN,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
> > structure
> > > >>> must be
> > > >>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>> when an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these
> > types of
> > > >>> volumes
> > > >>>>>>>>>>>> have not
> > > >>>>>>>>>>>>>>>>>>> been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> > structure by an
> > > >>>>> admin.
> > > >>>>>>>>>>>> Once
> > > >>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for
> > this
> > > >>> volume,
> > > >>>>>>> it
> > > >>>>>>>>>>>>>> attaches
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in
> > question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
> > John Burwell
> > > <
> > > >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the
> > Dynamic
> > > type
> > > >>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>> behavior
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask
> > about the
> > > >>>>>>>>>>>> difference.
> > > >>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
> > definition
> > > of
> > > >>> a
> > > >>>>>>>>>>>> Dynamic
> > > >>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike
> > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is
> > > uniquely
> > > >>>>> used
> > > >>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage
> > that is
> > > >>> directly
> > > >>>>>>>>>>>> based on
> > > >>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of
> > going
> > > to
> > > >>> the
> > > >>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that
> > iSCSI
> > > >>> target as
> > > >>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>> does
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> > supported for
> > > >>>>>>>>>>>> XenServer.
> > > >>>>>>>>>>>>>> For
> > > >>>>>>>>>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to
> > the
> > > >>> hypervisor
> > > >>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>> perform
> > > >>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM,
> > John
> > > Burwell
> > > >>> <
> > > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what
> > is the
> > > >>>>> difference
> > > >>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD
> > considered
> > > >>>>>>> Dynamic?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike
> > > Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
> > defined in
> > > the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> > StoragePoolType {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
> > directory
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), //
> > NFS or CIFS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN,
> > with a
> > > >>>>> clusterfs
> > > >>>>>>>>>>>>>> overlay
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
> > Comstar
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local
> > LVM SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS
> > storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> > XenServer, Storage
> > > >>> Pool
> > > >>>>>>>>>>>> is set
> > > >>>>>>>>>>>>>>>> up
> > > >>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local
> > EXT SR
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
> > zone-wide
> > > >>> storage
> > > >>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
> > shared) {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM,
> > Mike
> > > >>> Tutkowski
> > > >>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another
> > storage
> > > >>>>> company
> > > >>>>>>>>>>>> wants
> > > >>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality
> > of
> > > Service
> > > >>>>>>>>>>>> feature. It
> > > >>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well.
> > They would
> > > >>> need
> > > >>>>>>> only
> > > >>>>>>>>>>>>>>>>>>> implement a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
> > necessary
> > > >>> changes to
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39
> > PM, Mike
> > > >>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
> > XenServer code
> > > is
> > > >>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>> specific.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is
> > now aware
> > > >>> of
> > > >>>>>>>>>>>> dynamic,
> > > >>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> > implementation of
> > > >>> this
> > > >>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>> storage).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with
> > Edison's
> > > >>> storage
> > > >>>>>>>>>>>> framework
> > > >>>>>>>>>>>>>>>>>>>>>> changes.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
> > framework that
> > > >>>>> supported
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
> > However, when I
> > > >>>>>>>>>>>> visited with
> > > >>>>>>>>>>>>>>>> him
> > > >>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it
> > was not
> > > >>>>>>>>>>>> complete. We
> > > >>>>>>>>>>>>>>>>>>> realized
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do
> > with these
> > > >>>>>>> volumes
> > > >>>>>>>>>>>> unless
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this
> > new type of
> > > >>>>> storage
> > > >>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28
> > PM, John
> > > >>> Burwell
> > > >>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me
> > that any
> > > >>>>> operation
> > > >>>>>>>>>>>> in the
> > > >>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about
> > details. I
> > > >>> expect
> > > >>>>>>>>>>>> to see
> > > >>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> Storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations
> > that can be
> > > >>>>>>>>>>>>>> composed/driven
> > > >>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> > allocate
> > > space/create
> > > >>>>>>>>>>>> structures
> > > >>>>>>>>>>>>>>>> per
> > > >>>>>>>>>>>>>>>>>>>>>> their
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> > dependency, we are
> > > >>> going
> > > >>>>>>> to
> > > >>>>>>>>>>>> end
> > > >>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the
> > system
> > > >>>>>>> increasingly
> > > >>>>>>>>>>>>>>>>>>> difficult to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> > understanding that the
> > > >>> Xen
> > > >>>>>>>>>>>> specific
> > > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> > CitrixResourceBase
> > > class?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12
> > PM, Mike
> > > >>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit
> > more
> > > >>> detail:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from
> > one setup
> > > >>>>>>> method
> > > >>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>> XenServer,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on
> > the
> > > storage
> > > >>>>>>>>>>>> system,
> > > >>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure
> > to make
> > > use
> > > >>> of
> > > >>>>>>> the
> > > >>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>> (ex.
> > > >>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer
> > or a
> > > datastore
> > > >>> on
> > > >>>>>>>>>>>> ESX(i)).
> > > >>>>>>>>>>>>>>>> VMs
> > > >>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
> > system's volume.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage
> > framework,
> > > >>>>>>> storage
> > > >>>>>>>>>>>> need
> > > >>>>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>>>>> longer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create
> > a 1:1
> > > >>>>>>> relationship
> > > >>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data
> > disk
> > > >>> (necessary
> > > >>>>>>> for
> > > >>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>> Quality
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-
> > in that is
> > > >>> called
> > > >>>>>>> to
> > > >>>>>>>>>>>>>>>>>>> dynamically
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
> > storage
> > > framework
> > > >>> did
> > > >>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>> address
> > > >>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-
> > specific data
> > > >>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>> when
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've
> > been
> > > >>> enhancing
> > > >>>>> it
> > > >>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>> so.
> > > >>>>>>>>>>>>>>>>>>> I've
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> > submitted. I've got
> > > >>> ESX(i)
> > > >>>>>>> in
> > > >>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>> sandbox
> > > >>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
> > freeze date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 1:03 PM, Mike
> > > >>>>>>>>>>>> Tutkowski <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by
> > itself - is
> > > >>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>> agnostic.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
> > volume-attach
> > > >>> logic
> > > >>>>>>>>>>>> (in the
> > > >>>>>>>>>>>>>>>>>>> agent
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the
> > plug-in to
> > > >>> have
> > > >>>>>>> it
> > > >>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>>>> volume
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to
> > attach the
> > > >>> volume
> > > >>>>>>> to
> > > >>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>> hypervisor,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough
> > to recognize
> > > >>> it's
> > > >>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>> invoked on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has
> > just been
> > > >>> created)
> > > >>>>>>> and
> > > >>>>>>>>>>>>>> create,
> > > >>>>>>>>>>>>>>>>>>>>>> say, a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> > XenServer) or a
> > > >>> datastore
> > > >>>>>>>>>>>> (for
> > > >>>>>>>>>>>>>>>>>>> VMware) to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just
> > created.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most
> > of my time
> > > >>>>>>> recently
> > > >>>>>>>>>>>> making
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> > 12:48 PM,
> > > John
> > > >>>>>>>>>>>> Burwell <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why
> > the the storage
> > > >>>>>>> driver
> > > >>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21
> > PM, Mike
> > > >>>>>>> Tutkowski
> > > >>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
> > would like to
> > > >>> support
> > > >>>>>>>>>>>> all
> > > >>>>>>>>>>>>>>>>>>> hypervisors
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm
> > just out of
> > > >>> time
> > > >>>>>>>>>>>> for 4.2
> > > >>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>> get
> > > >>>>>>>>>>>>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in
> > supports
> > > >>>>>>> XenServer.
> > > >>>>>>>>>>>>>>>>>>> Depending on
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature
> > freeze, I
> > > >>> have
> > > >>>>> it
> > > >>>>>>>>>>>> working
> > > >>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear,
> > this is
> > > all
> > > >>> in
> > > >>>>>>>>>>>> regards
> > > >>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>> Disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute
> > Offerings post
> > > 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at 11:14 AM,
> > > >>> Kelcey
> > > >>>>>>>>>>>> Jamison
> > > >>>>>>>>>>>>>>>>>>> Damage
> > > >>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on
> > supporting
> > > >>> KVM in
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>> cycle
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> > Message -----
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> > Tutkowski" <
> > > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> > dev@cloudstack.apache.org
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June
> > 3, 2013
> > > >>> 10:12:32 AM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
> > > >>>>> disk_io_throttling
> > > >>>>>>>>>>>> to
> > > >>>>>>>>>>>>>> MASTER
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging
> > Wei's feature
> > > >>>>> first,
> > > >>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>> mine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for
> > KVM only,
> > > >>> then
> > > >>>>> it
> > > >>>>>>>>>>>> is a
> > > >>>>>>>>>>>>>> non
> > > >>>>>>>>>>>>>>>>>>> issue
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> > at 8:55 AM,
> > > Wei
> > > >>>>>>> ZHOU
> > > >>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as
> > no one works
> > > >>> on
> > > >>>>>>>>>>>> billing
> > > >>>>>>>>>>>>>> now,
> > > >>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
> > themselves. They
> > > >>> can
> > > >>>>>>> get
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a
> > VMs and
> > > volumes
> > > >>>>> for
> > > >>>>>>>>>>>>>>>>>>> calculation.
> > > >>>>>>>>>>>>>>>>>>>>>> Of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
> > exact
> > > limitation
> > > >>>>>>> value
> > > >>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for
> > nics as
> > > well.
> > > >>> I
> > > >>>>>>> can
> > > >>>>>>>>>>>> work
> > > >>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
> > throttling?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> > misunstand the second
> > > the
> > > >>>>>>>>>>>> question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what
> > you said about
> > > the
> > > >>>>> two
> > > >>>>>>>>>>>>>> features.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> > Burwell <
> > > >>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> > 2:13 AM, Wei
> > > >>> ZHOU
> > > >>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's
> > aswer helps you.
> > > >>> I am
> > > >>>>>>>>>>>> trying
> > > >>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>> adding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing
> > should
> > > >>> depend
> > > >>>>> on
> > > >>>>>>>>>>>> IO
> > > >>>>>>>>>>>>>>>>>>> statistics
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please
> > review
> > > >>>>>>>>>>>> disk_io_stat if
> > > >>>>>>>>>>>>>> you
> > > >>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
> > statistics
> > > including
> > > >>>>>>>>>>>> bytes/iops
> > > >>>>>>>>>>>>>>>>>>>>>> read/write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS
> > model,
> > > customers
> > > >>>>> are
> > > >>>>>>>>>>>> billed
> > > >>>>>>>>>>>>>>>> more
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS,
> > as well as,
> > > for
> > > >>>>>>> those
> > > >>>>>>>>>>>>>>>>>>> operations (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > http://aws.amazon.com/ebs/).
> > > I
> > > >>>>>>> would
> > > >>>>>>>>>>>>>> imagine
> > > >>>>>>>>>>>>>>>>>>> our
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ
> > similar cost
> > > >>>>> models.
> > > >>>>>>>>>>>> Could
> > > >>>>>>>>>>>>>> an
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in
> > the current
> > > >>> patch?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean
> > IOPS runtime
> > > >>>>> change?
> > > >>>>>>>>>>>> KVM
> > > >>>>>>>>>>>>>>>>>>> supports
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a
> > running
> > > virtual
> > > >>>>>>>>>>>> machine
> > > >>>>>>>>>>>>>>>> through
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does
> > not support
> > > >>>>> changing
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>> parameters
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
> > offering or disk
> > > >>>>>>> offering).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the
> > Java interface
> > > >>> level.
> > > >>>>>>> I
> > > >>>>>>>>>>>>>>>> apologize
> > > >>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> > generalize allocation
> > > >>>>>>>>>>>> algorithms
> > > >>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
> > service guarantees
> > > >>>>>>>>>>>> provided by a
> > > >>>>>>>>>>>>>>>>>>>>>> resource?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good
> > question.
> > > >>> Maybe it
> > > >>>>>>> is
> > > >>>>>>>>>>>>>> better
> > > >>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > disk_io_throttling as Mike
> > > >>> needs to
> > > >>>>>>>>>>>> consider
> > > >>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on
> > my thoughts
> > > in a
> > > >>>>>>> later
> > > >>>>>>>>>>>>>>>> response
> > > >>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
> > between these two
> > > >>>>>>>>>>>> features. I
> > > >>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be
> > merged before
> > > >>>>>>>>>>>> SolidFire, but
> > > >>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
> > between the
> > > branches
> > > >>>>>>>>>>>> (possibly
> > > >>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>> have
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling)
> > to
> > > >>> coordinate on
> > > >>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>> issue.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> > Burwell <
> > > >>>>>>>>>>>> jburwell@basho.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I
> > want to
> > > understand
> > > >>>>> are
> > > >>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>> following:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
> > value in
> > > capturing
> > > >>>>> IOPS
> > > >>>>>>>>>>>>>> policies
> > > >>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g.
> > for
> > > >>> billing/usage
> > > >>>>>>>>>>>>>> purposes,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there
> > be a common
> > > >>>>>>>>>>>> interface model
> > > >>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at
> > runtime?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
> > conflicting
> > > >>> provisioned
> > > >>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and
> > storage device
> > > >>>>>>>>>>>> reconciled?
> > > >>>>>>>>>>>>>> In
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
> > believe (and
> > > billed)
> > > >>>>> for
> > > >>>>>>>>>>>> more
> > > >>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device
> > has been
> > > >>> configured
> > > >>>>>>> to
> > > >>>>>>>>>>>>>>>> deliver.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
> > configuration
> > > >>> between a
> > > >>>>>>>>>>>> VM and a
> > > >>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
> > later
> > > modification
> > > >>> to
> > > >>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013,
> > at 8:38 PM,
> > > >>> Mike
> > > >>>>>>>>>>>> Tutkowski
> > > >>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > mike.tutkowski@solidfire.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's
> > feature deals
> > > >>> with
> > > >>>>>>>>>>>>>>>> controlling
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor
> > side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is
> > focused on
> > > >>>>>>> controlling
> > > >>>>>>>>>>>> IOPS
> > > >>>>>>>>>>>>>>>> from
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
> > helps. :)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2,
> > 2013 at 6:35
> > > PM,
> > > >>>>>>> John
> > > >>>>>>>>>>>>>> Burwell
> > > >>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is
> > that no
> > > features
> > > >>>>>>>>>>>> should be
> > > >>>>>>>>>>>>>>>>>>> merged
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have
> > been resolved
> > > and
> > > >>> it
> > > >>>>>>> is
> > > >>>>>>>>>>>> ready
> > > >>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>> turn
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs
> > discrete
> > > >>> read/write
> > > >>>>>>>>>>>> ops issue
> > > >>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't
> > think this
> > > >>> criteria
> > > >>>>>>>>>>>> has been
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how
> > does this work
> > > >>>>>>>>>>>>>>>> intersect/compliment
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>> https://reviews.apache.org/r/11479/
> > > >>>>>>>>>>>> )?
> > > >>>>>>>>>>>>>> As I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves
> > provisioned
> > > >>> IOPS. I
> > > >>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>> like
> > > >>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > where provisioned
> > > >>> IOPS
> > > >>>>> in
> > > >>>>>>>>>>>> KVM and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013,
> > at 6:47 AM,
> > > >>> Wei
> > > >>>>>>>>>>>> ZHOU <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will
> > change it next
> > > >>> week.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
> > Wido den Hollander
> > > <
> > > >>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > 06/01/2013 08:24 AM, Wei
> > > >>> ZHOU
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have
> > pushed the
> > > >>>>>>> features
> > > >>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
> > object thems for
> > > >>>>>>> technical
> > > >>>>>>>>>>>>>> reason
> > > >>>>>>>>>>>>>>>>>>> till
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake
> > of clarity I
> > > just
> > > >>>>>>> want
> > > >>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>> mention
> > > >>>>>>>>>>>>>>>>>>>>>> again
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps
> > to R/W IOps
> > > >>> asap
> > > >>>>> so
> > > >>>>>>>>>>>> that we
> > > >>>>>>>>>>>>>>>>>>> never
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the
> > groundwork for
> > > >>> the
> > > >>>>>>> I/O
> > > >>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
> > prevent that we
> > > create
> > > >>>>>>>>>>>> legacy from
> > > >>>>>>>>>>>>>>>> day
> > > >>>>>>>>>>>>>>>>>>>>>> #1.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> > Wido den
> > > Hollander <
> > > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > 05/31/2013 03:59 PM, John
> > > >>>>>>> Burwell
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
> > enhancement must
> > > to
> > > >>>>>>>>>>>> discretely
> > > >>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how
> > it could be
> > > >>> fixed
> > > >>>>>>>>>>>> later
> > > >>>>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS
> > into read
> > > and
> > > >>>>>>>>>>>> write.
> > > >>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > unless/until we
> > > decided
> > > >>> to
> > > >>>>>>>>>>>> break
> > > >>>>>>>>>>>>>>>>>>> backwards
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei
> > meant was merging
> > > it
> > > >>>>> into
> > > >>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>> now
> > > >>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch
> > and add Read /
> > > >>> Write
> > > >>>>>>> IOps
> > > >>>>>>>>>>>>>> before
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> > released with Read
> > > and
> > > >>>>>>> Write
> > > >>>>>>>>>>>>>> instead
> > > >>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
> > make the May 31st
> > > >>>>>>> feature
> > > >>>>>>>>>>>>>> freeze
> > > >>>>>>>>>>>>>>>>>>> date.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other
> > threads) then it
> > > >>> won't
> > > >>>>>>> be
> > > >>>>>>>>>>>>>>>> necessary
> > > >>>>>>>>>>>>>>>>>>>>>> to do
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> > completely agree that
> > > >>>>> there
> > > >>>>>>>>>>>> is no
> > > >>>>>>>>>>>>>>>>>>>>>> association
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31,
> > 2013, at 9:51 AM,
> > > >>> Wido
> > > >>>>>>>>>>>> den
> > > >>>>>>>>>>>>>>>>>>> Hollander <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > 05/31/2013 03:13 PM, Wei
> > > >>> ZHOU
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good
> > question.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
> > about at the
> > > >>>>> beginning.
> > > >>>>>>>>>>>>>> Finally I
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of
> > read and write
> > > >>>>>>> mainly
> > > >>>>>>>>>>>>>> because
> > > >>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
> > difference of sent
> > > >>> and
> > > >>>>>>>>>>>> received
> > > >>>>>>>>>>>>>>>>>>> bytes as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> > reasoning seems odd.
> > > >>>>>>>>>>>> Networking and
> > > >>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is
> > much more
> > > >>> expensive
> > > >>>>> in
> > > >>>>>>>>>>>> most
> > > >>>>>>>>>>>>>>>>>>>>>> situations
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing
> > it will be some
> > > >>>>>>>>>>>> copy-paste
> > > >>>>>>>>>>>>>>>> work.
> > > >>>>>>>>>>>>>>>>>>> It
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented
> > in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For
> > the deadline
> > > of
> > > >>>>>>>>>>>> feature
> > > >>>>>>>>>>>>>>>> freeze,
> > > >>>>>>>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>>>>>> will
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
> > needed.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a
> > feature we
> > > >>> can't
> > > >>>>>>>>>>>> miss. But
> > > >>>>>>>>>>>>>>>> if
> > > >>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we
> > have to make sure
> > > we
> > > >>>>>>> don't
> > > >>>>>>>>>>>>>> release
> > > >>>>>>>>>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that
> > will confuse
> > > >>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> > Wido den
> > > Hollander <
> > > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> > 05/30/2013 06:03 PM, Wei
> > > >>> ZHOU
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to
> > merge
> > > >>>>>>>>>>>> disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>> branch
> > > >>>>>>>>>>>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
> > object, I will
> > > merge
> > > >>>>>>> into
> > > >>>>>>>>>>>> master
> > > >>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>> 48
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose
> > is :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> > machines are running
> > > >>> on
> > > >>>>>>> the
> > > >>>>>>>>>>>> same
> > > >>>>>>>>>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage).
> > Because of
> > > the
> > > >>>>> rate
> > > >>>>>>>>>>>>>>>> limitation
> > > >>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has
> > large disk
> > > >>> operation,
> > > >>>>>>> it
> > > >>>>>>>>>>>> may
> > > >>>>>>>>>>>>>>>> affect
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs
> > running on the
> > > same
> > > >>>>>>>>>>>> storage
> > > >>>>>>>>>>>>>>>> device.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary
> > to set the
> > > >>>>> maximum
> > > >>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>> limit
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the
> > code I see
> > > you
> > > >>>>> make
> > > >>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>>> difference
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and
> > libvirt support
> > > >>> setting
> > > >>>>>>>>>>>> both a
> > > >>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which
> > could benefit a
> > > >>> lot of
> > > >>>>>>>>>>>> users.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
> > strange, in the
> > > >>> polling
> > > >>>>>>>>>>>> side you
> > > >>>>>>>>>>>>>>>>>>>>>> collect
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on
> > the throttling
> > > >>> side
> > > >>>>>>>>>>>> you only
> > > >>>>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are
> > usually much
> > > >>> more
> > > >>>>>>>>>>>> expensive
> > > >>>>>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>>>>>> Read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid
> > use-case where
> > > >>> that
> > > >>>>>>> an
> > > >>>>>>>>>>>> admin
> > > >>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>> set
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read
> > IOps.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only
> > supports KVM
> > > >>> at
> > > >>>>>>> this
> > > >>>>>>>>>>>>>> point I
> > > >>>>>>>>>>>>>>>>>>>>>> think
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at
> > least have the
> > > >>>>>>>>>>>> mechanism in
> > > >>>>>>>>>>>>>>>> place
> > > >>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
> > would be a lot of
> > > >>>>> work.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor
> > doesn't
> > > >>> support
> > > >>>>>>>>>>>> setting
> > > >>>>>>>>>>>>>>>>>>> different
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can
> > always sum
> > > both
> > > >>> up
> > > >>>>>>>>>>>> and set
> > > >>>>>>>>>>>>>>>> that
> > > >>>>>>>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> > explain why you
> > > >>>>>>> implemented
> > > >>>>>>>>>>>> it
> > > >>>>>>>>>>>>>> this
> > > >>>>>>>>>>>>>>>>>>> way?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature
> > includes:
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> > maximum rate of
> > > >>> VMs
> > > >>>>>>> (in
> > > >>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the
> > maximum rate
> > > of
> > > >>>>> VMs
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the
> > disk rate
> > > (total
> > > >>>>> bps
> > > >>>>>>>>>>>> and
> > > >>>>>>>>>>>>>> iops)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> > > >>>>>>>>>>>>>> https://issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > jira/browse/CLOUDSTACK-1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > 1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > 1192<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
> > update later) :
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>
> > >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > VM+Disk+IO+Throttling<https://
> > > >>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>
> > > >>>>>
> > > >>>
> > >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check
> > list :-
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
> > check the branch's
> > > >>> RAT
> > > >>>>>>>>>>>> execution
> > > >>>>>>>>>>>>>>>>>>>>>> success?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there
> > new dependencies
> > > >>>>>>>>>>>> introduced?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> > automated testing
> > > (unit
> > > >>>>> and
> > > >>>>>>>>>>>>>>>>>>> integration)
> > > >>>>>>>>>>>>>>>>>>>>>> is
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are
> > added.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> > testing has been done
> > > >>> to
> > > >>>>>>>>>>>> check for
> > > >>>>>>>>>>>>>>>>>>>>>> potential
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> > bytes rate and
> > > >>> IOPS
> > > >>>>>>>>>>>> rate on
> > > >>>>>>>>>>>>>>>>>>>>>> CloudStack
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
> > operations, including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop,
> > start, reboot,
> > > >>>>>>>>>>>> destroy,
> > > >>>>>>>>>>>>>>>> expunge.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume
> > operations,
> > > >>> including
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> > Detach
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the
> > code, you can
> > > >>> try
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > c30057635d04a2396f84c588127d7e******be42e503a7
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>
> > >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > > >>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > VM+Disk+IO+Throttling<https://
> > > >>>>>>> ****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > cwiki.apache.org/confluence/****
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>
> > > >>>>>
> > > >>>
> > >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> > rottling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > > >>> refs/heads/disk_io_throttling
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > 1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > 1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > 1301<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> > > >>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> > 2071
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> > 2071
> > > >>>>>>>>>>>>>> <
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> > 2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > (**CLOUDSTACK-1301
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk
> > I/O
> > > Throttling)
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> > Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> > CloudStack Developer,
> > > >>>>>>>>>>>> SolidFire
> > > >>>>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > > >>> mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> > way the world
> > > >>> uses
> > > >>>>>>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>>>> SolidFire
> > > >>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the world uses
> > > >>> the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>> SolidFire
> > > >>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> > the world uses
> > > the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > >>> SolidFire
> > > >>>>>>>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world uses the
> > > >>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer,
> > > SolidFire
> > > >>>>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> > Developer, SolidFire
> > > >>>>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> > world uses the
> > > >>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses the
> > > >>> cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > >>> Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> > uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire
> > > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> > the cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> > SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > cloud<
> > > >>>>>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>>>>> cloud<
> > > >>>>> http://solidfire.com/solution/overview/?video=play
> > > >>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>>>>>>>>
> > http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>>>> cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play
> > > >
> > > >>>>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>>>> Advancing the way the world uses the
> > > >>>>>>>>>>>>>
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> --
> > > >>>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>>> *(tm)*
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>>> o: 303.746.7302
> > > >>>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>>> *(tm)*
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> --
> > > >>>>>>>>> *Mike Tutkowski*
> > > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>>> o: 303.746.7302
> > > >>>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>>> *(tm)*
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> *Mike Tutkowski*
> > > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>>> o: 303.746.7302
> > > >>>>>>>> Advancing the way the world uses the cloud<
> > > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>>> *(tm)*
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> *Mike Tutkowski*
> > > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>>>>> e: mike.tutkowski@solidfire.com
> > > >>>>>>> o: 303.746.7302
> > > >>>>>>> Advancing the way the world uses the
> > > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>>>>> *(tm)*
> > > >>>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> *Mike Tutkowski*
> > > >>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>>> e: mike.tutkowski@solidfire.com
> > > >>>> o: 303.746.7302
> > > >>>> Advancing the way the world uses the
> > > >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>>> *(tm)*
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> *Mike Tutkowski*
> > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >> e: mike.tutkowski@solidfire.com
> > > >> o: 303.746.7302
> > > >> Advancing the way the world uses the cloud<
> > > http://solidfire.com/solution/overview/?video=play>
> > > >> *(tm)*
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkowski@solidfire.com
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > *(tm)*
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *(tm)*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
RE: [MERGE] disk_io_throttling to MASTER
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
> Sent: Friday, June 07, 2013 1:14 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> Hi John,
>
> How's about this:
>
> The driver can implement an isManaged() method. The VolumeManagerImpl
> can
> call into the driver to see if its managed. If it is, the VolumeManagerImpl
> (which is responsible for calling into the hypervisor to attach the disk)
> can call into the hypervisor to create the necessary hypervisor data
> structure (ex. for XenServer, a storage repository).
The problem here is that storage vendor may work differently with hypervisor, for example, SolidFire wants a SR per LUN, while maybe other vendor wants to totally bypass hypervisor, and assign the LUN directly to VM instance, see the discuss(http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201303.mbox/%3C06f219312189b019a8763a5777ecc430@mail.gmail.com%3E).
So I would let storage provider to implement attach disk to VM, instead of implemented by cloudstack itself.
>
> If that's what you're going for, that works for me. By the way, Edison's
> default storage plug-in (which handles the default storage behavior in
> CloudStack (ex. how pre 4.2 works)) does include code that talks to
> hypervisors. You might want to contact him and inform him of your concerns
> or that logic (as is) will make it to production.
>
> Please let me know if what I wrote in above (for my solution) is OK with
> you. :)
>
> Thanks!
>
>
> On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com> wrote:
>
> > Mike,
> >
> > Please see my responses in-line below.
> >
> > Thanks,
> > -John
> >
> > On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <mi...@solidfire.com>
> > wrote:
> >
> > > Hey John,
> > >
> > > I still have a bit more testing I'd like to do before I build up a patch
> > > file, but this is the gist of what I've done:
> > >
> > > * During a volume-attach operation, after VolumeManagerImpl tells
> > > VolumeServiceImpl to have the driver create a volume, I have
> > > VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it
> managed.
> > > If it is managed, VolumeServiceImpl has the driver perform whatever
> > > activity is required. In my case, this includes sending a message to the
> > > host where the VM is running to have, say XenServer, add a storage
> > > repository (based on the IP address of the SAN, the IQN of the SAN
> > volume,
> > > etc.) and a single VDI (the VDI consumes all of the space it can on the
> > > storage repository). After this, the normal attach-volume message is sent
> > > to the host by VolumeManagerImpl.
> >
> > There should be **no** code from a storage driver to a hypervisor. I
> > apologize for the repetition, but we simply can not have hypervisor
> > specific code in the storage layer. The circular dependencies between the
> > two layers are not sustainable in the long term. Either the VirtualManager
> > or Xen hypervisor plugin needs to be refactored/modified to coordinate
> > volume creation and then populating the SR. Ideally, we can generalize the
> > process flow for attaching volumes such that the Xen hypervisor plugin
> > would only implement callbacks to perform the attach action and create
> the
> > structure and SR. To my mind, the SolidFire driver should only be
> > allocating space and providing information about contents (e.g. space
> > available, space consumed, streams to a URI, file handle for a URI, etc)
> > and capabilities.
> >
> >
> > >
> > > * The reverse is performed for a detach-volume command.
> > >
> > > * Right now I simply "return true;" for isManaged() in my driver.
> > Edison's
> > > default driver simply does a "return false;". We could add a new
> > parameter
> > > to the createStoragePool API command, if we want, to remove the
> > hard-coded
> > > return values in the drivers (although my driver will probably just
> > ignore
> > > this parameter and always return true since it wouldn't make sense for it
> > > to ever return false). We'd need another column in the storage_pool
> table
> > > to store this value.
> >
> > Yes, I think we should have a parameter added to the createStoragePool
> > surfaced to the HTTP API that allows DataStores to be configured for
> > management when their underlying drivers support it. To simplify things,
> > this flag should only be mutable when the DataStore is created. It would be
> > a bit crazy to take a DataStore from managed to unmanaged after creation.
> >
> > >
> > > Sound like I'm in sync with what you were thinking?
> > >
> > >
> > >
> > >
> > > On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com
> > >> wrote:
> > >
> > >> I agree, John. Just wanted to point out that I have a working GUI for
> > you
> > >> to review (in that document), if you'd like to check it out.
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com>
> > wrote:
> > >>
> > >>> Mike,
> > >>>
> > >>> I would like the UIs of two features reviewed together to ensure
> > >>> consistency across the concepts of hypervisor throttled IOPs and
> > >>> storage device provisioned IOPs. I see the potential for confusion,
> > >>> and I think a side-by-side Ui review of these features will help
> > >>> minimize any potential confusion.
> > >>>
> > >>> As I mentioned, the term reconciliation issue will work itself if it
> > >>> is acceptable that a VM is only permitted utilize hypervisor throttled
> > >>> IOPs or storage provisioned IOPs.
> > >>>
> > >>> Thanks,
> > >>> -John
> > >>>
> > >>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> > >>> <mi...@solidfire.com> wrote:
> > >>>
> > >>>> Hi John,
> > >>>>
> > >>>> Yeah, when you get a chance, refer to the Google doc I sent to you
> the
> > >>>> other day to see how the GUI looks for provisioned storage IOPS.
> > >>>>
> > >>>> Several months ago, I put this topic out on the e-mail list and we
> > >>> decided
> > >>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering dialog.
> > >>>> Other storage vendors are coming out with QoS, so they should be
> able
> > to
> > >>>> leverage this GUI going forward (even if they, say, only use Max
> > IOPS).
> > >>>> These fields are optional and can be ignored for storage that does not
> > >>>> support provisioned IOPS. Just like the Disk Size field, the admin can
> > >>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
> > >>>>
> > >>>> I'm OK if we do an either/or model (either Wei's feature or mine, as
> > is
> > >>>> decided by the admin).
> > >>>>
> > >>>> I'm not sure what we can do about these two features expressing the
> > >>> speed
> > >>>> in different terms. I've never seen a SAN express the IOPS for QoS in
> > >>> any
> > >>>> way other than total IOPS (i.e. not broken in into read/write IOPS).
> > >>>>
> > >>>> Thanks
> > >>>>
> > >>>>
> > >>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com>
> > >>> wrote:
> > >>>>
> > >>>>> Wei,
> > >>>>>
> > >>>>> We have been down the rabbit hole a bit on the Storage/Hypervisor
> > layer
> > >>>>> separation, but we still need to reconcile the behavior of hypervisor
> > >>>>> throttled I/O and storage provisioned IOPS. I see the following
> > issues
> > >>>>> outstanding:
> > >>>>>
> > >>>>> 1. Hypervisor throttled IOPS are expressed as discrete read/write
> > >>> values
> > >>>>> where as storage provisioned IOPS are expressed as total IOPS.
> > >>>>> 2. How do we handle VMs with throttled IOPS attached to storage
> > volumes
> > >>>>> with provisioned IOPS?
> > >>>>> 3. How should usage data be captured for throttled and provisioned
> > IOPS
> > >>>>> that will permit providers to discriminate these guaranteed
> > operations
> > >>> in
> > >>>>> the event they want to bill for it differently?
> > >>>>> 4. What is the user experience for throttled and provisioned IOPS
> > that
> > >>>>> minimizes confusion of these concepts?
> > >>>>>
> > >>>>> My thinking is that a VM can have either utilize hypervisor throttled
> > >>> IOPS
> > >>>>> or storage provisioned IOPS. This policy would neatly solve items 1
> > >>> and 2.
> > >>>>> Since the two facilities would not be permitted to operate together,
> > >>> they
> > >>>>> do not need to be semantically compatible. I think item 3 can be
> > >>> resolved
> > >>>>> with an additional flag or two on the usage records. As for Item 4,
> > I
> > >>> am
> > >>>>> not familiar with how these two enhancements are depicted in the
> user
> > >>>>> interface. I think we need to review the UI enhancements for both
> > >>>>> enhancements and ensure they are consistent.
> > >>>>>
> > >>>>> Do these solutions make sense?
> > >>>>>
> > >>>>> Thanks,
> > >>>>> -John
> > >>>>>
> > >>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com>
> wrote:
> > >>>>>
> > >>>>>> John and Mike,
> > >>>>>>
> > >>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> > >>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-
> 2875 ) this
> > >>> week.
> > >>>>>> I will start to develop on iops/bps changes tomorrow, and ask for
> > >>> second
> > >>>>>> merge request after testing.
> > >>>>>>
> > >>>>>> -Wei
> > >>>>>>
> > >>>>>>
> > >>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> > >>>>>>
> > >>>>>>> I believe I understand where you're going with this, John, and
> have
> > >>> been
> > >>>>>>> re-working this section of code today.
> > >>>>>>>
> > >>>>>>> I should be able to run it by you tomorrow.
> > >>>>>>>
> > >>>>>>> Thanks for the comments,
> > >>>>>>> Mike
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell
> <jb...@basho.com>
> > >>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Mike,
> > >>>>>>>>
> > >>>>>>>> See my responses in-line below.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> -John
> > >>>>>>>>
> > >>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> > >>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Hi John,
> > >>>>>>>>
> > >>>>>>>> Thanks for the response.
> > >>>>>>>>
> > >>>>>>>> "I am fine with the VolumeManager determining whether or
> not a
> > >>> Volume
> > >>>>> is
> > >>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
> > >>> isManaged
> > >>>>>>>> method), and asking the device driver to allocate resources for
> > the
> > >>>>>>> volume
> > >>>>>>>> if it is managed."
> > >>>>>>>>
> > >>>>>>>> Are you thinking you'd like to see an isManaged() method
> added to
> > >>> the
> > >>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then the
> > >>> storage
> > >>>>>>>> framework could call the manage() (or whatever name) method
> (which
> > >>>>> would
> > >>>>>>> be
> > >>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and this
> > would
> > >>>>> call
> > >>>>>>>> into a new method in the hypervisor code to create, say on
> > >>> XenServer,
> > >>>>> an
> > >>>>>>> SR?
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> I would like to see canBeManaged() : boolean on
> DataStoreDriver.
> > >>> Since
> > >>>>>>>> the notion of Volumes only pertains to primary storage, I would
> > add
> > >>> a
> > >>>>>>>> allocateStorage and deallocateStorage (Storage is a straw man
> term
> > >>> --
> > >>>>>>>> something other than volume) methods to
> > >>>>> allocate/create/deallocate/delete
> > >>>>>>>> underlying storage. To my mind, managed is a mutable property
> of
> > >>>>>>> DataStore
> > >>>>>>>> which can be enabled if/when the underlying DataStoreDriver
> can be
> > >>>>>>> managed.
> > >>>>>>>> This approach allows operators to override manageability of
> > devices.
> > >>>>>>>>
> > >>>>>>>> In terms of orchestration/process flow for SR, the Xen plugin
> > would
> > >>> be
> > >>>>>>>> responsible for composing DataStore/Volume methods to
> create any
> > >>>>>>>> directories or files necessary for the SR. There should be no
> > >>>>>>> dependencies
> > >>>>>>>> from the Storage to the Hypervisor layer. As I said earlier, such
> > >>>>>>> circular
> > >>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Just want to make sure I'm on the same page with you.
> > >>>>>>>>
> > >>>>>>>> Thanks again, John
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell
> <jb...@basho.com>
> > >>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Mike,
> > >>>>>>>>>
> > >>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> > supports n
> > >>>>>>>>> devices types with each specific behaviors of m hypervisors.
> > Such
> > >>> a
> > >>>>>>>>> scenario will create an unmaintainable and untestable beast.
> > >>>>>>> Therefore, my
> > >>>>>>>>> thoughts and recommendations are driven to evolve the
> Storage
> > layer
> > >>>>>>> towards
> > >>>>>>>>> this separation.
> > >>>>>>>>>
> > >>>>>>>>> I am fine with the VolumeManager determining whether or
> not a
> > >>> Volume
> > >>>>> is
> > >>>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
> > >>>>> isManaged
> > >>>>>>>>> method), and asking the device driver to allocate resources for
> > the
> > >>>>>>> volume
> > >>>>>>>>> if it is managed. Furthermore, the device driver needs to
> > indicate
> > >>>>>>> whether
> > >>>>>>>>> or not it supports management operations. Finally, I think we
> > >>> need to
> > >>>>>>>>> provide the ability for an administrator to elect to have
> > something
> > >>>>>>> that is
> > >>>>>>>>> manageable be unmanaged (i.e. the driver is capable managing
> the
> > >>>>> device,
> > >>>>>>>>> but the administrator has elected to leave it unmanaged).
> > >>>>>>>>>
> > >>>>>>>>> Creation of a structure on the volume should be done in the
> Xen
> > >>>>>>>>> hypervisor module using methods exposed by the Storage
> layer to
> > >>>>> perform
> > >>>>>>>>> low-level operations (e.g. make directories, create a file, etc).
> > >>>>> This
> > >>>>>>>>> structure is specific to the operation of the Xen hypervisor, as
> > >>> such,
> > >>>>>>>>> should be confined to its implementation. From my
> perspective,
> > >>>>> nothing
> > >>>>>>> in
> > >>>>>>>>> the Storage layer should be concerned with content. From its
> > >>>>>>> perspective,
> > >>>>>>>>> structure and data are opaque. It provides the means to query
> > the
> > >>>>> data
> > >>>>>>> to
> > >>>>>>>>> support the interpretation of the content by higher-level
> layers
> > >>> (e.g.
> > >>>>>>>>> Hypervisors). To my mind, attach should be a composition of
> > >>>>> operations
> > >>>>>>>>> from the Storage layer that varies based on the Volume
> storage
> > >>>>> protocol
> > >>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> -John
> > >>>>>>>>>
> > >>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> > >>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Hi John,
> > >>>>>>>>>
> > >>>>>>>>> Alternatively to the way the attach logic is implemented in my
> > >>> patch,
> > >>>>> we
> > >>>>>>>>> could do the following:
> > >>>>>>>>>
> > >>>>>>>>> Leave the attach logic in the agent code alone. In
> > >>> VolumeManagerImpl
> > >>>>> we
> > >>>>>>>>> create an AttachVolumeCommand and send it to the
> hypervisor.
> > Before
> > >>>>> this
> > >>>>>>>>> command is sent, we could check to see if we're dealing with
> > >>> Dynamic
> > >>>>> (or
> > >>>>>>>>> whatever we want to call it) storage and - if so - send a "Create
> > >>> SR"
> > >>>>>>>>> command to the hypervisor. If this returns OK, we would then
> > >>> proceed
> > >>>>> to
> > >>>>>>> the
> > >>>>>>>>> AttachVolumeCommand, as usual.
> > >>>>>>>>>
> > >>>>>>>>> This way the attach logic remains the same and we just add
> > another
> > >>>>>>>>> command to the agent code that is called for this particular
> type
> > >>> of
> > >>>>>>>>> storage.
> > >>>>>>>>>
> > >>>>>>>>> What do you think?
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> > >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hey John,
> > >>>>>>>>>>
> > >>>>>>>>>> I created a document for a customer today that outlines how
> the
> > >>>>> plug-in
> > >>>>>>>>>> works from a user standpoint. This will probably be of use to
> > >>> you, as
> > >>>>>>> well,
> > >>>>>>>>>> as you perform the code review.
> > >>>>>>>>>>
> > >>>>>>>>>> I have shared this document with you (you should have
> received
> > >>> that
> > >>>>>>>>>> information in a separate e-mail).
> > >>>>>>>>>>
> > >>>>>>>>>> Talk to you later!
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> > >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Oh, OK, that sounds really good, John.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks and talk to you tomorrow! :)
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> > jburwell@basho.com
> > >>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts, but
> > want
> > >>> to
> > >>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> > >>>>>>> disk_io_throttle,
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>> object_store branches. I hope to collect them in a
> coherent
> > >>> form
> > >>>>>>>>>>>> tomorrow (5 June 2013).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>> -John
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> > >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the notion
> > of
> > >>> a
> > >>>>>>>>>>>> managed
> > >>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the
> notion
> > >>> of
> > >>>>>>>>>>>>> provisioned storage IOPS.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> The storage framework that leverages the plug-in was
> > >>> incomplete,
> > >>>>> so
> > >>>>>>>>>>>> I had
> > >>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I appreciate all the time you've been spending on this. :)
> Do
> > >>> you
> > >>>>>>>>>>>> have a
> > >>>>>>>>>>>>> recommendation as to how we should accomplish what
> you're
> > >>> looking
> > >>>>>>>>>>>> for?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> > >>> jburwell@basho.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the notion
> > of
> > >>> a
> > >>>>>>>>>>>> managed
> > >>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I want to see a separation of the management
> capabilities
> > >>> (i.e.
> > >>>>>>> can
> > >>>>>>>>>>>> a
> > >>>>>>>>>>>>>> device be managed/does an operator want it managed
> by
> > >>> CloudStack)
> > >>>>>>>>>>>> from the
> > >>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a
> semantic
> > >>> that
> > >>>>>>>>>>>> will
> > >>>>>>>>>>>>>> allow any type of storage device to be managed. I also
> want
> > >>> to
> > >>>>>>> make
> > >>>>>>>>>>>>>> progress on decoupling the storage types from the
> hypervisor
> > >>>>>>>>>>>> definitions.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> > >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> No problem. Answers are below in red.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> > >>>>> jburwell@basho.com
> > >>>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Could you please answer the following questions for
> me
> > with
> > >>>>>>>>>>>> regards to
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN
> > volumes?
> > >>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
> > >>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and
> CloudStack
> > >>>>>>>>>>>> Volumes?
> > >>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Are the LUN(s) created by the management server or
> > >>> externally
> > >>>>> by
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>> operator?
> > >>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume
> (same
> > as a
> > >>>>> SAN
> > >>>>>>>>>>>> LUN) is
> > >>>>>>>>>>>>>>> created by the management server (via the plug-in)
> the
> > first
> > >>>>> time
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but still
> > >>> want
> > >>>>> to
> > >>>>>>>>>>>> use a
> > >>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already today
> > (prior
> > >>> to
> > >>>>>>>>>>>> 4.2). The
> > >>>>>>>>>>>>>>> admin manually creates the SAN volume and - in this
> case -
> > >>>>>>>>>>>> multiple VMs
> > >>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>> data disks can share this SAN volume. While you can do
> this
> > >>>>>>> today,
> > >>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Are the SAN volumes by the management server or
> externally
> > >>> by
> > >>>>>>> the
> > >>>>>>>>>>>>>> operator?
> > >>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes
> are
> > >>>>>>> completely
> > >>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>> by the management server (via the plug-in). There is
> no
> > admin
> > >>>>>>>>>>>>>> interaction.
> > >>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume
> and a
> > >>>>>>> CloudStack
> > >>>>>>>>>>>>>> volume,
> > >>>>>>>>>>>>>>> which is necessary for any storage vendor that
> supports
> > true
> > >>>>> QoS.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I would like to clarify how these pieces are related
> and
> > >>>>>>> expected
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> operate.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs
> with
> > >>>>>>>>>>>> provisioned
> > >>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a single
> SAN
> > >>> volume
> > >>>>>>>>>>>> (LUN) to a
> > >>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have multiple
> > >>> CloudStack
> > >>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume house
> more
> > >>> than
> > >>>>>>> one
> > >>>>>>>>>>>>>>>>> CloudStack volume, then can do that today without
> any of
> > my
> > >>>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>> code.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume for
> > >>>>> CloudStack's
> > >>>>>>>>>>>> use
> > >>>>>>>>>>>>>> onto
> > >>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> > >>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me --
> > concepts
> > >>>>> from
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only
> > used
> > >>>>> (as
> > >>>>>>>>>>>> far as
> > >>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>> know)
> > >>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not
> PreSetup an
> > >>> SR).
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from
> VMware, it
> > >>> uses
> > >>>>>>> the
> > >>>>>>>>>>>> vmfs
> > >>>>>>>>>>>>>>>> type.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM,
> it uses
> > >>> the
> > >>>>>>>>>>>>>>>>>>> SharedMountPoint
> > >>>>>>>>>>>>>>>>>>>> type.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here
> was to
> > >>> make a
> > >>>>>>>>>>>> new type
> > >>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison
> added
> > >>> into
> > >>>>>>> the
> > >>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more specificy,
> though:
> > >>>>>>>>>>>> Dynamic_iSCSI
> > >>>>>>>>>>>>>>>>>>> versus,
> > >>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski
> <
> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or
> care
> > that
> > >>> it
> > >>>>>>> is
> > >>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>> used
> > >>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer
> questions
> > >>> about
> > >>>>> it
> > >>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>> storing."
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the
> SolidFire
> > >>> plug-in
> > >>>>>>>>>>>> needs to
> > >>>>>>>>>>>>>>>>>>> call
> > >>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor
> logic
> > >>>>> knows
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>> treat
> > >>>>>>>>>>>>>>>>>>> it as
> > >>>>>>>>>>>>>>>>>>>>> such.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint
> and just
> > >>>>>>>>>>>> calling it
> > >>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for the
> hypervisor
> > >>>>>>> attach
> > >>>>>>>>>>>> logic
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform the
> necessary
> > >>>>>>>>>>>> activities.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
> > >>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to
> manage a
> > >>>>>>> device,
> > >>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack
> (e.g. the
> > >>>>>>> device
> > >>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>> shared
> > >>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has chosen
> to
> > >>> dedicate
> > >>>>>>>>>>>> only a
> > >>>>>>>>>>>>>>>> portion
> > >>>>>>>>>>>>>>>>>>>>>> of it
> > >>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make
> sense?"
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN
> volume via
> > >>> the
> > >>>>>>>>>>>> plug-in
> > >>>>>>>>>>>>>>>>>>> (before
> > >>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would
> work unless
> > >>> the
> > >>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>> consumes
> > >>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of
> CloudStack, an
> > >>>>>>> operator
> > >>>>>>>>>>>>>>>> allocates
> > >>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it
> > through
> > >>> a
> > >>>>>>>>>>>> LUN. They
> > >>>>>>>>>>>>>>>>>>> simply
> > >>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to
> CloudStack,
> > >>> but
> > >>>>>>>>>>>> not allow
> > >>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my
> changes, we
> > >>> would
> > >>>>>>> be
> > >>>>>>>>>>>>>> passing
> > >>>>>>>>>>>>>>>>>>> in a
> > >>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the
> necessary
> > >>> hypervisor
> > >>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>> (like
> > >>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>> SR)
> > >>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the
> storage
> > >>>>>>> framework
> > >>>>>>>>>>>>>> itself
> > >>>>>>>>>>>>>>>>>>>>>> detect
> > >>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from
> the
> > >>> hypervisor
> > >>>>>>>>>>>> side and
> > >>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to create
> the
> > >>> SR
> > >>>>>>>>>>>> before the
> > >>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin
> should
> > have a
> > >>>>>>> rich
> > >>>>>>>>>>>> enough
> > >>>>>>>>>>>>>>>>>>>>>> interface to storage to determine available
> for
> > volume
> > >>>>>>>>>>>> storage.
> > >>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>> Xen,
> > >>>>>>>>>>>>>>>>>>>>>> this interface would allow the interrogation of
> the
> > >>>>> device
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> determine the
> > >>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself
> shouldn't
> > >>> know
> > >>>>>>>>>>>> or care
> > >>>>>>>>>>>>>>>>>>> that it
> > >>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to
> > answer
> > >>>>>>>>>>>> questions
> > >>>>>>>>>>>>>>>>>>> about it
> > >>>>>>>>>>>>>>>>>>>>>> is storing.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver
> (which
> > >>> is a
> > >>>>>>>>>>>> type of
> > >>>>>>>>>>>>>>>>>>>>>> so-called
> > >>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new
> Primary
> > >>>>>>>>>>>> Storage shows
> > >>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that
> > references
> > >>> the
> > >>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>> tag
> > >>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack
> volume. This
> > >>> leads
> > >>>>>>> to
> > >>>>>>>>>>>> a new
> > >>>>>>>>>>>>>>>> row
> > >>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack
> volume to a
> > >>> VM
> > >>>>>>>>>>>> (attach
> > >>>>>>>>>>>>>>>> disk).
> > >>>>>>>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the
> plug-in
> > >>> to
> > >>>>>>>>>>>> create a
> > >>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN).
> The
> > >>> plug-in
> > >>>>>>>>>>>> also
> > >>>>>>>>>>>>>>>> updates
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data
> (like the
> > >>> IQN of
> > >>>>>>>>>>>> the SAN
> > >>>>>>>>>>>>>>>>>>>>>> volume).
> > >>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> > CloudStack
> > >>>>>>>>>>>> volume is in
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the
> volume
> > >>> will be
> > >>>>>>>>>>>> in the
> > >>>>>>>>>>>>>>>>>>> 'Ready'
> > >>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the
> plug-in
> > >>> code
> > >>>>>>>>>>>> will not
> > >>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>> called
> > >>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and
> detects
> > the
> > >>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a
> > >>> hypervisor
> > >>>>>>> data
> > >>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on
> terminology,
> > >>> volume,
> > >>>>>>>>>>>> in these
> > >>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical
> allocation on
> > >>> the
> > >>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>>>>>> correct?"
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean
> 'SAN
> > >>> volume'.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can
> make in
> > >>>>>>>>>>>> CloudStack, I
> > >>>>>>>>>>>>>>>>>>> try to
> > >>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well make
> sense, but
> > >>> I'm
> > >>>>>>>>>>>> having a
> > >>>>>>>>>>>>>>>> hard
> > >>>>>>>>>>>>>>>>>>>>>> time
> > >>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on
> this
> > >>> conversation
> > >>>>>>>>>>>> as he
> > >>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I
> took.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John
> Burwell <
> > >>>>>>>>>>>>>>>> jburwell@basho.com
> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two
> distinct
> > >>> concepts
> > >>>>>>> --
> > >>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In
> both
> > >>> cases, we
> > >>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>> communicating with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
> > create/delete
> > >>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>> (Dynamic)
> > >>>>>>>>>>>>>>>>>>>>>> on the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the
> volume to
> > be
> > >>>>>>>>>>>> volume to be
> > >>>>>>>>>>>>>>>>>>>>>> managed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To
> ensure
> > >>> that
> > >>>>> we
> > >>>>>>>>>>>> are in
> > >>>>>>>>>>>>>>>>>>> sync on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these
> definitions,
> > >>> refers to
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>> physical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct?
> Minimally,
> > we
> > >>>>>>> must
> > >>>>>>>>>>>> be
> > >>>>>>>>>>>>>> able
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to
> move bits
> > >>> from
> > >>>>>>>>>>>> one place
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> another,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a
> > >>> storage
> > >>>>>>>>>>>> device
> > >>>>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>> able to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore,
> we can have
> > a
> > >>>>>>>>>>>> unmanaged
> > >>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we
> can have a
> > >>>>>>> managed
> > >>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable
> of
> > >>> allocating
> > >>>>>>>>>>>> LUNs and
> > >>>>>>>>>>>>>>>>>>> storing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may
> be able
> > >>> to
> > >>>>>>>>>>>> manage a
> > >>>>>>>>>>>>>>>>>>> device,
> > >>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it
> unmanaged by
> > >>>>>>> CloudStack
> > >>>>>>>>>>>> (e.g.
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> device is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the
> operator has
> > >>> chosen
> > >>>>>>> to
> > >>>>>>>>>>>>>>>> dedicate
> > >>>>>>>>>>>>>>>>>>>>>> only a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my
> > reasoning
> > >>>>> make
> > >>>>>>>>>>>> sense?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are
> reasonable, it
> > >>> seems
> > >>>>>>>>>>>>>> appropriate
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from
> > >>> StoragePoolType,
> > >>>>>>>>>>>> add the
> > >>>>>>>>>>>>>>>>>>> notion
> > >>>>>>>>>>>>>>>>>>>>>> of a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver
> that
> > >>>>> indicates
> > >>>>>>>>>>>> whether
> > >>>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>>>>>> not is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish
> a
> > >>> abstraction
> > >>>>>>>>>>>>>>>> representing a
> > >>>>>>>>>>>>>>>>>>>>>> physical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is
> > >>> associated
> > >>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>> volume.
> > >>>>>>>>>>>>>>>>>>>>>> With
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor
> drivers can
> > >>>>>>> declare
> > >>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>>>>>> protocols they
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a
> device
> > managed
> > >>> by
> > >>>>>>>>>>>>>> CloudStack,
> > >>>>>>>>>>>>>>>>>>>>>> utilize the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed by
> the driver to
> > >>>>>>> automate
> > >>>>>>>>>>>>>>>>>>> allocation.
> > >>>>>>>>>>>>>>>>>>>>>> If
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense,
> then we can
> > >>> sit
> > >>>>>>>>>>>> down and
> > >>>>>>>>>>>>>>>>>>> drill
> > >>>>>>>>>>>>>>>>>>>>>> down to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the
> current
> > iSCSI
> > >>>>>>> type
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> Dynamic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and
> create
> > a
> > >>>>>>>>>>>> Primary
> > >>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>> based
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time,
> the
> > iSCSI
> > >>>>>>>>>>>> volume must
> > >>>>>>>>>>>>>>>>>>> exist
> > >>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated).
> Future
> > >>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created
> behind the
> > >>>>>>> scenes.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in
> and
> > create
> > >>>>>>>>>>>> Primary
> > >>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>> based
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete
> volumes on
> > >>> its
> > >>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>> system
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the
> storage
> > >>>>>>>>>>>> framework). When
> > >>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>> user
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was
> created,
> > the
> > >>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>> tells
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After
> this is
> > >>> done,
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called.
> No
> > >>> hypervisor
> > >>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was
> just
> > >>> created.
> > >>>>>>> The
> > >>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike
> > Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should
> probably
> > have
> > >>>>>>>>>>>> defined them
> > >>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>> front
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-
> allocated
> > >>> (ex.
> > >>>>>>>>>>>> an admin
> > >>>>>>>>>>>>>>>>>>>>>> creates a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor
> data
> > >>> structure
> > >>>>>>> is
> > >>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then
> that
> > >>>>>>> hypervisor
> > >>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI)
> are later
> > >>>>>>> placed
> > >>>>>>>>>>>> on
> > >>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these
> cases, the
> > >>>>>>> attach
> > >>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>> assumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already
> in place
> > >>> and
> > >>>>>>>>>>>> simply
> > >>>>>>>>>>>>>>>>>>> attaches
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to
> the VM in
> > >>>>>>>>>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
> > >>>>> pre-allocated.
> > >>>>>>>>>>>> Instead
> > >>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a
> SAN
> > (not
> > >>> a
> > >>>>>>>>>>>> volume on
> > >>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>> SAN,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data
> structure
> > >>> must be
> > >>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>> when an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these
> types of
> > >>> volumes
> > >>>>>>>>>>>> have not
> > >>>>>>>>>>>>>>>>>>> been
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data
> structure by an
> > >>>>> admin.
> > >>>>>>>>>>>> Once
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for
> this
> > >>> volume,
> > >>>>>>> it
> > >>>>>>>>>>>>>> attaches
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in
> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM,
> John Burwell
> > <
> > >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the
> Dynamic
> > type
> > >>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>> behavior
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask
> about the
> > >>>>>>>>>>>> difference.
> > >>>>>>>>>>>>>>>>>>> Another
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the
> definition
> > of
> > >>> a
> > >>>>>>>>>>>> Dynamic
> > >>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike
> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is
> > uniquely
> > >>>>> used
> > >>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>> XenServer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage
> that is
> > >>> directly
> > >>>>>>>>>>>> based on
> > >>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>> iSCSI
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of
> going
> > to
> > >>> the
> > >>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that
> iSCSI
> > >>> target as
> > >>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>> does
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only
> supported for
> > >>>>>>>>>>>> XenServer.
> > >>>>>>>>>>>>>> For
> > >>>>>>>>>>>>>>>>>>> all
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> other
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to
> the
> > >>> hypervisor
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> perform
> > >>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> step
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM,
> John
> > Burwell
> > >>> <
> > >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what
> is the
> > >>>>> difference
> > >>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD
> considered
> > >>>>>>> Dynamic?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike
> > Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is
> defined in
> > the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum
> StoragePoolType {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local
> directory
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), //
> NFS or CIFS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN,
> with a
> > >>>>> clusterfs
> > >>>>>>>>>>>>>> overlay
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS
> Comstar
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local
> LVM SR
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS
> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for
> XenServer, Storage
> > >>> Pool
> > >>>>>>>>>>>> is set
> > >>>>>>>>>>>>>>>> up
> > >>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local
> EXT SR
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic,
> zone-wide
> > >>> storage
> > >>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>> SolidFire)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean
> shared) {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM,
> Mike
> > >>> Tutkowski
> > >>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another
> storage
> > >>>>> company
> > >>>>>>>>>>>> wants
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality
> of
> > Service
> > >>>>>>>>>>>> feature. It
> > >>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well.
> They would
> > >>> need
> > >>>>>>> only
> > >>>>>>>>>>>>>>>>>>> implement a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the
> necessary
> > >>> changes to
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39
> PM, Mike
> > >>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the
> XenServer code
> > is
> > >>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>> specific.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is
> now aware
> > >>> of
> > >>>>>>>>>>>> dynamic,
> > >>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an
> implementation of
> > >>> this
> > >>>>>>>>>>>> kind of
> > >>>>>>>>>>>>>>>>>>>>>> storage).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with
> Edison's
> > >>> storage
> > >>>>>>>>>>>> framework
> > >>>>>>>>>>>>>>>>>>>>>> changes.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new
> framework that
> > >>>>> supported
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> creation
> > >>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically.
> However, when I
> > >>>>>>>>>>>> visited with
> > >>>>>>>>>>>>>>>> him
> > >>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it
> was not
> > >>>>>>>>>>>> complete. We
> > >>>>>>>>>>>>>>>>>>> realized
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do
> with these
> > >>>>>>> volumes
> > >>>>>>>>>>>> unless
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this
> new type of
> > >>>>> storage
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> create
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28
> PM, John
> > >>> Burwell
> > >>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me
> that any
> > >>>>> operation
> > >>>>>>>>>>>> in the
> > >>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about
> details. I
> > >>> expect
> > >>>>>>>>>>>> to see
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> Storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations
> that can be
> > >>>>>>>>>>>>>> composed/driven
> > >>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to
> allocate
> > space/create
> > >>>>>>>>>>>> structures
> > >>>>>>>>>>>>>>>> per
> > >>>>>>>>>>>>>>>>>>>>>> their
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this
> dependency, we are
> > >>> going
> > >>>>>>> to
> > >>>>>>>>>>>> end
> > >>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the
> system
> > >>>>>>> increasingly
> > >>>>>>>>>>>>>>>>>>> difficult to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I
> understanding that the
> > >>> Xen
> > >>>>>>>>>>>> specific
> > >>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> code
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the
> CitrixResourceBase
> > class?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12
> PM, Mike
> > >>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit
> more
> > >>> detail:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from
> one setup
> > >>>>>>> method
> > >>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>> XenServer,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on
> the
> > storage
> > >>>>>>>>>>>> system,
> > >>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure
> to make
> > use
> > >>> of
> > >>>>>>> the
> > >>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>> (ex.
> > >>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer
> or a
> > datastore
> > >>> on
> > >>>>>>>>>>>> ESX(i)).
> > >>>>>>>>>>>>>>>> VMs
> > >>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage
> system's volume.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage
> framework,
> > >>>>>>> storage
> > >>>>>>>>>>>> need
> > >>>>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>>>>>> longer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create
> a 1:1
> > >>>>>>> relationship
> > >>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data
> disk
> > >>> (necessary
> > >>>>>>> for
> > >>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>> Quality
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-
> in that is
> > >>> called
> > >>>>>>> to
> > >>>>>>>>>>>>>>>>>>> dynamically
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> create
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the
> storage
> > framework
> > >>> did
> > >>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>> address
> > >>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-
> specific data
> > >>>>>>>>>>>> structure
> > >>>>>>>>>>>>>> when
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've
> been
> > >>> enhancing
> > >>>>> it
> > >>>>>>>>>>>> to do
> > >>>>>>>>>>>>>> so.
> > >>>>>>>>>>>>>>>>>>> I've
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> got
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and
> submitted. I've got
> > >>> ESX(i)
> > >>>>>>> in
> > >>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>> sandbox
> > >>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2
> freeze date.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 1:03 PM, Mike
> > >>>>>>>>>>>> Tutkowski <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by
> itself - is
> > >>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>> agnostic.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the
> volume-attach
> > >>> logic
> > >>>>>>>>>>>> (in the
> > >>>>>>>>>>>>>>>>>>> agent
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the
> plug-in to
> > >>> have
> > >>>>>>> it
> > >>>>>>>>>>>>>> create a
> > >>>>>>>>>>>>>>>>>>>>>> volume
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to
> attach the
> > >>> volume
> > >>>>>>> to
> > >>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>> hypervisor,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough
> to recognize
> > >>> it's
> > >>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>> invoked on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has
> just been
> > >>> created)
> > >>>>>>> and
> > >>>>>>>>>>>>>> create,
> > >>>>>>>>>>>>>>>>>>>>>> say, a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for
> XenServer) or a
> > >>> datastore
> > >>>>>>>>>>>> (for
> > >>>>>>>>>>>>>>>>>>> VMware) to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just
> created.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most
> of my time
> > >>>>>>> recently
> > >>>>>>>>>>>> making
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at
> 12:48 PM,
> > John
> > >>>>>>>>>>>> Burwell <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why
> the the storage
> > >>>>>>> driver
> > >>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21
> PM, Mike
> > >>>>>>> Tutkowski
> > >>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I
> would like to
> > >>> support
> > >>>>>>>>>>>> all
> > >>>>>>>>>>>>>>>>>>> hypervisors
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm
> just out of
> > >>> time
> > >>>>>>>>>>>> for 4.2
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> get
> > >>>>>>>>>>>>>>>>>>>>>> KVM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in
> supports
> > >>>>>>> XenServer.
> > >>>>>>>>>>>>>>>>>>> Depending on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> what
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature
> freeze, I
> > >>> have
> > >>>>> it
> > >>>>>>>>>>>> working
> > >>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear,
> this is
> > all
> > >>> in
> > >>>>>>>>>>>> regards
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> Disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute
> Offerings post
> > 4.2.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at 11:14 AM,
> > >>> Kelcey
> > >>>>>>>>>>>> Jamison
> > >>>>>>>>>>>>>>>>>>> Damage
> > >>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on
> supporting
> > >>> KVM in
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>> cycle
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> post
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original
> Message -----
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike
> Tutkowski" <
> > >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:
> dev@cloudstack.apache.org
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June
> 3, 2013
> > >>> 10:12:32 AM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
> > >>>>> disk_io_throttling
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>> MASTER
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging
> Wei's feature
> > >>>>> first,
> > >>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>> mine.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for
> KVM only,
> > >>> then
> > >>>>> it
> > >>>>>>>>>>>> is a
> > >>>>>>>>>>>>>> non
> > >>>>>>>>>>>>>>>>>>> issue
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013
> at 8:55 AM,
> > Wei
> > >>>>>>> ZHOU
> > >>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as
> no one works
> > >>> on
> > >>>>>>>>>>>> billing
> > >>>>>>>>>>>>>> now,
> > >>>>>>>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by
> themselves. They
> > >>> can
> > >>>>>>> get
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a
> VMs and
> > volumes
> > >>>>> for
> > >>>>>>>>>>>>>>>>>>> calculation.
> > >>>>>>>>>>>>>>>>>>>>>> Of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> course
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the
> exact
> > limitation
> > >>>>>>> value
> > >>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>> individual
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for
> nics as
> > well.
> > >>> I
> > >>>>>>> can
> > >>>>>>>>>>>> work
> > >>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O
> throttling?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my
> misunstand the second
> > the
> > >>>>>>>>>>>> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what
> you said about
> > the
> > >>>>> two
> > >>>>>>>>>>>>>> features.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> Burwell <
> > >>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at
> 2:13 AM, Wei
> > >>> ZHOU
> > >>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's
> aswer helps you.
> > >>> I am
> > >>>>>>>>>>>> trying
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>> adding
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing
> should
> > >>> depend
> > >>>>> on
> > >>>>>>>>>>>> IO
> > >>>>>>>>>>>>>>>>>>> statistics
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please
> review
> > >>>>>>>>>>>> disk_io_stat if
> > >>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO
> statistics
> > including
> > >>>>>>>>>>>> bytes/iops
> > >>>>>>>>>>>>>>>>>>>>>> read/write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS
> model,
> > customers
> > >>>>> are
> > >>>>>>>>>>>> billed
> > >>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS,
> as well as,
> > for
> > >>>>>>> those
> > >>>>>>>>>>>>>>>>>>> operations (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> http://aws.amazon.com/ebs/).
> > I
> > >>>>>>> would
> > >>>>>>>>>>>>>> imagine
> > >>>>>>>>>>>>>>>>>>> our
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ
> similar cost
> > >>>>> models.
> > >>>>>>>>>>>> Could
> > >>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in
> the current
> > >>> patch?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean
> IOPS runtime
> > >>>>> change?
> > >>>>>>>>>>>> KVM
> > >>>>>>>>>>>>>>>>>>> supports
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a
> running
> > virtual
> > >>>>>>>>>>>> machine
> > >>>>>>>>>>>>>>>> through
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> command
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does
> not support
> > >>>>> changing
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> parameters
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer
> offering or disk
> > >>>>>>> offering).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the
> Java interface
> > >>> level.
> > >>>>>>> I
> > >>>>>>>>>>>>>>>> apologize
> > >>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> being
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more
> generalize allocation
> > >>>>>>>>>>>> algorithms
> > >>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>> set
> > >>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the
> service guarantees
> > >>>>>>>>>>>> provided by a
> > >>>>>>>>>>>>>>>>>>>>>> resource?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good
> question.
> > >>> Maybe it
> > >>>>>>> is
> > >>>>>>>>>>>>>> better
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> disk_io_throttling as Mike
> > >>> needs to
> > >>>>>>>>>>>> consider
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on
> my thoughts
> > in a
> > >>>>>>> later
> > >>>>>>>>>>>>>>>> response
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points
> between these two
> > >>>>>>>>>>>> features. I
> > >>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be
> merged before
> > >>>>>>>>>>>> SolidFire, but
> > >>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination
> between the
> > branches
> > >>>>>>>>>>>> (possibly
> > >>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>> have
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling)
> to
> > >>> coordinate on
> > >>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>> issue.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John
> Burwell <
> > >>>>>>>>>>>> jburwell@basho.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I
> want to
> > understand
> > >>>>> are
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>> following:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there
> value in
> > capturing
> > >>>>> IOPS
> > >>>>>>>>>>>>>> policies
> > >>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g.
> for
> > >>> billing/usage
> > >>>>>>>>>>>>>> purposes,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there
> be a common
> > >>>>>>>>>>>> interface model
> > >>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at
> runtime?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are
> conflicting
> > >>> provisioned
> > >>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and
> storage device
> > >>>>>>>>>>>> reconciled?
> > >>>>>>>>>>>>>> In
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to
> believe (and
> > billed)
> > >>>>> for
> > >>>>>>>>>>>> more
> > >>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device
> has been
> > >>> configured
> > >>>>>>> to
> > >>>>>>>>>>>>>>>> deliver.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent
> configuration
> > >>> between a
> > >>>>>>>>>>>> VM and a
> > >>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a
> later
> > modification
> > >>> to
> > >>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013,
> at 8:38 PM,
> > >>> Mike
> > >>>>>>>>>>>> Tutkowski
> > >>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> mike.tutkowski@solidfire.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's
> feature deals
> > >>> with
> > >>>>>>>>>>>>>>>> controlling
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> max
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor
> side.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is
> focused on
> > >>>>>>> controlling
> > >>>>>>>>>>>> IOPS
> > >>>>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that
> helps. :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2,
> 2013 at 6:35
> > PM,
> > >>>>>>> John
> > >>>>>>>>>>>>>> Burwell
> > >>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is
> that no
> > features
> > >>>>>>>>>>>> should be
> > >>>>>>>>>>>>>>>>>>> merged
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have
> been resolved
> > and
> > >>> it
> > >>>>>>> is
> > >>>>>>>>>>>> ready
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> turn
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs
> discrete
> > >>> read/write
> > >>>>>>>>>>>> ops issue
> > >>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't
> think this
> > >>> criteria
> > >>>>>>>>>>>> has been
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how
> does this work
> > >>>>>>>>>>>>>>>> intersect/compliment
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> https://reviews.apache.org/r/11479/
> > >>>>>>>>>>>> )?
> > >>>>>>>>>>>>>> As I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves
> provisioned
> > >>> IOPS. I
> > >>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> where provisioned
> > >>> IOPS
> > >>>>> in
> > >>>>>>>>>>>> KVM and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013,
> at 6:47 AM,
> > >>> Wei
> > >>>>>>>>>>>> ZHOU <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will
> change it next
> > >>> week.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1
> Wido den Hollander
> > <
> > >>>>>>>>>>>>>> wido@widodh.nl
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> 06/01/2013 08:24 AM, Wei
> > >>> ZHOU
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have
> pushed the
> > >>>>>>> features
> > >>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>> master.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone
> object thems for
> > >>>>>>> technical
> > >>>>>>>>>>>>>> reason
> > >>>>>>>>>>>>>>>>>>> till
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake
> of clarity I
> > just
> > >>>>>>> want
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> mention
> > >>>>>>>>>>>>>>>>>>>>>> again
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps
> to R/W IOps
> > >>> asap
> > >>>>> so
> > >>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>>>> never
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the
> groundwork for
> > >>> the
> > >>>>>>> I/O
> > >>>>>>>>>>>>>>>> throttling
> > >>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however
> prevent that we
> > create
> > >>>>>>>>>>>> legacy from
> > >>>>>>>>>>>>>>>> day
> > >>>>>>>>>>>>>>>>>>>>>> #1.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> Wido den
> > Hollander <
> > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> 05/31/2013 03:59 PM, John
> > >>>>>>> Burwell
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this
> enhancement must
> > to
> > >>>>>>>>>>>> discretely
> > >>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how
> it could be
> > >>> fixed
> > >>>>>>>>>>>> later
> > >>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS
> into read
> > and
> > >>>>>>>>>>>> write.
> > >>>>>>>>>>>>>>>>>>>>>> Therefore, we
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> unless/until we
> > decided
> > >>> to
> > >>>>>>>>>>>> break
> > >>>>>>>>>>>>>>>>>>> backwards
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei
> meant was merging
> > it
> > >>>>> into
> > >>>>>>>>>>>> master
> > >>>>>>>>>>>>>>>> now
> > >>>>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch
> and add Read /
> > >>> Write
> > >>>>>>> IOps
> > >>>>>>>>>>>>>> before
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be
> released with Read
> > and
> > >>>>>>> Write
> > >>>>>>>>>>>>>> instead
> > >>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to
> make the May 31st
> > >>>>>>> feature
> > >>>>>>>>>>>>>> freeze
> > >>>>>>>>>>>>>>>>>>> date.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other
> threads) then it
> > >>> won't
> > >>>>>>> be
> > >>>>>>>>>>>>>>>> necessary
> > >>>>>>>>>>>>>>>>>>>>>> to do
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also
> completely agree that
> > >>>>> there
> > >>>>>>>>>>>> is no
> > >>>>>>>>>>>>>>>>>>>>>> association
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31,
> 2013, at 9:51 AM,
> > >>> Wido
> > >>>>>>>>>>>> den
> > >>>>>>>>>>>>>>>>>>> Hollander <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> 05/31/2013 03:13 PM, Wei
> > >>> ZHOU
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good
> question.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought
> about at the
> > >>>>> beginning.
> > >>>>>>>>>>>>>> Finally I
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of
> read and write
> > >>>>>>> mainly
> > >>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the
> difference of sent
> > >>> and
> > >>>>>>>>>>>> received
> > >>>>>>>>>>>>>>>>>>> bytes as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That
> reasoning seems odd.
> > >>>>>>>>>>>> Networking and
> > >>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is
> much more
> > >>> expensive
> > >>>>> in
> > >>>>>>>>>>>> most
> > >>>>>>>>>>>>>>>>>>>>>> situations
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing
> it will be some
> > >>>>>>>>>>>> copy-paste
> > >>>>>>>>>>>>>>>> work.
> > >>>>>>>>>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented
> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For
> the deadline
> > of
> > >>>>>>>>>>>> feature
> > >>>>>>>>>>>>>>>> freeze,
> > >>>>>>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if
> needed.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a
> feature we
> > >>> can't
> > >>>>>>>>>>>> miss. But
> > >>>>>>>>>>>>>>>> if
> > >>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we
> have to make sure
> > we
> > >>>>>>> don't
> > >>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that
> will confuse
> > >>>>>>> users.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31
> Wido den
> > Hollander <
> > >>>>>>>>>>>>>>>> wido@widodh.nl>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On
> 05/30/2013 06:03 PM, Wei
> > >>> ZHOU
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to
> merge
> > >>>>>>>>>>>> disk_io_throttling
> > >>>>>>>>>>>>>>>>>>> branch
> > >>>>>>>>>>>>>>>>>>>>>> into
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody
> object, I will
> > merge
> > >>>>>>> into
> > >>>>>>>>>>>> master
> > >>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>> 48
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose
> is :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual
> machines are running
> > >>> on
> > >>>>>>> the
> > >>>>>>>>>>>> same
> > >>>>>>>>>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage).
> Because of
> > the
> > >>>>> rate
> > >>>>>>>>>>>>>>>> limitation
> > >>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has
> large disk
> > >>> operation,
> > >>>>>>> it
> > >>>>>>>>>>>> may
> > >>>>>>>>>>>>>>>> affect
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs
> running on the
> > same
> > >>>>>>>>>>>> storage
> > >>>>>>>>>>>>>>>> device.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary
> to set the
> > >>>>> maximum
> > >>>>>>>>>>>> rate
> > >>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>> limit
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the
> code I see
> > you
> > >>>>> make
> > >>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>>> difference
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and
> libvirt support
> > >>> setting
> > >>>>>>>>>>>> both a
> > >>>>>>>>>>>>>>>>>>> different
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which
> could benefit a
> > >>> lot of
> > >>>>>>>>>>>> users.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also
> strange, in the
> > >>> polling
> > >>>>>>>>>>>> side you
> > >>>>>>>>>>>>>>>>>>>>>> collect
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> both
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on
> the throttling
> > >>> side
> > >>>>>>>>>>>> you only
> > >>>>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are
> usually much
> > >>> more
> > >>>>>>>>>>>> expensive
> > >>>>>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>>>>>> Read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid
> use-case where
> > >>> that
> > >>>>>>> an
> > >>>>>>>>>>>> admin
> > >>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>> set
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read
> IOps.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only
> supports KVM
> > >>> at
> > >>>>>>> this
> > >>>>>>>>>>>>>> point I
> > >>>>>>>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at
> least have the
> > >>>>>>>>>>>> mechanism in
> > >>>>>>>>>>>>>>>> place
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later
> would be a lot of
> > >>>>> work.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor
> doesn't
> > >>> support
> > >>>>>>>>>>>> setting
> > >>>>>>>>>>>>>>>>>>> different
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> values
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can
> always sum
> > both
> > >>> up
> > >>>>>>>>>>>> and set
> > >>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you
> explain why you
> > >>>>>>> implemented
> > >>>>>>>>>>>> it
> > >>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>> way?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature
> includes:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> maximum rate of
> > >>> VMs
> > >>>>>>> (in
> > >>>>>>>>>>>>>>>>>>>>>> disk_offering,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the
> maximum rate
> > of
> > >>>>> VMs
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the
> disk rate
> > (total
> > >>>>> bps
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>>>> iops)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> > >>>>>>>>>>>>>> https://issues.apache.org/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> jira/browse/CLOUDSTACK-1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> 1192>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> 1192<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will
> update later) :
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>
> >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > VM+Disk+IO+Throttling<https://
> > >>>>>>> ****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> cwiki.apache.org/confluence/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> rottling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check
> list :-
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you
> check the branch's
> > >>> RAT
> > >>>>>>>>>>>> execution
> > >>>>>>>>>>>>>>>>>>>>>> success?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there
> new dependencies
> > >>>>>>>>>>>> introduced?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> automated testing
> > (unit
> > >>>>> and
> > >>>>>>>>>>>>>>>>>>> integration)
> > >>>>>>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are
> added.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What
> testing has been done
> > >>> to
> > >>>>>>>>>>>> check for
> > >>>>>>>>>>>>>>>>>>>>>> potential
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the
> bytes rate and
> > >>> IOPS
> > >>>>>>>>>>>> rate on
> > >>>>>>>>>>>>>>>>>>>>>> CloudStack
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM
> operations, including
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop,
> start, reboot,
> > >>>>>>>>>>>> destroy,
> > >>>>>>>>>>>>>>>> expunge.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume
> operations,
> > >>> including
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach,
> Detach
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the
> code, you can
> > >>> try
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> c30057635d04a2396f84c588127d7e******be42e503a7
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>
> >
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> > >>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > VM+Disk+IO+Throttling<https://
> > >>>>>>> ****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> cwiki.apache.org/confluence/****
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Th
> rottling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > >>> refs/heads/disk_io_throttling
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> 1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> 1301>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> 1301<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> > >>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> 2071
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-
> 2071
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-
> 2071
> > >>>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-
> 2071>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> (**CLOUDSTACK-1301
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk
> I/O
> > Throttling)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike
> Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior
> CloudStack Developer,
> > >>>>>>>>>>>> SolidFire
> > >>>>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> > >>> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the
> way the world
> > >>> uses
> > >>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > >>>>> SolidFire
> > >>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the world uses
> > >>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > >>> SolidFire
> > >>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way
> the world uses
> > the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > >>> SolidFire
> > >>>>>>>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world uses the
> > >>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer,
> > SolidFire
> > >>>>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack
> Developer, SolidFire
> > >>>>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the
> world uses the
> > >>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > >>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses the
> > >>> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > >>> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> > Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the cloud<
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> > >>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> > >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>>>>> cloud<
> > >>>>> http://solidfire.com/solution/overview/?video=play
> > >>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>>>> cloud<
> > >>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play
> > >
> > >>>>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>>>> Advancing the way the world uses the
> > >>>>>>>>>>>>>
> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>>> *(tm)*
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>>> o: 303.746.7302
> > >>>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>>> *(tm)*
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> *Mike Tutkowski*
> > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>>> o: 303.746.7302
> > >>>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>>> *(tm)*
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> *Mike Tutkowski*
> > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>>> o: 303.746.7302
> > >>>>>>>> Advancing the way the world uses the cloud<
> > >>>>>>> http://solidfire.com/solution/overview/?video=play>
> > >>>>>>>> *(tm)*
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> *Mike Tutkowski*
> > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>>>>> e: mike.tutkowski@solidfire.com
> > >>>>>>> o: 303.746.7302
> > >>>>>>> Advancing the way the world uses the
> > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>>>>> *(tm)*
> > >>>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> *Mike Tutkowski*
> > >>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >>>> e: mike.tutkowski@solidfire.com
> > >>>> o: 303.746.7302
> > >>>> Advancing the way the world uses the
> > >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >>>> *(tm)*
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> *Mike Tutkowski*
> > >> *Senior CloudStack Developer, SolidFire Inc.*
> > >> e: mike.tutkowski@solidfire.com
> > >> o: 303.746.7302
> > >> Advancing the way the world uses the cloud<
> > http://solidfire.com/solution/overview/?video=play>
> > >> *(tm)*
> > >>
> > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *(tm)*
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
How's about this:
The driver can implement an isManaged() method. The VolumeManagerImpl can
call into the driver to see if its managed. If it is, the VolumeManagerImpl
(which is responsible for calling into the hypervisor to attach the disk)
can call into the hypervisor to create the necessary hypervisor data
structure (ex. for XenServer, a storage repository).
If that's what you're going for, that works for me. By the way, Edison's
default storage plug-in (which handles the default storage behavior in
CloudStack (ex. how pre 4.2 works)) does include code that talks to
hypervisors. You might want to contact him and inform him of your concerns
or that logic (as is) will make it to production.
Please let me know if what I wrote in above (for my solution) is OK with
you. :)
Thanks!
On Fri, Jun 7, 2013 at 1:49 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Please see my responses in-line below.
>
> Thanks,
> -John
>
> On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hey John,
> >
> > I still have a bit more testing I'd like to do before I build up a patch
> > file, but this is the gist of what I've done:
> >
> > * During a volume-attach operation, after VolumeManagerImpl tells
> > VolumeServiceImpl to have the driver create a volume, I have
> > VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it managed.
> > If it is managed, VolumeServiceImpl has the driver perform whatever
> > activity is required. In my case, this includes sending a message to the
> > host where the VM is running to have, say XenServer, add a storage
> > repository (based on the IP address of the SAN, the IQN of the SAN
> volume,
> > etc.) and a single VDI (the VDI consumes all of the space it can on the
> > storage repository). After this, the normal attach-volume message is sent
> > to the host by VolumeManagerImpl.
>
> There should be **no** code from a storage driver to a hypervisor. I
> apologize for the repetition, but we simply can not have hypervisor
> specific code in the storage layer. The circular dependencies between the
> two layers are not sustainable in the long term. Either the VirtualManager
> or Xen hypervisor plugin needs to be refactored/modified to coordinate
> volume creation and then populating the SR. Ideally, we can generalize the
> process flow for attaching volumes such that the Xen hypervisor plugin
> would only implement callbacks to perform the attach action and create the
> structure and SR. To my mind, the SolidFire driver should only be
> allocating space and providing information about contents (e.g. space
> available, space consumed, streams to a URI, file handle for a URI, etc)
> and capabilities.
>
>
> >
> > * The reverse is performed for a detach-volume command.
> >
> > * Right now I simply "return true;" for isManaged() in my driver.
> Edison's
> > default driver simply does a "return false;". We could add a new
> parameter
> > to the createStoragePool API command, if we want, to remove the
> hard-coded
> > return values in the drivers (although my driver will probably just
> ignore
> > this parameter and always return true since it wouldn't make sense for it
> > to ever return false). We'd need another column in the storage_pool table
> > to store this value.
>
> Yes, I think we should have a parameter added to the createStoragePool
> surfaced to the HTTP API that allows DataStores to be configured for
> management when their underlying drivers support it. To simplify things,
> this flag should only be mutable when the DataStore is created. It would be
> a bit crazy to take a DataStore from managed to unmanaged after creation.
>
> >
> > Sound like I'm in sync with what you were thinking?
> >
> >
> >
> >
> > On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >> wrote:
> >
> >> I agree, John. Just wanted to point out that I have a working GUI for
> you
> >> to review (in that document), if you'd like to check it out.
> >>
> >> Thanks!
> >>
> >>
> >> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Mike,
> >>>
> >>> I would like the UIs of two features reviewed together to ensure
> >>> consistency across the concepts of hypervisor throttled IOPs and
> >>> storage device provisioned IOPs. I see the potential for confusion,
> >>> and I think a side-by-side Ui review of these features will help
> >>> minimize any potential confusion.
> >>>
> >>> As I mentioned, the term reconciliation issue will work itself if it
> >>> is acceptable that a VM is only permitted utilize hypervisor throttled
> >>> IOPs or storage provisioned IOPs.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> >>> <mi...@solidfire.com> wrote:
> >>>
> >>>> Hi John,
> >>>>
> >>>> Yeah, when you get a chance, refer to the Google doc I sent to you the
> >>>> other day to see how the GUI looks for provisioned storage IOPS.
> >>>>
> >>>> Several months ago, I put this topic out on the e-mail list and we
> >>> decided
> >>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering dialog.
> >>>> Other storage vendors are coming out with QoS, so they should be able
> to
> >>>> leverage this GUI going forward (even if they, say, only use Max
> IOPS).
> >>>> These fields are optional and can be ignored for storage that does not
> >>>> support provisioned IOPS. Just like the Disk Size field, the admin can
> >>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
> >>>>
> >>>> I'm OK if we do an either/or model (either Wei's feature or mine, as
> is
> >>>> decided by the admin).
> >>>>
> >>>> I'm not sure what we can do about these two features expressing the
> >>> speed
> >>>> in different terms. I've never seen a SAN express the IOPS for QoS in
> >>> any
> >>>> way other than total IOPS (i.e. not broken in into read/write IOPS).
> >>>>
> >>>> Thanks
> >>>>
> >>>>
> >>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com>
> >>> wrote:
> >>>>
> >>>>> Wei,
> >>>>>
> >>>>> We have been down the rabbit hole a bit on the Storage/Hypervisor
> layer
> >>>>> separation, but we still need to reconcile the behavior of hypervisor
> >>>>> throttled I/O and storage provisioned IOPS. I see the following
> issues
> >>>>> outstanding:
> >>>>>
> >>>>> 1. Hypervisor throttled IOPS are expressed as discrete read/write
> >>> values
> >>>>> where as storage provisioned IOPS are expressed as total IOPS.
> >>>>> 2. How do we handle VMs with throttled IOPS attached to storage
> volumes
> >>>>> with provisioned IOPS?
> >>>>> 3. How should usage data be captured for throttled and provisioned
> IOPS
> >>>>> that will permit providers to discriminate these guaranteed
> operations
> >>> in
> >>>>> the event they want to bill for it differently?
> >>>>> 4. What is the user experience for throttled and provisioned IOPS
> that
> >>>>> minimizes confusion of these concepts?
> >>>>>
> >>>>> My thinking is that a VM can have either utilize hypervisor throttled
> >>> IOPS
> >>>>> or storage provisioned IOPS. This policy would neatly solve items 1
> >>> and 2.
> >>>>> Since the two facilities would not be permitted to operate together,
> >>> they
> >>>>> do not need to be semantically compatible. I think item 3 can be
> >>> resolved
> >>>>> with an additional flag or two on the usage records. As for Item 4,
> I
> >>> am
> >>>>> not familiar with how these two enhancements are depicted in the user
> >>>>> interface. I think we need to review the UI enhancements for both
> >>>>> enhancements and ensure they are consistent.
> >>>>>
> >>>>> Do these solutions make sense?
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com> wrote:
> >>>>>
> >>>>>> John and Mike,
> >>>>>>
> >>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> >>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this
> >>> week.
> >>>>>> I will start to develop on iops/bps changes tomorrow, and ask for
> >>> second
> >>>>>> merge request after testing.
> >>>>>>
> >>>>>> -Wei
> >>>>>>
> >>>>>>
> >>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> >>>>>>
> >>>>>>> I believe I understand where you're going with this, John, and have
> >>> been
> >>>>>>> re-working this section of code today.
> >>>>>>>
> >>>>>>> I should be able to run it by you tomorrow.
> >>>>>>>
> >>>>>>> Thanks for the comments,
> >>>>>>> Mike
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> See my responses in-line below.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi John,
> >>>>>>>>
> >>>>>>>> Thanks for the response.
> >>>>>>>>
> >>>>>>>> "I am fine with the VolumeManager determining whether or not a
> >>> Volume
> >>>>> is
> >>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
> >>> isManaged
> >>>>>>>> method), and asking the device driver to allocate resources for
> the
> >>>>>>> volume
> >>>>>>>> if it is managed."
> >>>>>>>>
> >>>>>>>> Are you thinking you'd like to see an isManaged() method added to
> >>> the
> >>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then the
> >>> storage
> >>>>>>>> framework could call the manage() (or whatever name) method (which
> >>>>> would
> >>>>>>> be
> >>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and this
> would
> >>>>> call
> >>>>>>>> into a new method in the hypervisor code to create, say on
> >>> XenServer,
> >>>>> an
> >>>>>>> SR?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I would like to see canBeManaged() : boolean on DataStoreDriver.
> >>> Since
> >>>>>>>> the notion of Volumes only pertains to primary storage, I would
> add
> >>> a
> >>>>>>>> allocateStorage and deallocateStorage (Storage is a straw man term
> >>> --
> >>>>>>>> something other than volume) methods to
> >>>>> allocate/create/deallocate/delete
> >>>>>>>> underlying storage. To my mind, managed is a mutable property of
> >>>>>>> DataStore
> >>>>>>>> which can be enabled if/when the underlying DataStoreDriver can be
> >>>>>>> managed.
> >>>>>>>> This approach allows operators to override manageability of
> devices.
> >>>>>>>>
> >>>>>>>> In terms of orchestration/process flow for SR, the Xen plugin
> would
> >>> be
> >>>>>>>> responsible for composing DataStore/Volume methods to create any
> >>>>>>>> directories or files necessary for the SR. There should be no
> >>>>>>> dependencies
> >>>>>>>> from the Storage to the Hypervisor layer. As I said earlier, such
> >>>>>>> circular
> >>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Just want to make sure I'm on the same page with you.
> >>>>>>>>
> >>>>>>>> Thanks again, John
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> Fundamentally, we can't end up with a Storage layer that
> supports n
> >>>>>>>>> devices types with each specific behaviors of m hypervisors.
> Such
> >>> a
> >>>>>>>>> scenario will create an unmaintainable and untestable beast.
> >>>>>>> Therefore, my
> >>>>>>>>> thoughts and recommendations are driven to evolve the Storage
> layer
> >>>>>>> towards
> >>>>>>>>> this separation.
> >>>>>>>>>
> >>>>>>>>> I am fine with the VolumeManager determining whether or not a
> >>> Volume
> >>>>> is
> >>>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
> >>>>> isManaged
> >>>>>>>>> method), and asking the device driver to allocate resources for
> the
> >>>>>>> volume
> >>>>>>>>> if it is managed. Furthermore, the device driver needs to
> indicate
> >>>>>>> whether
> >>>>>>>>> or not it supports management operations. Finally, I think we
> >>> need to
> >>>>>>>>> provide the ability for an administrator to elect to have
> something
> >>>>>>> that is
> >>>>>>>>> manageable be unmanaged (i.e. the driver is capable managing the
> >>>>> device,
> >>>>>>>>> but the administrator has elected to leave it unmanaged).
> >>>>>>>>>
> >>>>>>>>> Creation of a structure on the volume should be done in the Xen
> >>>>>>>>> hypervisor module using methods exposed by the Storage layer to
> >>>>> perform
> >>>>>>>>> low-level operations (e.g. make directories, create a file, etc).
> >>>>> This
> >>>>>>>>> structure is specific to the operation of the Xen hypervisor, as
> >>> such,
> >>>>>>>>> should be confined to its implementation. From my perspective,
> >>>>> nothing
> >>>>>>> in
> >>>>>>>>> the Storage layer should be concerned with content. From its
> >>>>>>> perspective,
> >>>>>>>>> structure and data are opaque. It provides the means to query
> the
> >>>>> data
> >>>>>>> to
> >>>>>>>>> support the interpretation of the content by higher-level layers
> >>> (e.g.
> >>>>>>>>> Hypervisors). To my mind, attach should be a composition of
> >>>>> operations
> >>>>>>>>> from the Storage layer that varies based on the Volume storage
> >>>>> protocol
> >>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi John,
> >>>>>>>>>
> >>>>>>>>> Alternatively to the way the attach logic is implemented in my
> >>> patch,
> >>>>> we
> >>>>>>>>> could do the following:
> >>>>>>>>>
> >>>>>>>>> Leave the attach logic in the agent code alone. In
> >>> VolumeManagerImpl
> >>>>> we
> >>>>>>>>> create an AttachVolumeCommand and send it to the hypervisor.
> Before
> >>>>> this
> >>>>>>>>> command is sent, we could check to see if we're dealing with
> >>> Dynamic
> >>>>> (or
> >>>>>>>>> whatever we want to call it) storage and - if so - send a "Create
> >>> SR"
> >>>>>>>>> command to the hypervisor. If this returns OK, we would then
> >>> proceed
> >>>>> to
> >>>>>>> the
> >>>>>>>>> AttachVolumeCommand, as usual.
> >>>>>>>>>
> >>>>>>>>> This way the attach logic remains the same and we just add
> another
> >>>>>>>>> command to the agent code that is called for this particular type
> >>> of
> >>>>>>>>> storage.
> >>>>>>>>>
> >>>>>>>>> What do you think?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hey John,
> >>>>>>>>>>
> >>>>>>>>>> I created a document for a customer today that outlines how the
> >>>>> plug-in
> >>>>>>>>>> works from a user standpoint. This will probably be of use to
> >>> you, as
> >>>>>>> well,
> >>>>>>>>>> as you perform the code review.
> >>>>>>>>>>
> >>>>>>>>>> I have shared this document with you (you should have received
> >>> that
> >>>>>>>>>> information in a separate e-mail).
> >>>>>>>>>>
> >>>>>>>>>> Talk to you later!
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <
> jburwell@basho.com
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts, but
> want
> >>> to
> >>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
> >>>>>>> disk_io_throttle,
> >>>>>>>>>>>> and
> >>>>>>>>>>>> object_store branches. I hope to collect them in a coherent
> >>> form
> >>>>>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the notion
> of
> >>> a
> >>>>>>>>>>>> managed
> >>>>>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the notion
> >>> of
> >>>>>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The storage framework that leverages the plug-in was
> >>> incomplete,
> >>>>> so
> >>>>>>>>>>>> I had
> >>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I appreciate all the time you've been spending on this. :) Do
> >>> you
> >>>>>>>>>>>> have a
> >>>>>>>>>>>>> recommendation as to how we should accomplish what you're
> >>> looking
> >>>>>>>>>>>> for?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> >>> jburwell@basho.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the notion
> of
> >>> a
> >>>>>>>>>>>> managed
> >>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I want to see a separation of the management capabilities
> >>> (i.e.
> >>>>>>> can
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>> device be managed/does an operator want it managed by
> >>> CloudStack)
> >>>>>>>>>>>> from the
> >>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a semantic
> >>> that
> >>>>>>>>>>>> will
> >>>>>>>>>>>>>> allow any type of storage device to be managed. I also want
> >>> to
> >>>>>>> make
> >>>>>>>>>>>>>> progress on decoupling the storage types from the hypervisor
> >>>>>>>>>>>> definitions.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> >>>>> jburwell@basho.com
> >>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Could you please answer the following questions for me
> with
> >>>>>>>>>>>> regards to
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN
> volumes?
> >>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
> >>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and CloudStack
> >>>>>>>>>>>> Volumes?
> >>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Are the LUN(s) created by the management server or
> >>> externally
> >>>>> by
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume (same
> as a
> >>>>> SAN
> >>>>>>>>>>>> LUN) is
> >>>>>>>>>>>>>>> created by the management server (via the plug-in) the
> first
> >>>>> time
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but still
> >>> want
> >>>>> to
> >>>>>>>>>>>> use a
> >>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already today
> (prior
> >>> to
> >>>>>>>>>>>> 4.2). The
> >>>>>>>>>>>>>>> admin manually creates the SAN volume and - in this case -
> >>>>>>>>>>>> multiple VMs
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> data disks can share this SAN volume. While you can do this
> >>>>>>> today,
> >>>>>>>>>>>> it is
> >>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Are the SAN volumes by the management server or externally
> >>> by
> >>>>>>> the
> >>>>>>>>>>>>>> operator?
> >>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes are
> >>>>>>> completely
> >>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>> by the management server (via the plug-in). There is no
> admin
> >>>>>>>>>>>>>> interaction.
> >>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume and a
> >>>>>>> CloudStack
> >>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>> which is necessary for any storage vendor that supports
> true
> >>>>> QoS.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I would like to clarify how these pieces are related and
> >>>>>>> expected
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs with
> >>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a single SAN
> >>> volume
> >>>>>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have multiple
> >>> CloudStack
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume house more
> >>> than
> >>>>>>> one
> >>>>>>>>>>>>>>>>> CloudStack volume, then can do that today without any of
> my
> >>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>> code.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume for
> >>>>> CloudStack's
> >>>>>>>>>>>> use
> >>>>>>>>>>>>>> onto
> >>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me --
> concepts
> >>>>> from
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only
> used
> >>>>> (as
> >>>>>>>>>>>> far as
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not PreSetup an
> >>> SR).
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from VMware, it
> >>> uses
> >>>>>>> the
> >>>>>>>>>>>> vmfs
> >>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM, it uses
> >>> the
> >>>>>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here was to
> >>> make a
> >>>>>>>>>>>> new type
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison added
> >>> into
> >>>>>>> the
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more specificy, though:
> >>>>>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or care
> that
> >>> it
> >>>>>>> is
> >>>>>>>>>>>> being
> >>>>>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer questions
> >>> about
> >>>>> it
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the SolidFire
> >>> plug-in
> >>>>>>>>>>>> needs to
> >>>>>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic
> >>>>> knows
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> treat
> >>>>>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint and just
> >>>>>>>>>>>> calling it
> >>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for the hypervisor
> >>>>>>> attach
> >>>>>>>>>>>> logic
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform the necessary
> >>>>>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
> >>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to manage a
> >>>>>>> device,
> >>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
> >>>>>>> device
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has chosen to
> >>> dedicate
> >>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via
> >>> the
> >>>>>>>>>>>> plug-in
> >>>>>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would work unless
> >>> the
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of CloudStack, an
> >>>>>>> operator
> >>>>>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it
> through
> >>> a
> >>>>>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to CloudStack,
> >>> but
> >>>>>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my changes, we
> >>> would
> >>>>>>> be
> >>>>>>>>>>>>>> passing
> >>>>>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the necessary
> >>> hypervisor
> >>>>>>>>>>>> support
> >>>>>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the storage
> >>>>>>> framework
> >>>>>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from the
> >>> hypervisor
> >>>>>>>>>>>> side and
> >>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to create the
> >>> SR
> >>>>>>>>>>>> before the
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin should
> have a
> >>>>>>> rich
> >>>>>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>>>>> interface to storage to determine available for
> volume
> >>>>>>>>>>>> storage.
> >>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>>>>> this interface would allow the interrogation of the
> >>>>> device
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself shouldn't
> >>> know
> >>>>>>>>>>>> or care
> >>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to
> answer
> >>>>>>>>>>>> questions
> >>>>>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver (which
> >>> is a
> >>>>>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
> >>>>>>>>>>>> Storage shows
> >>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that
> references
> >>> the
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack volume. This
> >>> leads
> >>>>>>> to
> >>>>>>>>>>>> a new
> >>>>>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack volume to a
> >>> VM
> >>>>>>>>>>>> (attach
> >>>>>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the plug-in
> >>> to
> >>>>>>>>>>>> create a
> >>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN). The
> >>> plug-in
> >>>>>>>>>>>> also
> >>>>>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data (like the
> >>> IQN of
> >>>>>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the
> CloudStack
> >>>>>>>>>>>> volume is in
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the volume
> >>> will be
> >>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the plug-in
> >>> code
> >>>>>>>>>>>> will not
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and detects
> the
> >>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a
> >>> hypervisor
> >>>>>>> data
> >>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on terminology,
> >>> volume,
> >>>>>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical allocation on
> >>> the
> >>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN
> >>> volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can make in
> >>>>>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well make sense, but
> >>> I'm
> >>>>>>>>>>>> having a
> >>>>>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on this
> >>> conversation
> >>>>>>>>>>>> as he
> >>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two distinct
> >>> concepts
> >>>>>>> --
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In both
> >>> cases, we
> >>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to
> create/delete
> >>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the volume to
> be
> >>>>>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure
> >>> that
> >>>>> we
> >>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these definitions,
> >>> refers to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct? Minimally,
> we
> >>>>>>> must
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to move bits
> >>> from
> >>>>>>>>>>>> one place
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a
> >>> storage
> >>>>>>>>>>>> device
> >>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have
> a
> >>>>>>>>>>>> unmanaged
> >>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
> >>>>>>> managed
> >>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable of
> >>> allocating
> >>>>>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able
> >>> to
> >>>>>>>>>>>> manage a
> >>>>>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
> >>>>>>> CloudStack
> >>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the operator has
> >>> chosen
> >>>>>>> to
> >>>>>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my
> reasoning
> >>>>> make
> >>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it
> >>> seems
> >>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from
> >>> StoragePoolType,
> >>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver that
> >>>>> indicates
> >>>>>>>>>>>> whether
> >>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish a
> >>> abstraction
> >>>>>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is
> >>> associated
> >>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
> >>>>>>> declare
> >>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a device
> managed
> >>> by
> >>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed by the driver to
> >>>>>>> automate
> >>>>>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can
> >>> sit
> >>>>>>>>>>>> down and
> >>>>>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the current
> iSCSI
> >>>>>>> type
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create
> a
> >>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the
> iSCSI
> >>>>>>>>>>>> volume must
> >>>>>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
> >>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
> >>>>>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and
> create
> >>>>>>>>>>>> Primary
> >>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on
> >>> its
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
> >>>>>>>>>>>> framework). When
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was created,
> the
> >>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is
> >>> done,
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called. No
> >>> hypervisor
> >>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was just
> >>> created.
> >>>>>>> The
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike
> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should probably
> have
> >>>>>>>>>>>> defined them
> >>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated
> >>> (ex.
> >>>>>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data
> >>> structure
> >>>>>>> is
> >>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
> >>>>>>> hypervisor
> >>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
> >>>>>>> placed
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
> >>>>>>> attach
> >>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already in place
> >>> and
> >>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
> >>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
> >>>>> pre-allocated.
> >>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN
> (not
> >>> a
> >>>>>>>>>>>> volume on
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure
> >>> must be
> >>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these types of
> >>> volumes
> >>>>>>>>>>>> have not
> >>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an
> >>>>> admin.
> >>>>>>>>>>>> Once
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this
> >>> volume,
> >>>>>>> it
> >>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell
> <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic
> type
> >>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
> >>>>>>>>>>>> difference.
> >>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the definition
> of
> >>> a
> >>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is
> uniquely
> >>>>> used
> >>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is
> >>> directly
> >>>>>>>>>>>> based on
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going
> to
> >>> the
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI
> >>> target as
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
> >>>>>>>>>>>> XenServer.
> >>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the
> >>> hypervisor
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John
> Burwell
> >>> <
> >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the
> >>>>> difference
> >>>>>>>>>>>> between
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
> >>>>>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike
> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a
> >>>>> clusterfs
> >>>>>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage
> >>> Pool
> >>>>>>>>>>>> is set
> >>>>>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide
> >>> storage
> >>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike
> >>> Tutkowski
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage
> >>>>> company
> >>>>>>>>>>>> wants
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of
> Service
> >>>>>>>>>>>> feature. It
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would
> >>> need
> >>>>>>> only
> >>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary
> >>> changes to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code
> is
> >>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware
> >>> of
> >>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of
> >>> this
> >>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's
> >>> storage
> >>>>>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that
> >>>>> supported
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
> >>>>>>>>>>>> visited with
> >>>>>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
> >>>>>>>>>>>> complete. We
> >>>>>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
> >>>>>>> volumes
> >>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of
> >>>>> storage
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John
> >>> Burwell
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any
> >>>>> operation
> >>>>>>>>>>>> in the
> >>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I
> >>> expect
> >>>>>>>>>>>> to see
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> >>>>>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate
> space/create
> >>>>>>>>>>>> structures
> >>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are
> >>> going
> >>>>>>> to
> >>>>>>>>>>>> end
> >>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
> >>>>>>> increasingly
> >>>>>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the
> >>> Xen
> >>>>>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase
> class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
> >>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more
> >>> detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
> >>>>>>> method
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the
> storage
> >>>>>>>>>>>> system,
> >>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make
> use
> >>> of
> >>>>>>> the
> >>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a
> datastore
> >>> on
> >>>>>>>>>>>> ESX(i)).
> >>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
> >>>>>>> storage
> >>>>>>>>>>>> need
> >>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
> >>>>>>> relationship
> >>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk
> >>> (necessary
> >>>>>>> for
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is
> >>> called
> >>>>>>> to
> >>>>>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage
> framework
> >>> did
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
> >>>>>>>>>>>> structure
> >>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been
> >>> enhancing
> >>>>> it
> >>>>>>>>>>>> to do
> >>>>>>>>>>>>>> so.
> >>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got
> >>> ESX(i)
> >>>>>>> in
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
> >>>>>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach
> >>> logic
> >>>>>>>>>>>> (in the
> >>>>>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to
> >>> have
> >>>>>>> it
> >>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the
> >>> volume
> >>>>>>> to
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize
> >>> it's
> >>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been
> >>> created)
> >>>>>>> and
> >>>>>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a
> >>> datastore
> >>>>>>>>>>>> (for
> >>>>>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
> >>>>>>> recently
> >>>>>>>>>>>> making
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM,
> John
> >>>>>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
> >>>>>>> driver
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
> >>>>>>> Tutkowski
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to
> >>> support
> >>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of
> >>> time
> >>>>>>>>>>>> for 4.2
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
> >>>>>>> XenServer.
> >>>>>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I
> >>> have
> >>>>> it
> >>>>>>>>>>>> working
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is
> all
> >>> in
> >>>>>>>>>>>> regards
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post
> 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM,
> >>> Kelcey
> >>>>>>>>>>>> Jamison
> >>>>>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting
> >>> KVM in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013
> >>> 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
> >>>>> disk_io_throttling
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature
> >>>>> first,
> >>>>>>>>>>>> then
> >>>>>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only,
> >>> then
> >>>>> it
> >>>>>>>>>>>> is a
> >>>>>>>>>>>>>> non
> >>>>>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM,
> Wei
> >>>>>>> ZHOU
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works
> >>> on
> >>>>>>>>>>>> billing
> >>>>>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They
> >>> can
> >>>>>>> get
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and
> volumes
> >>>>> for
> >>>>>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact
> limitation
> >>>>>>> value
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as
> well.
> >>> I
> >>>>>>> can
> >>>>>>>>>>>> work
> >>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second
> the
> >>>>>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about
> the
> >>>>> two
> >>>>>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei
> >>> ZHOU
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you.
> >>> I am
> >>>>>>>>>>>> trying
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should
> >>> depend
> >>>>> on
> >>>>>>>>>>>> IO
> >>>>>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
> >>>>>>>>>>>> disk_io_stat if
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics
> including
> >>>>>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model,
> customers
> >>>>> are
> >>>>>>>>>>>> billed
> >>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as,
> for
> >>>>>>> those
> >>>>>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/).
> I
> >>>>>>> would
> >>>>>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost
> >>>>> models.
> >>>>>>>>>>>> Could
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current
> >>> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime
> >>>>> change?
> >>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running
> virtual
> >>>>>>>>>>>> machine
> >>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support
> >>>>> changing
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
> >>>>>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface
> >>> level.
> >>>>>>> I
> >>>>>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
> >>>>>>>>>>>> algorithms
> >>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
> >>>>>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question.
> >>> Maybe it
> >>>>>>> is
> >>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike
> >>> needs to
> >>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts
> in a
> >>>>>>> later
> >>>>>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
> >>>>>>>>>>>> features. I
> >>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
> >>>>>>>>>>>> SolidFire, but
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the
> branches
> >>>>>>>>>>>> (possibly
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to
> >>> coordinate on
> >>>>>>>>>>>> this
> >>>>>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to
> understand
> >>>>> are
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in
> capturing
> >>>>> IOPS
> >>>>>>>>>>>>>> policies
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for
> >>> billing/usage
> >>>>>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
> >>>>>>>>>>>> interface model
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting
> >>> provisioned
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
> >>>>>>>>>>>> reconciled?
> >>>>>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and
> billed)
> >>>>> for
> >>>>>>>>>>>> more
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been
> >>> configured
> >>>>>>> to
> >>>>>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration
> >>> between a
> >>>>>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later
> modification
> >>> to
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM,
> >>> Mike
> >>>>>>>>>>>> Tutkowski
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals
> >>> with
> >>>>>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
> >>>>>>> controlling
> >>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35
> PM,
> >>>>>>> John
> >>>>>>>>>>>>>> Burwell
> >>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no
> features
> >>>>>>>>>>>> should be
> >>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved
> and
> >>> it
> >>>>>>> is
> >>>>>>>>>>>> ready
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete
> >>> read/write
> >>>>>>>>>>>> ops issue
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this
> >>> criteria
> >>>>>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >>>>>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://reviews.apache.org/r/11479/
> >>>>>>>>>>>> )?
> >>>>>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned
> >>> IOPS. I
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned
> >>> IOPS
> >>>>> in
> >>>>>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM,
> >>> Wei
> >>>>>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next
> >>> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander
> <
> >>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei
> >>> ZHOU
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
> >>>>>>> features
> >>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
> >>>>>>> technical
> >>>>>>>>>>>>>> reason
> >>>>>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I
> just
> >>>>>>> want
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps
> >>> asap
> >>>>> so
> >>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for
> >>> the
> >>>>>>> I/O
> >>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we
> create
> >>>>>>>>>>>> legacy from
> >>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den
> Hollander <
> >>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
> >>>>>>> Burwell
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must
> to
> >>>>>>>>>>>> discretely
> >>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be
> >>> fixed
> >>>>>>>>>>>> later
> >>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read
> and
> >>>>>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we
> decided
> >>> to
> >>>>>>>>>>>> break
> >>>>>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging
> it
> >>>>> into
> >>>>>>>>>>>> master
> >>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read /
> >>> Write
> >>>>>>> IOps
> >>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read
> and
> >>>>>>> Write
> >>>>>>>>>>>>>> instead
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
> >>>>>>> feature
> >>>>>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it
> >>> won't
> >>>>>>> be
> >>>>>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that
> >>>>> there
> >>>>>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM,
> >>> Wido
> >>>>>>>>>>>> den
> >>>>>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei
> >>> ZHOU
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the
> >>>>> beginning.
> >>>>>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
> >>>>>>> mainly
> >>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent
> >>> and
> >>>>>>>>>>>> received
> >>>>>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
> >>>>>>>>>>>> Networking and
> >>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more
> >>> expensive
> >>>>> in
> >>>>>>>>>>>> most
> >>>>>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
> >>>>>>>>>>>> copy-paste
> >>>>>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline
> of
> >>>>>>>>>>>> feature
> >>>>>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we
> >>> can't
> >>>>>>>>>>>> miss. But
> >>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure
> we
> >>>>>>> don't
> >>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
> >>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den
> Hollander <
> >>>>>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei
> >>> ZHOU
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
> >>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will
> merge
> >>>>>>> into
> >>>>>>>>>>>> master
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running
> >>> on
> >>>>>>> the
> >>>>>>>>>>>> same
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of
> the
> >>>>> rate
> >>>>>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk
> >>> operation,
> >>>>>>> it
> >>>>>>>>>>>> may
> >>>>>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the
> same
> >>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the
> >>>>> maximum
> >>>>>>>>>>>> rate
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see
> you
> >>>>> make
> >>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support
> >>> setting
> >>>>>>>>>>>> both a
> >>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a
> >>> lot of
> >>>>>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the
> >>> polling
> >>>>>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling
> >>> side
> >>>>>>>>>>>> you only
> >>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much
> >>> more
> >>>>>>>>>>>> expensive
> >>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where
> >>> that
> >>>>>>> an
> >>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM
> >>> at
> >>>>>>> this
> >>>>>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
> >>>>>>>>>>>> mechanism in
> >>>>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of
> >>>>> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't
> >>> support
> >>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum
> both
> >>> up
> >>>>>>>>>>>> and set
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
> >>>>>>> implemented
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of
> >>> VMs
> >>>>>>> (in
> >>>>>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate
> of
> >>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate
> (total
> >>>>> bps
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> >>>>>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> VM+Disk+IO+Throttling<https://
> >>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's
> >>> RAT
> >>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
> >>>>>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing
> (unit
> >>>>> and
> >>>>>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done
> >>> to
> >>>>>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and
> >>> IOPS
> >>>>>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
> >>>>>>>>>>>> destroy,
> >>>>>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations,
> >>> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can
> >>> try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> VM+Disk+IO+Throttling<https://
> >>>>>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O
> Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
> >>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
> >>> uses
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire
> >>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>> SolidFire
> >>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>> SolidFire
> >>>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>> cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Please see my responses in-line below.
Thanks,
-John
On Jun 7, 2013, at 1:50 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hey John,
>
> I still have a bit more testing I'd like to do before I build up a patch
> file, but this is the gist of what I've done:
>
> * During a volume-attach operation, after VolumeManagerImpl tells
> VolumeServiceImpl to have the driver create a volume, I have
> VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it managed.
> If it is managed, VolumeServiceImpl has the driver perform whatever
> activity is required. In my case, this includes sending a message to the
> host where the VM is running to have, say XenServer, add a storage
> repository (based on the IP address of the SAN, the IQN of the SAN volume,
> etc.) and a single VDI (the VDI consumes all of the space it can on the
> storage repository). After this, the normal attach-volume message is sent
> to the host by VolumeManagerImpl.
There should be **no** code from a storage driver to a hypervisor. I apologize for the repetition, but we simply can not have hypervisor specific code in the storage layer. The circular dependencies between the two layers are not sustainable in the long term. Either the VirtualManager or Xen hypervisor plugin needs to be refactored/modified to coordinate volume creation and then populating the SR. Ideally, we can generalize the process flow for attaching volumes such that the Xen hypervisor plugin would only implement callbacks to perform the attach action and create the structure and SR. To my mind, the SolidFire driver should only be allocating space and providing information about contents (e.g. space available, space consumed, streams to a URI, file handle for a URI, etc) and capabilities.
>
> * The reverse is performed for a detach-volume command.
>
> * Right now I simply "return true;" for isManaged() in my driver. Edison's
> default driver simply does a "return false;". We could add a new parameter
> to the createStoragePool API command, if we want, to remove the hard-coded
> return values in the drivers (although my driver will probably just ignore
> this parameter and always return true since it wouldn't make sense for it
> to ever return false). We'd need another column in the storage_pool table
> to store this value.
Yes, I think we should have a parameter added to the createStoragePool surfaced to the HTTP API that allows DataStores to be configured for management when their underlying drivers support it. To simplify things, this flag should only be mutable when the DataStore is created. It would be a bit crazy to take a DataStore from managed to unmanaged after creation.
>
> Sound like I'm in sync with what you were thinking?
>
>
>
>
> On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
>> wrote:
>
>> I agree, John. Just wanted to point out that I have a working GUI for you
>> to review (in that document), if you'd like to check it out.
>>
>> Thanks!
>>
>>
>> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> I would like the UIs of two features reviewed together to ensure
>>> consistency across the concepts of hypervisor throttled IOPs and
>>> storage device provisioned IOPs. I see the potential for confusion,
>>> and I think a side-by-side Ui review of these features will help
>>> minimize any potential confusion.
>>>
>>> As I mentioned, the term reconciliation issue will work itself if it
>>> is acceptable that a VM is only permitted utilize hypervisor throttled
>>> IOPs or storage provisioned IOPs.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>>> <mi...@solidfire.com> wrote:
>>>
>>>> Hi John,
>>>>
>>>> Yeah, when you get a chance, refer to the Google doc I sent to you the
>>>> other day to see how the GUI looks for provisioned storage IOPS.
>>>>
>>>> Several months ago, I put this topic out on the e-mail list and we
>>> decided
>>>> to place the Min, Max, and Burst IOPS in the Add Disk Offering dialog.
>>>> Other storage vendors are coming out with QoS, so they should be able to
>>>> leverage this GUI going forward (even if they, say, only use Max IOPS).
>>>> These fields are optional and can be ignored for storage that does not
>>>> support provisioned IOPS. Just like the Disk Size field, the admin can
>>>> choose to allow the end user to fill in Min, Max, and Burst IOPS.
>>>>
>>>> I'm OK if we do an either/or model (either Wei's feature or mine, as is
>>>> decided by the admin).
>>>>
>>>> I'm not sure what we can do about these two features expressing the
>>> speed
>>>> in different terms. I've never seen a SAN express the IOPS for QoS in
>>> any
>>>> way other than total IOPS (i.e. not broken in into read/write IOPS).
>>>>
>>>> Thanks
>>>>
>>>>
>>>> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>>>
>>>>> Wei,
>>>>>
>>>>> We have been down the rabbit hole a bit on the Storage/Hypervisor layer
>>>>> separation, but we still need to reconcile the behavior of hypervisor
>>>>> throttled I/O and storage provisioned IOPS. I see the following issues
>>>>> outstanding:
>>>>>
>>>>> 1. Hypervisor throttled IOPS are expressed as discrete read/write
>>> values
>>>>> where as storage provisioned IOPS are expressed as total IOPS.
>>>>> 2. How do we handle VMs with throttled IOPS attached to storage volumes
>>>>> with provisioned IOPS?
>>>>> 3. How should usage data be captured for throttled and provisioned IOPS
>>>>> that will permit providers to discriminate these guaranteed operations
>>> in
>>>>> the event they want to bill for it differently?
>>>>> 4. What is the user experience for throttled and provisioned IOPS that
>>>>> minimizes confusion of these concepts?
>>>>>
>>>>> My thinking is that a VM can have either utilize hypervisor throttled
>>> IOPS
>>>>> or storage provisioned IOPS. This policy would neatly solve items 1
>>> and 2.
>>>>> Since the two facilities would not be permitted to operate together,
>>> they
>>>>> do not need to be semantically compatible. I think item 3 can be
>>> resolved
>>>>> with an additional flag or two on the usage records. As for Item 4, I
>>> am
>>>>> not familiar with how these two enhancements are depicted in the user
>>>>> interface. I think we need to review the UI enhancements for both
>>>>> enhancements and ensure they are consistent.
>>>>>
>>>>> Do these solutions make sense?
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com> wrote:
>>>>>
>>>>>> John and Mike,
>>>>>>
>>>>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
>>>>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this
>>> week.
>>>>>> I will start to develop on iops/bps changes tomorrow, and ask for
>>> second
>>>>>> merge request after testing.
>>>>>>
>>>>>> -Wei
>>>>>>
>>>>>>
>>>>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>>>>>>
>>>>>>> I believe I understand where you're going with this, John, and have
>>> been
>>>>>>> re-working this section of code today.
>>>>>>>
>>>>>>> I should be able to run it by you tomorrow.
>>>>>>>
>>>>>>> Thanks for the comments,
>>>>>>> Mike
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> See my responses in-line below.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi John,
>>>>>>>>
>>>>>>>> Thanks for the response.
>>>>>>>>
>>>>>>>> "I am fine with the VolumeManager determining whether or not a
>>> Volume
>>>>> is
>>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
>>> isManaged
>>>>>>>> method), and asking the device driver to allocate resources for the
>>>>>>> volume
>>>>>>>> if it is managed."
>>>>>>>>
>>>>>>>> Are you thinking you'd like to see an isManaged() method added to
>>> the
>>>>>>>> PrimaryDataStoreDriver interface? If it returns true, then the
>>> storage
>>>>>>>> framework could call the manage() (or whatever name) method (which
>>>>> would
>>>>>>> be
>>>>>>>> new to the PrimaryDataStoreDriver interface, as well) and this would
>>>>> call
>>>>>>>> into a new method in the hypervisor code to create, say on
>>> XenServer,
>>>>> an
>>>>>>> SR?
>>>>>>>>
>>>>>>>>
>>>>>>>> I would like to see canBeManaged() : boolean on DataStoreDriver.
>>> Since
>>>>>>>> the notion of Volumes only pertains to primary storage, I would add
>>> a
>>>>>>>> allocateStorage and deallocateStorage (Storage is a straw man term
>>> --
>>>>>>>> something other than volume) methods to
>>>>> allocate/create/deallocate/delete
>>>>>>>> underlying storage. To my mind, managed is a mutable property of
>>>>>>> DataStore
>>>>>>>> which can be enabled if/when the underlying DataStoreDriver can be
>>>>>>> managed.
>>>>>>>> This approach allows operators to override manageability of devices.
>>>>>>>>
>>>>>>>> In terms of orchestration/process flow for SR, the Xen plugin would
>>> be
>>>>>>>> responsible for composing DataStore/Volume methods to create any
>>>>>>>> directories or files necessary for the SR. There should be no
>>>>>>> dependencies
>>>>>>>> from the Storage to the Hypervisor layer. As I said earlier, such
>>>>>>> circular
>>>>>>>> dependencies will lead to a tangled, unmaintainable mess.
>>>>>>>>
>>>>>>>>
>>>>>>>> Just want to make sure I'm on the same page with you.
>>>>>>>>
>>>>>>>> Thanks again, John
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> Fundamentally, we can't end up with a Storage layer that supports n
>>>>>>>>> devices types with each specific behaviors of m hypervisors. Such
>>> a
>>>>>>>>> scenario will create an unmaintainable and untestable beast.
>>>>>>> Therefore, my
>>>>>>>>> thoughts and recommendations are driven to evolve the Storage layer
>>>>>>> towards
>>>>>>>>> this separation.
>>>>>>>>>
>>>>>>>>> I am fine with the VolumeManager determining whether or not a
>>> Volume
>>>>> is
>>>>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
>>>>> isManaged
>>>>>>>>> method), and asking the device driver to allocate resources for the
>>>>>>> volume
>>>>>>>>> if it is managed. Furthermore, the device driver needs to indicate
>>>>>>> whether
>>>>>>>>> or not it supports management operations. Finally, I think we
>>> need to
>>>>>>>>> provide the ability for an administrator to elect to have something
>>>>>>> that is
>>>>>>>>> manageable be unmanaged (i.e. the driver is capable managing the
>>>>> device,
>>>>>>>>> but the administrator has elected to leave it unmanaged).
>>>>>>>>>
>>>>>>>>> Creation of a structure on the volume should be done in the Xen
>>>>>>>>> hypervisor module using methods exposed by the Storage layer to
>>>>> perform
>>>>>>>>> low-level operations (e.g. make directories, create a file, etc).
>>>>> This
>>>>>>>>> structure is specific to the operation of the Xen hypervisor, as
>>> such,
>>>>>>>>> should be confined to its implementation. From my perspective,
>>>>> nothing
>>>>>>> in
>>>>>>>>> the Storage layer should be concerned with content. From its
>>>>>>> perspective,
>>>>>>>>> structure and data are opaque. It provides the means to query the
>>>>> data
>>>>>>> to
>>>>>>>>> support the interpretation of the content by higher-level layers
>>> (e.g.
>>>>>>>>> Hypervisors). To my mind, attach should be a composition of
>>>>> operations
>>>>>>>>> from the Storage layer that varies based on the Volume storage
>>>>> protocol
>>>>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi John,
>>>>>>>>>
>>>>>>>>> Alternatively to the way the attach logic is implemented in my
>>> patch,
>>>>> we
>>>>>>>>> could do the following:
>>>>>>>>>
>>>>>>>>> Leave the attach logic in the agent code alone. In
>>> VolumeManagerImpl
>>>>> we
>>>>>>>>> create an AttachVolumeCommand and send it to the hypervisor. Before
>>>>> this
>>>>>>>>> command is sent, we could check to see if we're dealing with
>>> Dynamic
>>>>> (or
>>>>>>>>> whatever we want to call it) storage and - if so - send a "Create
>>> SR"
>>>>>>>>> command to the hypervisor. If this returns OK, we would then
>>> proceed
>>>>> to
>>>>>>> the
>>>>>>>>> AttachVolumeCommand, as usual.
>>>>>>>>>
>>>>>>>>> This way the attach logic remains the same and we just add another
>>>>>>>>> command to the agent code that is called for this particular type
>>> of
>>>>>>>>> storage.
>>>>>>>>>
>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hey John,
>>>>>>>>>>
>>>>>>>>>> I created a document for a customer today that outlines how the
>>>>> plug-in
>>>>>>>>>> works from a user standpoint. This will probably be of use to
>>> you, as
>>>>>>> well,
>>>>>>>>>> as you perform the code review.
>>>>>>>>>>
>>>>>>>>>> I have shared this document with you (you should have received
>>> that
>>>>>>>>>> information in a separate e-mail).
>>>>>>>>>>
>>>>>>>>>> Talk to you later!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>>>>
>>>>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jburwell@basho.com
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> I am never at a loss for an opinion. I some thoughts, but want
>>> to
>>>>>>>>>>>> confirm assumptions and ideas against the solidfire,
>>>>>>> disk_io_throttle,
>>>>>>>>>>>> and
>>>>>>>>>>>> object_store branches. I hope to collect them in a coherent
>>> form
>>>>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the notion of
>>> a
>>>>>>>>>>>> managed
>>>>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>>>>
>>>>>>>>>>>>> Technically, the SolidFire plug-in just introduces the notion
>>> of
>>>>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The storage framework that leverages the plug-in was
>>> incomplete,
>>>>> so
>>>>>>>>>>>> I had
>>>>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I appreciate all the time you've been spending on this. :) Do
>>> you
>>>>>>>>>>>> have a
>>>>>>>>>>>>> recommendation as to how we should accomplish what you're
>>> looking
>>>>>>>>>>>> for?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>>> jburwell@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the notion of
>>> a
>>>>>>>>>>>> managed
>>>>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I want to see a separation of the management capabilities
>>> (i.e.
>>>>>>> can
>>>>>>>>>>>> a
>>>>>>>>>>>>>> device be managed/does an operator want it managed by
>>> CloudStack)
>>>>>>>>>>>> from the
>>>>>>>>>>>>>> storage protocol. Ideally, we should end up with a semantic
>>> that
>>>>>>>>>>>> will
>>>>>>>>>>>>>> allow any type of storage device to be managed. I also want
>>> to
>>>>>>> make
>>>>>>>>>>>>>> progress on decoupling the storage types from the hypervisor
>>>>>>>>>>>> definitions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>>>>> jburwell@basho.com
>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Could you please answer the following questions for me with
>>>>>>>>>>>> regards to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>>>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
>>>>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What is the cardinality between SAN Volumes and CloudStack
>>>>>>>>>>>> Volumes?
>>>>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Are the LUN(s) created by the management server or
>>> externally
>>>>> by
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume (same as a
>>>>> SAN
>>>>>>>>>>>> LUN) is
>>>>>>>>>>>>>>> created by the management server (via the plug-in) the first
>>>>> time
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but still
>>> want
>>>>> to
>>>>>>>>>>>> use a
>>>>>>>>>>>>>>> SolidFire volume (LUN), you can do this already today (prior
>>> to
>>>>>>>>>>>> 4.2). The
>>>>>>>>>>>>>>> admin manually creates the SAN volume and - in this case -
>>>>>>>>>>>> multiple VMs
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> data disks can share this SAN volume. While you can do this
>>>>>>> today,
>>>>>>>>>>>> it is
>>>>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Are the SAN volumes by the management server or externally
>>> by
>>>>>>> the
>>>>>>>>>>>>>> operator?
>>>>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes are
>>>>>>> completely
>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>> by the management server (via the plug-in). There is no admin
>>>>>>>>>>>>>> interaction.
>>>>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume and a
>>>>>>> CloudStack
>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>> which is necessary for any storage vendor that supports true
>>>>> QoS.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I would like to clarify how these pieces are related and
>>>>>>> expected
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs with
>>>>>>>>>>>> provisioned
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In the storage QoS world, we need to map a single SAN
>>> volume
>>>>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>>>>> single CloudStack volume. We cannot have multiple
>>> CloudStack
>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If the user wants to have a single SAN volume house more
>>> than
>>>>>>> one
>>>>>>>>>>>>>>>>> CloudStack volume, then can do that today without any of my
>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume for
>>>>> CloudStack's
>>>>>>>>>>>> use
>>>>>>>>>>>>>> onto
>>>>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> You are coming to the part which concerns me -- concepts
>>>>> from
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only used
>>>>> (as
>>>>>>>>>>>> far as
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not PreSetup an
>>> SR).
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from VMware, it
>>> uses
>>>>>>> the
>>>>>>>>>>>> vmfs
>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM, it uses
>>> the
>>>>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here was to
>>> make a
>>>>>>>>>>>> new type
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison added
>>> into
>>>>>>> the
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> framework. Maybe it should be more specificy, though:
>>>>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or care that
>>> it
>>>>>>> is
>>>>>>>>>>>> being
>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer questions
>>> about
>>>>> it
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I see...so your concern here is that the SolidFire
>>> plug-in
>>>>>>>>>>>> needs to
>>>>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic
>>>>> knows
>>>>>>>>>>>> to
>>>>>>>>>>>>>> treat
>>>>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint and just
>>>>>>>>>>>> calling it
>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>> whatever. We would just need a way for the hypervisor
>>>>>>> attach
>>>>>>>>>>>> logic
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> detect this new requirement and perform the necessary
>>>>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to manage a
>>>>>>> device,
>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
>>>>>>> device
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>> many services, and the operator has chosen to
>>> dedicate
>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via
>>> the
>>>>>>>>>>>> plug-in
>>>>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would work unless
>>> the
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> My thinking is that, independent of CloudStack, an
>>>>>>> operator
>>>>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through
>>> a
>>>>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to CloudStack,
>>> but
>>>>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my changes, we
>>> would
>>>>>>> be
>>>>>>>>>>>>>> passing
>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the necessary
>>> hypervisor
>>>>>>>>>>>> support
>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the storage
>>>>>>> framework
>>>>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from the
>>> hypervisor
>>>>>>>>>>>> side and
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to create the
>>> SR
>>>>>>>>>>>> before the
>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin should have a
>>>>>>> rich
>>>>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>>>>> interface to storage to determine available for volume
>>>>>>>>>>>> storage.
>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>>>>> this interface would allow the interrogation of the
>>>>> device
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself shouldn't
>>> know
>>>>>>>>>>>> or care
>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>>>>>>>>>>>> questions
>>>>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver (which
>>> is a
>>>>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
>>>>>>>>>>>> Storage shows
>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that references
>>> the
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack volume. This
>>> leads
>>>>>>> to
>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack volume to a
>>> VM
>>>>>>>>>>>> (attach
>>>>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the plug-in
>>> to
>>>>>>>>>>>> create a
>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN). The
>>> plug-in
>>>>>>>>>>>> also
>>>>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data (like the
>>> IQN of
>>>>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
>>>>>>>>>>>> volume is in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the volume
>>> will be
>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the plug-in
>>> code
>>>>>>>>>>>> will not
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a
>>> hypervisor
>>>>>>> data
>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on terminology,
>>> volume,
>>>>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical allocation on
>>> the
>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN
>>> volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>>>>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well make sense, but
>>> I'm
>>>>>>>>>>>> having a
>>>>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on this
>>> conversation
>>>>>>>>>>>> as he
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I took.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two distinct
>>> concepts
>>>>>>> --
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In both
>>> cases, we
>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the volume to be
>>>>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure
>>> that
>>>>> we
>>>>>>>>>>>> are in
>>>>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these definitions,
>>> refers to
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we
>>>>>>> must
>>>>>>>>>>>> be
>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to move bits
>>> from
>>>>>>>>>>>> one place
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a
>>> storage
>>>>>>>>>>>> device
>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
>>>>>>>>>>>> unmanaged
>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
>>>>>>> managed
>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable of
>>> allocating
>>>>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able
>>> to
>>>>>>>>>>>> manage a
>>>>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
>>>>>>> CloudStack
>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the operator has
>>> chosen
>>>>>>> to
>>>>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning
>>>>> make
>>>>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it
>>> seems
>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from
>>> StoragePoolType,
>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver that
>>>>> indicates
>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish a
>>> abstraction
>>>>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is
>>> associated
>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
>>>>>>> declare
>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a device managed
>>> by
>>>>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed by the driver to
>>>>>>> automate
>>>>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can
>>> sit
>>>>>>>>>>>> down and
>>>>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the current iSCSI
>>>>>>> type
>>>>>>>>>>>> and
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
>>>>>>>>>>>> volume must
>>>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
>>>>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
>>>>>>>>>>>> Primary
>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on
>>> its
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
>>>>>>>>>>>> framework). When
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is
>>> done,
>>>>>>>>>>>> the
>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called. No
>>> hypervisor
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was just
>>> created.
>>>>>>> The
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should probably have
>>>>>>>>>>>> defined them
>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated
>>> (ex.
>>>>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data
>>> structure
>>>>>>> is
>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
>>>>>>> hypervisor
>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
>>>>>>> placed
>>>>>>>>>>>> on
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
>>>>>>> attach
>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already in place
>>> and
>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
>>>>> pre-allocated.
>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not
>>> a
>>>>>>>>>>>> volume on
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure
>>> must be
>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these types of
>>> volumes
>>>>>>>>>>>> have not
>>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an
>>>>> admin.
>>>>>>>>>>>> Once
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this
>>> volume,
>>>>>>> it
>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>>>>>>>>>>>> difference.
>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the definition of
>>> a
>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely
>>>>> used
>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is
>>> directly
>>>>>>>>>>>> based on
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to
>>> the
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI
>>> target as
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
>>>>>>>>>>>> XenServer.
>>>>>>>>>>>>>> For
>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the
>>> hypervisor
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell
>>> <
>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the
>>>>> difference
>>>>>>>>>>>> between
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
>>>>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a
>>>>> clusterfs
>>>>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage
>>> Pool
>>>>>>>>>>>> is set
>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide
>>> storage
>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike
>>> Tutkowski
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage
>>>>> company
>>>>>>>>>>>> wants
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>>>>>>>>>>>> feature. It
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would
>>> need
>>>>>>> only
>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary
>>> changes to
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware
>>> of
>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of
>>> this
>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's
>>> storage
>>>>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that
>>>>> supported
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
>>>>>>>>>>>> visited with
>>>>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
>>>>>>>>>>>> complete. We
>>>>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
>>>>>>> volumes
>>>>>>>>>>>> unless
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of
>>>>> storage
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John
>>> Burwell
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any
>>>>> operation
>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I
>>> expect
>>>>>>>>>>>> to see
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>>>>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>>>>>>>>>>>> structures
>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are
>>> going
>>>>>>> to
>>>>>>>>>>>> end
>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
>>>>>>> increasingly
>>>>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the
>>> Xen
>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more
>>> detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
>>>>>>> method
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
>>>>>>>>>>>> system,
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use
>>> of
>>>>>>> the
>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore
>>> on
>>>>>>>>>>>> ESX(i)).
>>>>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
>>>>>>> storage
>>>>>>>>>>>> need
>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
>>>>>>> relationship
>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk
>>> (necessary
>>>>>>> for
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is
>>> called
>>>>>>> to
>>>>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework
>>> did
>>>>>>>>>>>> not
>>>>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
>>>>>>>>>>>> structure
>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been
>>> enhancing
>>>>> it
>>>>>>>>>>>> to do
>>>>>>>>>>>>>> so.
>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got
>>> ESX(i)
>>>>>>> in
>>>>>>>>>>>> my
>>>>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
>>>>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach
>>> logic
>>>>>>>>>>>> (in the
>>>>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to
>>> have
>>>>>>> it
>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the
>>> volume
>>>>>>> to
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize
>>> it's
>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been
>>> created)
>>>>>>> and
>>>>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a
>>> datastore
>>>>>>>>>>>> (for
>>>>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
>>>>>>> recently
>>>>>>>>>>>> making
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
>>>>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
>>>>>>> driver
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
>>>>>>> Tutkowski
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to
>>> support
>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of
>>> time
>>>>>>>>>>>> for 4.2
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
>>>>>>> XenServer.
>>>>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I
>>> have
>>>>> it
>>>>>>>>>>>> working
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all
>>> in
>>>>>>>>>>>> regards
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM,
>>> Kelcey
>>>>>>>>>>>> Jamison
>>>>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting
>>> KVM in
>>>>>>>>>>>> the
>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013
>>> 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
>>>>> disk_io_throttling
>>>>>>>>>>>> to
>>>>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature
>>>>> first,
>>>>>>>>>>>> then
>>>>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only,
>>> then
>>>>> it
>>>>>>>>>>>> is a
>>>>>>>>>>>>>> non
>>>>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei
>>>>>>> ZHOU
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works
>>> on
>>>>>>>>>>>> billing
>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They
>>> can
>>>>>>> get
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes
>>>>> for
>>>>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation
>>>>>>> value
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well.
>>> I
>>>>>>> can
>>>>>>>>>>>> work
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the
>>>>> two
>>>>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei
>>> ZHOU
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you.
>>> I am
>>>>>>>>>>>> trying
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should
>>> depend
>>>>> on
>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
>>>>>>>>>>>> disk_io_stat if
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>>>>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers
>>>>> are
>>>>>>>>>>>> billed
>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for
>>>>>>> those
>>>>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I
>>>>>>> would
>>>>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost
>>>>> models.
>>>>>>>>>>>> Could
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current
>>> patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime
>>>>> change?
>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
>>>>>>>>>>>> machine
>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support
>>>>> changing
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
>>>>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface
>>> level.
>>>>>>> I
>>>>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
>>>>>>>>>>>> algorithms
>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
>>>>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question.
>>> Maybe it
>>>>>>> is
>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike
>>> needs to
>>>>>>>>>>>> consider
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a
>>>>>>> later
>>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
>>>>>>>>>>>> features. I
>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
>>>>>>>>>>>> SolidFire, but
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>>>>>>>>>>>> (possibly
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to
>>> coordinate on
>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand
>>>>> are
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing
>>>>> IOPS
>>>>>>>>>>>>>> policies
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for
>>> billing/usage
>>>>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
>>>>>>>>>>>> interface model
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting
>>> provisioned
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>>>>>>>>>>>> reconciled?
>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed)
>>>>> for
>>>>>>>>>>>> more
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been
>>> configured
>>>>>>> to
>>>>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration
>>> between a
>>>>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification
>>> to
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM,
>>> Mike
>>>>>>>>>>>> Tutkowski
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals
>>> with
>>>>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
>>>>>>> controlling
>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM,
>>>>>>> John
>>>>>>>>>>>>>> Burwell
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
>>>>>>>>>>>> should be
>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and
>>> it
>>>>>>> is
>>>>>>>>>>>> ready
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete
>>> read/write
>>>>>>>>>>>> ops issue
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this
>>> criteria
>>>>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>>>>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://reviews.apache.org/r/11479/
>>>>>>>>>>>> )?
>>>>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned
>>> IOPS. I
>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned
>>> IOPS
>>>>> in
>>>>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM,
>>> Wei
>>>>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next
>>> week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei
>>> ZHOU
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
>>>>>>> features
>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
>>>>>>> technical
>>>>>>>>>>>>>> reason
>>>>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just
>>>>>>> want
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps
>>> asap
>>>>> so
>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for
>>> the
>>>>>>> I/O
>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
>>>>>>>>>>>> legacy from
>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
>>>>>>> Burwell
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>>>>>>>>>>>> discretely
>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be
>>> fixed
>>>>>>>>>>>> later
>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
>>>>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided
>>> to
>>>>>>>>>>>> break
>>>>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it
>>>>> into
>>>>>>>>>>>> master
>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read /
>>> Write
>>>>>>> IOps
>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and
>>>>>>> Write
>>>>>>>>>>>>>> instead
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
>>>>>>> feature
>>>>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it
>>> won't
>>>>>>> be
>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that
>>>>> there
>>>>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM,
>>> Wido
>>>>>>>>>>>> den
>>>>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei
>>> ZHOU
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the
>>>>> beginning.
>>>>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
>>>>>>> mainly
>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent
>>> and
>>>>>>>>>>>> received
>>>>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
>>>>>>>>>>>> Networking and
>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more
>>> expensive
>>>>> in
>>>>>>>>>>>> most
>>>>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
>>>>>>>>>>>> copy-paste
>>>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we
>>> can't
>>>>>>>>>>>> miss. But
>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we
>>>>>>> don't
>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei
>>> ZHOU
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge
>>>>>>> into
>>>>>>>>>>>> master
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running
>>> on
>>>>>>> the
>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the
>>>>> rate
>>>>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk
>>> operation,
>>>>>>> it
>>>>>>>>>>>> may
>>>>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the
>>>>> maximum
>>>>>>>>>>>> rate
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you
>>>>> make
>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support
>>> setting
>>>>>>>>>>>> both a
>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a
>>> lot of
>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the
>>> polling
>>>>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling
>>> side
>>>>>>>>>>>> you only
>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much
>>> more
>>>>>>>>>>>> expensive
>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where
>>> that
>>>>>>> an
>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM
>>> at
>>>>>>> this
>>>>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
>>>>>>>>>>>> mechanism in
>>>>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of
>>>>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't
>>> support
>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both
>>> up
>>>>>>>>>>>> and set
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
>>>>>>> implemented
>>>>>>>>>>>> it
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of
>>> VMs
>>>>>>> (in
>>>>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of
>>>>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total
>>>>> bps
>>>>>>>>>>>> and
>>>>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's
>>> RAT
>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>>>>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit
>>>>> and
>>>>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done
>>> to
>>>>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and
>>> IOPS
>>>>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
>>>>>>>>>>>> destroy,
>>>>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations,
>>> including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can
>>> try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>>> refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>>> uses
>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>> SolidFire
>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>> SolidFire
>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>> SolidFire
>>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hey John,
I still have a bit more testing I'd like to do before I build up a patch
file, but this is the gist of what I've done:
* During a volume-attach operation, after VolumeManagerImpl tells
VolumeServiceImpl to have the driver create a volume, I have
VolumeManagerImpl tell VolumeServiceImpl to ask the driver if it managed.
If it is managed, VolumeServiceImpl has the driver perform whatever
activity is required. In my case, this includes sending a message to the
host where the VM is running to have, say XenServer, add a storage
repository (based on the IP address of the SAN, the IQN of the SAN volume,
etc.) and a single VDI (the VDI consumes all of the space it can on the
storage repository). After this, the normal attach-volume message is sent
to the host by VolumeManagerImpl.
* The reverse is performed for a detach-volume command.
* Right now I simply "return true;" for isManaged() in my driver. Edison's
default driver simply does a "return false;". We could add a new parameter
to the createStoragePool API command, if we want, to remove the hard-coded
return values in the drivers (although my driver will probably just ignore
this parameter and always return true since it wouldn't make sense for it
to ever return false). We'd need another column in the storage_pool table
to store this value.
Sound like I'm in sync with what you were thinking?
On Thu, Jun 6, 2013 at 9:34 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> I agree, John. Just wanted to point out that I have a working GUI for you
> to review (in that document), if you'd like to check it out.
>
> Thanks!
>
>
> On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I would like the UIs of two features reviewed together to ensure
>> consistency across the concepts of hypervisor throttled IOPs and
>> storage device provisioned IOPs. I see the potential for confusion,
>> and I think a side-by-side Ui review of these features will help
>> minimize any potential confusion.
>>
>> As I mentioned, the term reconciliation issue will work itself if it
>> is acceptable that a VM is only permitted utilize hypervisor throttled
>> IOPs or storage provisioned IOPs.
>>
>> Thanks,
>> -John
>>
>> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
>> <mi...@solidfire.com> wrote:
>>
>> > Hi John,
>> >
>> > Yeah, when you get a chance, refer to the Google doc I sent to you the
>> > other day to see how the GUI looks for provisioned storage IOPS.
>> >
>> > Several months ago, I put this topic out on the e-mail list and we
>> decided
>> > to place the Min, Max, and Burst IOPS in the Add Disk Offering dialog.
>> > Other storage vendors are coming out with QoS, so they should be able to
>> > leverage this GUI going forward (even if they, say, only use Max IOPS).
>> > These fields are optional and can be ignored for storage that does not
>> > support provisioned IOPS. Just like the Disk Size field, the admin can
>> > choose to allow the end user to fill in Min, Max, and Burst IOPS.
>> >
>> > I'm OK if we do an either/or model (either Wei's feature or mine, as is
>> > decided by the admin).
>> >
>> > I'm not sure what we can do about these two features expressing the
>> speed
>> > in different terms. I've never seen a SAN express the IOPS for QoS in
>> any
>> > way other than total IOPS (i.e. not broken in into read/write IOPS).
>> >
>> > Thanks
>> >
>> >
>> > On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Wei,
>> >>
>> >> We have been down the rabbit hole a bit on the Storage/Hypervisor layer
>> >> separation, but we still need to reconcile the behavior of hypervisor
>> >> throttled I/O and storage provisioned IOPS. I see the following issues
>> >> outstanding:
>> >>
>> >> 1. Hypervisor throttled IOPS are expressed as discrete read/write
>> values
>> >> where as storage provisioned IOPS are expressed as total IOPS.
>> >> 2. How do we handle VMs with throttled IOPS attached to storage volumes
>> >> with provisioned IOPS?
>> >> 3. How should usage data be captured for throttled and provisioned IOPS
>> >> that will permit providers to discriminate these guaranteed operations
>> in
>> >> the event they want to bill for it differently?
>> >> 4. What is the user experience for throttled and provisioned IOPS that
>> >> minimizes confusion of these concepts?
>> >>
>> >> My thinking is that a VM can have either utilize hypervisor throttled
>> IOPS
>> >> or storage provisioned IOPS. This policy would neatly solve items 1
>> and 2.
>> >> Since the two facilities would not be permitted to operate together,
>> they
>> >> do not need to be semantically compatible. I think item 3 can be
>> resolved
>> >> with an additional flag or two on the usage records. As for Item 4, I
>> am
>> >> not familiar with how these two enhancements are depicted in the user
>> >> interface. I think we need to review the UI enhancements for both
>> >> enhancements and ensure they are consistent.
>> >>
>> >> Do these solutions make sense?
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com> wrote:
>> >>
>> >>> John and Mike,
>> >>>
>> >>> I was busy working on other issues (CLOUDSTACK-2780/2729,
>> >>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this
>> week.
>> >>> I will start to develop on iops/bps changes tomorrow, and ask for
>> second
>> >>> merge request after testing.
>> >>>
>> >>> -Wei
>> >>>
>> >>>
>> >>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>> >>>
>> >>>> I believe I understand where you're going with this, John, and have
>> been
>> >>>> re-working this section of code today.
>> >>>>
>> >>>> I should be able to run it by you tomorrow.
>> >>>>
>> >>>> Thanks for the comments,
>> >>>> Mike
>> >>>>
>> >>>>
>> >>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>>
>> >>>>> Mike,
>> >>>>>
>> >>>>> See my responses in-line below.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> -John
>> >>>>>
>> >>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>> Hi John,
>> >>>>>
>> >>>>> Thanks for the response.
>> >>>>>
>> >>>>> "I am fine with the VolumeManager determining whether or not a
>> Volume
>> >> is
>> >>>>> managed (i.e. not based on the StoragePoolType, but an actual
>> isManaged
>> >>>>> method), and asking the device driver to allocate resources for the
>> >>>> volume
>> >>>>> if it is managed."
>> >>>>>
>> >>>>> Are you thinking you'd like to see an isManaged() method added to
>> the
>> >>>>> PrimaryDataStoreDriver interface? If it returns true, then the
>> storage
>> >>>>> framework could call the manage() (or whatever name) method (which
>> >> would
>> >>>> be
>> >>>>> new to the PrimaryDataStoreDriver interface, as well) and this would
>> >> call
>> >>>>> into a new method in the hypervisor code to create, say on
>> XenServer,
>> >> an
>> >>>> SR?
>> >>>>>
>> >>>>>
>> >>>>> I would like to see canBeManaged() : boolean on DataStoreDriver.
>> Since
>> >>>>> the notion of Volumes only pertains to primary storage, I would add
>> a
>> >>>>> allocateStorage and deallocateStorage (Storage is a straw man term
>> --
>> >>>>> something other than volume) methods to
>> >> allocate/create/deallocate/delete
>> >>>>> underlying storage. To my mind, managed is a mutable property of
>> >>>> DataStore
>> >>>>> which can be enabled if/when the underlying DataStoreDriver can be
>> >>>> managed.
>> >>>>> This approach allows operators to override manageability of devices.
>> >>>>>
>> >>>>> In terms of orchestration/process flow for SR, the Xen plugin would
>> be
>> >>>>> responsible for composing DataStore/Volume methods to create any
>> >>>>> directories or files necessary for the SR. There should be no
>> >>>> dependencies
>> >>>>> from the Storage to the Hypervisor layer. As I said earlier, such
>> >>>> circular
>> >>>>> dependencies will lead to a tangled, unmaintainable mess.
>> >>>>>
>> >>>>>
>> >>>>> Just want to make sure I'm on the same page with you.
>> >>>>>
>> >>>>> Thanks again, John
>> >>>>>
>> >>>>>
>> >>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> Fundamentally, we can't end up with a Storage layer that supports n
>> >>>>>> devices types with each specific behaviors of m hypervisors. Such
>> a
>> >>>>>> scenario will create an unmaintainable and untestable beast.
>> >>>> Therefore, my
>> >>>>>> thoughts and recommendations are driven to evolve the Storage layer
>> >>>> towards
>> >>>>>> this separation.
>> >>>>>>
>> >>>>>> I am fine with the VolumeManager determining whether or not a
>> Volume
>> >> is
>> >>>>>> managed (i.e. not based on the StoragePoolType, but an actual
>> >> isManaged
>> >>>>>> method), and asking the device driver to allocate resources for the
>> >>>> volume
>> >>>>>> if it is managed. Furthermore, the device driver needs to indicate
>> >>>> whether
>> >>>>>> or not it supports management operations. Finally, I think we
>> need to
>> >>>>>> provide the ability for an administrator to elect to have something
>> >>>> that is
>> >>>>>> manageable be unmanaged (i.e. the driver is capable managing the
>> >> device,
>> >>>>>> but the administrator has elected to leave it unmanaged).
>> >>>>>>
>> >>>>>> Creation of a structure on the volume should be done in the Xen
>> >>>>>> hypervisor module using methods exposed by the Storage layer to
>> >> perform
>> >>>>>> low-level operations (e.g. make directories, create a file, etc).
>> >> This
>> >>>>>> structure is specific to the operation of the Xen hypervisor, as
>> such,
>> >>>>>> should be confined to its implementation. From my perspective,
>> >> nothing
>> >>>> in
>> >>>>>> the Storage layer should be concerned with content. From its
>> >>>> perspective,
>> >>>>>> structure and data are opaque. It provides the means to query the
>> >> data
>> >>>> to
>> >>>>>> support the interpretation of the content by higher-level layers
>> (e.g.
>> >>>>>> Hypervisors). To my mind, attach should be a composition of
>> >> operations
>> >>>>>> from the Storage layer that varies based on the Volume storage
>> >> protocol
>> >>>>>> (iSCSI, local file system, NFS, RBD, etc).
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>> Hi John,
>> >>>>>>
>> >>>>>> Alternatively to the way the attach logic is implemented in my
>> patch,
>> >> we
>> >>>>>> could do the following:
>> >>>>>>
>> >>>>>> Leave the attach logic in the agent code alone. In
>> VolumeManagerImpl
>> >> we
>> >>>>>> create an AttachVolumeCommand and send it to the hypervisor. Before
>> >> this
>> >>>>>> command is sent, we could check to see if we're dealing with
>> Dynamic
>> >> (or
>> >>>>>> whatever we want to call it) storage and - if so - send a "Create
>> SR"
>> >>>>>> command to the hypervisor. If this returns OK, we would then
>> proceed
>> >> to
>> >>>> the
>> >>>>>> AttachVolumeCommand, as usual.
>> >>>>>>
>> >>>>>> This way the attach logic remains the same and we just add another
>> >>>>>> command to the agent code that is called for this particular type
>> of
>> >>>>>> storage.
>> >>>>>>
>> >>>>>> What do you think?
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>
>> >>>>>>> Hey John,
>> >>>>>>>
>> >>>>>>> I created a document for a customer today that outlines how the
>> >> plug-in
>> >>>>>>> works from a user standpoint. This will probably be of use to
>> you, as
>> >>>> well,
>> >>>>>>> as you perform the code review.
>> >>>>>>>
>> >>>>>>> I have shared this document with you (you should have received
>> that
>> >>>>>>> information in a separate e-mail).
>> >>>>>>>
>> >>>>>>> Talk to you later!
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>
>> >>>>>>>> Oh, OK, that sounds really good, John.
>> >>>>>>>>
>> >>>>>>>> Thanks and talk to you tomorrow! :)
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jburwell@basho.com
>> >>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Mike,
>> >>>>>>>>>
>> >>>>>>>>> I am never at a loss for an opinion. I some thoughts, but want
>> to
>> >>>>>>>>> confirm assumptions and ideas against the solidfire,
>> >>>> disk_io_throttle,
>> >>>>>>>>> and
>> >>>>>>>>> object_store branches. I hope to collect them in a coherent
>> form
>> >>>>>>>>> tomorrow (5 June 2013).
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> -John
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> "So, in essence, the SolidFire plugin introduces the notion of
>> a
>> >>>>>>>>> managed
>> >>>>>>>>>> iSCSI device and provisioned IOPS."
>> >>>>>>>>>>
>> >>>>>>>>>> Technically, the SolidFire plug-in just introduces the notion
>> of
>> >>>>>>>>>> provisioned storage IOPS.
>> >>>>>>>>>>
>> >>>>>>>>>> The storage framework that leverages the plug-in was
>> incomplete,
>> >> so
>> >>>>>>>>> I had
>> >>>>>>>>>> to try to add in the notion of a managed iSCSI device.
>> >>>>>>>>>>
>> >>>>>>>>>> I appreciate all the time you've been spending on this. :) Do
>> you
>> >>>>>>>>> have a
>> >>>>>>>>>> recommendation as to how we should accomplish what you're
>> looking
>> >>>>>>>>> for?
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks!
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
>> jburwell@basho.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Mike,
>> >>>>>>>>>>>
>> >>>>>>>>>>> So, in essence, the SolidFire plugin introduces the notion of
>> a
>> >>>>>>>>> managed
>> >>>>>>>>>>> iSCSI device and provisioned IOPS.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I want to see a separation of the management capabilities
>> (i.e.
>> >>>> can
>> >>>>>>>>> a
>> >>>>>>>>>>> device be managed/does an operator want it managed by
>> CloudStack)
>> >>>>>>>>> from the
>> >>>>>>>>>>> storage protocol. Ideally, we should end up with a semantic
>> that
>> >>>>>>>>> will
>> >>>>>>>>>>> allow any type of storage device to be managed. I also want
>> to
>> >>>> make
>> >>>>>>>>>>> progress on decoupling the storage types from the hypervisor
>> >>>>>>>>> definitions.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> -John
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>> >>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> No problem. Answers are below in red.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>> >> jburwell@basho.com
>> >>>>>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Could you please answer the following questions for me with
>> >>>>>>>>> regards to
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>> operation of the SolidFire plugin:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>> >>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
>> >>>>>>>>>>>> 1 SAN volume : 1 LUN
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> What is the cardinality between SAN Volumes and CloudStack
>> >>>>>>>>> Volumes?
>> >>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Are the LUN(s) created by the management server or
>> externally
>> >> by
>> >>>>>>>>> the
>> >>>>>>>>>>>>> operator?
>> >>>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume (same as a
>> >> SAN
>> >>>>>>>>> LUN) is
>> >>>>>>>>>>>> created by the management server (via the plug-in) the first
>> >> time
>> >>>>>>>>> the
>> >>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but still
>> want
>> >> to
>> >>>>>>>>> use a
>> >>>>>>>>>>>> SolidFire volume (LUN), you can do this already today (prior
>> to
>> >>>>>>>>> 4.2). The
>> >>>>>>>>>>>> admin manually creates the SAN volume and - in this case -
>> >>>>>>>>> multiple VMs
>> >>>>>>>>>>> and
>> >>>>>>>>>>>> data disks can share this SAN volume. While you can do this
>> >>>> today,
>> >>>>>>>>> it is
>> >>>>>>>>>>>> not useful if you want to enforce storage QoS.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Are the SAN volumes by the management server or externally
>> by
>> >>>> the
>> >>>>>>>>>>> operator?
>> >>>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes are
>> >>>> completely
>> >>>>>>>>>>> managed
>> >>>>>>>>>>>> by the management server (via the plug-in). There is no admin
>> >>>>>>>>>>> interaction.
>> >>>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume and a
>> >>>> CloudStack
>> >>>>>>>>>>> volume,
>> >>>>>>>>>>>> which is necessary for any storage vendor that supports true
>> >> QoS.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I would like to clarify how these pieces are related and
>> >>>> expected
>> >>>>>>>>> to
>> >>>>>>>>>>>>> operate.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> >>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs with
>> >>>>>>>>> provisioned
>> >>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>> won't be cut off by the underlying storage."
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> In the storage QoS world, we need to map a single SAN
>> volume
>> >>>>>>>>> (LUN) to a
>> >>>>>>>>>>>>>> single CloudStack volume. We cannot have multiple
>> CloudStack
>> >>>>>>>>> volumes
>> >>>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> If the user wants to have a single SAN volume house more
>> than
>> >>>> one
>> >>>>>>>>>>>>>> CloudStack volume, then can do that today without any of my
>> >>>>>>>>> plug-in
>> >>>>>>>>>>> code.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> "The administrator will allocate a SAN volume for
>> >> CloudStack's
>> >>>>>>>>> use
>> >>>>>>>>>>> onto
>> >>>>>>>>>>>>>>> which CloudStack volumes will be created."
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I think we crossed e-mails. :)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Check out my recent e-mail on this.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>> >>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> You are coming to the part which concerns me -- concepts
>> >> from
>> >>>>>>>>> the
>> >>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only used
>> >> (as
>> >>>>>>>>> far as
>> >>>>>>>>>>> I
>> >>>>>>>>>>>>>>>> know)
>> >>>>>>>>>>>>>>>>> in regards to XenServer (when you have not PreSetup an
>> SR).
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from VMware, it
>> uses
>> >>>> the
>> >>>>>>>>> vmfs
>> >>>>>>>>>>>>> type.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM, it uses
>> the
>> >>>>>>>>>>>>>>>> SharedMountPoint
>> >>>>>>>>>>>>>>>>> type.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here was to
>> make a
>> >>>>>>>>> new type
>> >>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison added
>> into
>> >>>> the
>> >>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>> framework. Maybe it should be more specificy, though:
>> >>>>>>>>> Dynamic_iSCSI
>> >>>>>>>>>>>>>>>> versus,
>> >>>>>>>>>>>>>>>>> say, Dynamic_FC.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or care that
>> it
>> >>>> is
>> >>>>>>>>> being
>> >>>>>>>>>>>>> used
>> >>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer questions
>> about
>> >> it
>> >>>>>>>>> is
>> >>>>>>>>>>>>>>>> storing."
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I see...so your concern here is that the SolidFire
>> plug-in
>> >>>>>>>>> needs to
>> >>>>>>>>>>>>>>>> call
>> >>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic
>> >> knows
>> >>>>>>>>> to
>> >>>>>>>>>>> treat
>> >>>>>>>>>>>>>>>> it as
>> >>>>>>>>>>>>>>>>>> such.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint and just
>> >>>>>>>>> calling it
>> >>>>>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>>> or
>> >>>>>>>>>>>>>>>>>> whatever. We would just need a way for the hypervisor
>> >>>> attach
>> >>>>>>>>> logic
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>> detect this new requirement and perform the necessary
>> >>>>>>>>> activities.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>> >>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> See my responses in-line.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to manage a
>> >>>> device,
>> >>>>>>>>> an
>> >>>>>>>>>>>>>>>> operator
>> >>>>>>>>>>>>>>>>>>> may
>> >>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
>> >>>> device
>> >>>>>>>>> is
>> >>>>>>>>>>>>> shared
>> >>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>> many services, and the operator has chosen to
>> dedicate
>> >>>>>>>>> only a
>> >>>>>>>>>>>>> portion
>> >>>>>>>>>>>>>>>>>>> of it
>> >>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via
>> the
>> >>>>>>>>> plug-in
>> >>>>>>>>>>>>>>>> (before
>> >>>>>>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would work unless
>> the
>> >>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>> consumes
>> >>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> My thinking is that, independent of CloudStack, an
>> >>>> operator
>> >>>>>>>>>>>>> allocates
>> >>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through
>> a
>> >>>>>>>>> LUN. They
>> >>>>>>>>>>>>>>>> simply
>> >>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to CloudStack,
>> but
>> >>>>>>>>> not allow
>> >>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my changes, we
>> would
>> >>>> be
>> >>>>>>>>>>> passing
>> >>>>>>>>>>>>>>>> in a
>> >>>>>>>>>>>>>>>>>>>> SAN volume that does not have the necessary
>> hypervisor
>> >>>>>>>>> support
>> >>>>>>>>>>>>> (like
>> >>>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>> SR)
>> >>>>>>>>>>>>>>>>>>>> and the logic will fail.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the storage
>> >>>> framework
>> >>>>>>>>>>> itself
>> >>>>>>>>>>>>>>>>>>> detect
>> >>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from the
>> hypervisor
>> >>>>>>>>> side and
>> >>>>>>>>>>>>>>>> have
>> >>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>> call into the agent code specifically to create the
>> SR
>> >>>>>>>>> before the
>> >>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin should have a
>> >>>> rich
>> >>>>>>>>> enough
>> >>>>>>>>>>>>>>>>>>> interface to storage to determine available for volume
>> >>>>>>>>> storage.
>> >>>>>>>>>>> For
>> >>>>>>>>>>>>>>>> Xen,
>> >>>>>>>>>>>>>>>>>>> this interface would allow the interrogation of the
>> >> device
>> >>>>>>>>> to
>> >>>>>>>>>>>>>>>> determine the
>> >>>>>>>>>>>>>>>>>>> SR is present. The storage device itself shouldn't
>> know
>> >>>>>>>>> or care
>> >>>>>>>>>>>>>>>> that it
>> >>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>> >>>>>>>>> questions
>> >>>>>>>>>>>>>>>> about it
>> >>>>>>>>>>>>>>>>>>> is storing.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver (which
>> is a
>> >>>>>>>>> type of
>> >>>>>>>>>>>>>>>>>>> so-called
>> >>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
>> >>>>>>>>> Storage shows
>> >>>>>>>>>>>>> up
>> >>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> applicable zone.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that references
>> the
>> >>>>>>>>> storage
>> >>>>>>>>>>>>> tag
>> >>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack volume. This
>> leads
>> >>>> to
>> >>>>>>>>> a new
>> >>>>>>>>>>>>> row
>> >>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack volume to a
>> VM
>> >>>>>>>>> (attach
>> >>>>>>>>>>>>> disk).
>> >>>>>>>>>>>>>>>>>>> This
>> >>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the plug-in
>> to
>> >>>>>>>>> create a
>> >>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>> volume
>> >>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN). The
>> plug-in
>> >>>>>>>>> also
>> >>>>>>>>>>>>> updates
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data (like the
>> IQN of
>> >>>>>>>>> the SAN
>> >>>>>>>>>>>>>>>>>>> volume).
>> >>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
>> >>>>>>>>> volume is in
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the volume
>> will be
>> >>>>>>>>> in the
>> >>>>>>>>>>>>>>>> 'Ready'
>> >>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the plug-in
>> code
>> >>>>>>>>> will not
>> >>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>> called
>> >>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>> >>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>> volume
>> >>>>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a
>> hypervisor
>> >>>> data
>> >>>>>>>>>>>>> structure
>> >>>>>>>>>>>>>>>>>>> (ex.
>> >>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on terminology,
>> volume,
>> >>>>>>>>> in these
>> >>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical allocation on
>> the
>> >>>>>>>>> device,
>> >>>>>>>>>>>>>>>>>>> correct?"
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN
>> volume'.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>> >>>>>>>>> CloudStack, I
>> >>>>>>>>>>>>>>>> try to
>> >>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> What you say here may very well make sense, but
>> I'm
>> >>>>>>>>> having a
>> >>>>>>>>>>>>> hard
>> >>>>>>>>>>>>>>>>>>> time
>> >>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on this
>> conversation
>> >>>>>>>>> as he
>> >>>>>>>>>>> was
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I took.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> What do you think?
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>> >>>>>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two distinct
>> concepts
>> >>>> --
>> >>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In both
>> cases, we
>> >>>>>>>>> are
>> >>>>>>>>>>>>>>>>>>> communicating with
>> >>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
>> >>>>>>>>> volumes
>> >>>>>>>>>>>>>>>> (Dynamic)
>> >>>>>>>>>>>>>>>>>>> on the
>> >>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the volume to be
>> >>>>>>>>> volume to be
>> >>>>>>>>>>>>>>>>>>> managed
>> >>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure
>> that
>> >> we
>> >>>>>>>>> are in
>> >>>>>>>>>>>>>>>> sync on
>> >>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these definitions,
>> refers to
>> >>>>>>>>> the
>> >>>>>>>>>>>>> physical
>> >>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we
>> >>>> must
>> >>>>>>>>> be
>> >>>>>>>>>>> able
>> >>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to move bits
>> from
>> >>>>>>>>> one place
>> >>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>> another,
>> >>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a
>> storage
>> >>>>>>>>> device
>> >>>>>>>>>>> may
>> >>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>> able to
>> >>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
>> >>>>>>>>> unmanaged
>> >>>>>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
>> >>>> managed
>> >>>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable of
>> allocating
>> >>>>>>>>> LUNs and
>> >>>>>>>>>>>>>>>> storing
>> >>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able
>> to
>> >>>>>>>>> manage a
>> >>>>>>>>>>>>>>>> device,
>> >>>>>>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
>> >>>> CloudStack
>> >>>>>>>>> (e.g.
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> device is
>> >>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the operator has
>> chosen
>> >>>> to
>> >>>>>>>>>>>>> dedicate
>> >>>>>>>>>>>>>>>>>>> only a
>> >>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning
>> >> make
>> >>>>>>>>> sense?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it
>> seems
>> >>>>>>>>>>> appropriate
>> >>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from
>> StoragePoolType,
>> >>>>>>>>> add the
>> >>>>>>>>>>>>>>>> notion
>> >>>>>>>>>>>>>>>>>>> of a
>> >>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver that
>> >> indicates
>> >>>>>>>>> whether
>> >>>>>>>>>>>>> or
>> >>>>>>>>>>>>>>>>>>> not is
>> >>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish a
>> abstraction
>> >>>>>>>>>>>>> representing a
>> >>>>>>>>>>>>>>>>>>> physical
>> >>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is
>> associated
>> >>>>>>>>> with a
>> >>>>>>>>>>>>> volume.
>> >>>>>>>>>>>>>>>>>>> With
>> >>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
>> >>>> declare
>> >>>>>>>>> which
>> >>>>>>>>>>>>>>>>>>> protocols they
>> >>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a device managed
>> by
>> >>>>>>>>>>> CloudStack,
>> >>>>>>>>>>>>>>>>>>> utilize the
>> >>>>>>>>>>>>>>>>>>>>>>>> management operations exposed by the driver to
>> >>>> automate
>> >>>>>>>>>>>>>>>> allocation.
>> >>>>>>>>>>>>>>>>>>> If
>> >>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can
>> sit
>> >>>>>>>>> down and
>> >>>>>>>>>>>>>>>> drill
>> >>>>>>>>>>>>>>>>>>> down to
>> >>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the current iSCSI
>> >>>> type
>> >>>>>>>>> and
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> Dynamic
>> >>>>>>>>>>>>>>>>>>>>>>>> type:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
>> >>>>>>>>> Primary
>> >>>>>>>>>>>>> Storage
>> >>>>>>>>>>>>>>>>>>> based
>> >>>>>>>>>>>>>>>>>>>>>>>> on
>> >>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
>> >>>>>>>>> volume must
>> >>>>>>>>>>>>>>>> exist
>> >>>>>>>>>>>>>>>>>>> on
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
>> >>>>>>>>> CloudStack
>> >>>>>>>>>>>>> volumes
>> >>>>>>>>>>>>>>>> are
>> >>>>>>>>>>>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
>> >>>> scenes.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
>> >>>>>>>>> Primary
>> >>>>>>>>>>>>> Storage
>> >>>>>>>>>>>>>>>>>>> based
>> >>>>>>>>>>>>>>>>>>>>>>>> on a
>> >>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on
>> its
>> >>>>>>>>> storage
>> >>>>>>>>>>>>>>>> system
>> >>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
>> >>>>>>>>> framework). When
>> >>>>>>>>>>> a
>> >>>>>>>>>>>>>>>> user
>> >>>>>>>>>>>>>>>>>>>>>>>> wants to
>> >>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>> >>>>>>>>> framework
>> >>>>>>>>>>>>> tells
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is
>> done,
>> >>>>>>>>> the
>> >>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called. No
>> hypervisor
>> >>>>>>>>> data
>> >>>>>>>>>>>>>>>> structure
>> >>>>>>>>>>>>>>>>>>>>>>>> exists
>> >>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was just
>> created.
>> >>>> The
>> >>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should probably have
>> >>>>>>>>> defined them
>> >>>>>>>>>>>>> up
>> >>>>>>>>>>>>>>>>>>> front
>> >>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated
>> (ex.
>> >>>>>>>>> an admin
>> >>>>>>>>>>>>>>>>>>> creates a
>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data
>> structure
>> >>>> is
>> >>>>>>>>>>> created
>> >>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> consume
>> >>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
>> >>>> hypervisor
>> >>>>>>>>> data
>> >>>>>>>>>>>>>>>>>>> structure
>> >>>>>>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
>> >>>> placed
>> >>>>>>>>> on
>> >>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
>> >>>> attach
>> >>>>>>>>> logic
>> >>>>>>>>>>>>>>>> assumes
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already in place
>> and
>> >>>>>>>>> simply
>> >>>>>>>>>>>>>>>> attaches
>> >>>>>>>>>>>>>>>>>>>>>>>> the VDI
>> >>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
>> >>>>>>>>> question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
>> >> pre-allocated.
>> >>>>>>>>> Instead
>> >>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not
>> a
>> >>>>>>>>> volume on
>> >>>>>>>>>>> a
>> >>>>>>>>>>>>>>>> SAN,
>> >>>>>>>>>>>>>>>>>>>>>>>> but the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure
>> must be
>> >>>>>>>>> created
>> >>>>>>>>>>>>>>>> when an
>> >>>>>>>>>>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these types of
>> volumes
>> >>>>>>>>> have not
>> >>>>>>>>>>>>>>>> been
>> >>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>> >>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an
>> >> admin.
>> >>>>>>>>> Once
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this
>> volume,
>> >>>> it
>> >>>>>>>>>>> attaches
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> one and
>> >>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>> >>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
>> >>>>>>>>> attach
>> >>>>>>>>>>>>> behavior
>> >>>>>>>>>>>>>>>>>>>>>>>> works in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>> >>>>>>>>> difference.
>> >>>>>>>>>>>>>>>> Another
>> >>>>>>>>>>>>>>>>>>>>>>>> way to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the definition of
>> a
>> >>>>>>>>> Dynamic
>> >>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>> pool type?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely
>> >> used
>> >>>>>>>>> by
>> >>>>>>>>>>>>>>>> XenServer
>> >>>>>>>>>>>>>>>>>>>>>>>> when you
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is
>> directly
>> >>>>>>>>> based on
>> >>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to
>> the
>> >>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI
>> target as
>> >>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>> does
>> >>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> part
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
>> >>>>>>>>> XenServer.
>> >>>>>>>>>>> For
>> >>>>>>>>>>>>>>>> all
>> >>>>>>>>>>>>>>>>>>>>>>>> other
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the
>> hypervisor
>> >>>>>>>>> and
>> >>>>>>>>>>>>> perform
>> >>>>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>>>>> step
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell
>> <
>> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the
>> >> difference
>> >>>>>>>>> between
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
>> >>>> Dynamic?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>> >>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> class
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a
>> >> clusterfs
>> >>>>>>>>>>> overlay
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage
>> Pool
>> >>>>>>>>> is set
>> >>>>>>>>>>>>> up
>> >>>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide
>> storage
>> >>>> (ex.
>> >>>>>>>>>>>>>>>>>>> SolidFire)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike
>> Tutkowski
>> >>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage
>> >> company
>> >>>>>>>>> wants
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> implement a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>> >>>>>>>>> feature. It
>> >>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would
>> need
>> >>>> only
>> >>>>>>>>>>>>>>>> implement a
>> >>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary
>> changes to
>> >>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
>> >>>> Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
>> >>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>> specific.
>> >>>>>>>>>>>>>>>>>>>>>>>> It is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware
>> of
>> >>>>>>>>> dynamic,
>> >>>>>>>>>>>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of
>> this
>> >>>>>>>>> kind of
>> >>>>>>>>>>>>>>>>>>> storage).
>> >>>>>>>>>>>>>>>>>>>>>>>> This
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's
>> storage
>> >>>>>>>>> framework
>> >>>>>>>>>>>>>>>>>>> changes.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that
>> >> supported
>> >>>>>>>>> the
>> >>>>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
>> >>>>>>>>> visited with
>> >>>>>>>>>>>>> him
>> >>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>> Portland
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
>> >>>>>>>>> complete. We
>> >>>>>>>>>>>>>>>> realized
>> >>>>>>>>>>>>>>>>>>>>>>>> there
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
>> >>>> volumes
>> >>>>>>>>> unless
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of
>> >> storage
>> >>>>>>>>> and
>> >>>>>>>>>>>>> create
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John
>> Burwell
>> >>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any
>> >> operation
>> >>>>>>>>> in the
>> >>>>>>>>>>>>>>>> Storage
>> >>>>>>>>>>>>>>>>>>>>>>>> layer
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I
>> expect
>> >>>>>>>>> to see
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> Storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>> >>>>>>>>>>> composed/driven
>> >>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>> >>>>>>>>> structures
>> >>>>>>>>>>>>> per
>> >>>>>>>>>>>>>>>>>>> their
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are
>> going
>> >>>> to
>> >>>>>>>>> end
>> >>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>>>>>> massive
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
>> >>>> increasingly
>> >>>>>>>>>>>>>>>> difficult to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the
>> Xen
>> >>>>>>>>> specific
>> >>>>>>>>>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>>>>>>>> code
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
>> >>>>>>>>> Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more
>> detail:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
>> >>>> method
>> >>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>> XenServer,
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
>> >>>>>>>>> system,
>> >>>>>>>>>>> then
>> >>>>>>>>>>>>> go
>> >>>>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use
>> of
>> >>>> the
>> >>>>>>>>>>> volume
>> >>>>>>>>>>>>>>>> (ex.
>> >>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore
>> on
>> >>>>>>>>> ESX(i)).
>> >>>>>>>>>>>>> VMs
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
>> >>>> storage
>> >>>>>>>>> need
>> >>>>>>>>>>> no
>> >>>>>>>>>>>>>>>>>>> longer
>> >>>>>>>>>>>>>>>>>>>>>>>> be so
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
>> >>>> relationship
>> >>>>>>>>>>> between
>> >>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk
>> (necessary
>> >>>> for
>> >>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>> Quality
>> >>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is
>> called
>> >>>> to
>> >>>>>>>>>>>>>>>> dynamically
>> >>>>>>>>>>>>>>>>>>>>>>>> create
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework
>> did
>> >>>>>>>>> not
>> >>>>>>>>>>>>> address
>> >>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
>> >>>>>>>>> structure
>> >>>>>>>>>>> when
>> >>>>>>>>>>>>>>>>>>>>>>>> performing an
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been
>> enhancing
>> >> it
>> >>>>>>>>> to do
>> >>>>>>>>>>> so.
>> >>>>>>>>>>>>>>>> I've
>> >>>>>>>>>>>>>>>>>>>>>>>> got
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got
>> ESX(i)
>> >>>> in
>> >>>>>>>>> my
>> >>>>>>>>>>>>> sandbox
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
>> >>>>>>>>> Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
>> >>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>> agnostic.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach
>> logic
>> >>>>>>>>> (in the
>> >>>>>>>>>>>>>>>> agent
>> >>>>>>>>>>>>>>>>>>>>>>>> code).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to
>> have
>> >>>> it
>> >>>>>>>>>>> create a
>> >>>>>>>>>>>>>>>>>>> volume
>> >>>>>>>>>>>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the
>> volume
>> >>>> to
>> >>>>>>>>> a
>> >>>>>>>>>>>>>>>> hypervisor,
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize
>> it's
>> >>>>>>>>> being
>> >>>>>>>>>>>>>>>> invoked on
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been
>> created)
>> >>>> and
>> >>>>>>>>>>> create,
>> >>>>>>>>>>>>>>>>>>> say, a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a
>> datastore
>> >>>>>>>>> (for
>> >>>>>>>>>>>>>>>> VMware) to
>> >>>>>>>>>>>>>>>>>>>>>>>> make
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> use
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
>> >>>> recently
>> >>>>>>>>> making
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
>> >>>>>>>>> Burwell <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
>> >>>> driver
>> >>>>>>>>> is
>> >>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
>> >>>> Tutkowski
>> >>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to
>> support
>> >>>>>>>>> all
>> >>>>>>>>>>>>>>>> hypervisors
>> >>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of
>> time
>> >>>>>>>>> for 4.2
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>>>>> get
>> >>>>>>>>>>>>>>>>>>> KVM
>> >>>>>>>>>>>>>>>>>>>>>>>> in.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
>> >>>> XenServer.
>> >>>>>>>>>>>>>>>> Depending on
>> >>>>>>>>>>>>>>>>>>>>>>>> what
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I
>> have
>> >> it
>> >>>>>>>>> working
>> >>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>> VMware in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> my
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all
>> in
>> >>>>>>>>> regards
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>>>>> Disk
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM,
>> Kelcey
>> >>>>>>>>> Jamison
>> >>>>>>>>>>>>>>>> Damage
>> >>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting
>> KVM in
>> >>>>>>>>> the
>> >>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>> cycle
>> >>>>>>>>>>>>>>>>>>>>>>>> post
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013
>> 10:12:32 AM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
>> >> disk_io_throttling
>> >>>>>>>>> to
>> >>>>>>>>>>> MASTER
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature
>> >> first,
>> >>>>>>>>> then
>> >>>>>>>>>>>>> mine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only,
>> then
>> >> it
>> >>>>>>>>> is a
>> >>>>>>>>>>> non
>> >>>>>>>>>>>>>>>> issue
>> >>>>>>>>>>>>>>>>>>>>>>>> as I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei
>> >>>> ZHOU
>> >>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works
>> on
>> >>>>>>>>> billing
>> >>>>>>>>>>> now,
>> >>>>>>>>>>>>>>>>>>> users
>> >>>>>>>>>>>>>>>>>>>>>>>> need
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They
>> can
>> >>>> get
>> >>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> service_offering
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes
>> >> for
>> >>>>>>>>>>>>>>>> calculation.
>> >>>>>>>>>>>>>>>>>>> Of
>> >>>>>>>>>>>>>>>>>>>>>>>> course
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation
>> >>>> value
>> >>>>>>>>> of
>> >>>>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>>>>>>>>> volume,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well.
>> I
>> >>>> can
>> >>>>>>>>> work
>> >>>>>>>>>>> on
>> >>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>> later. Do
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> you
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>> >>>>>>>>> question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the
>> >> two
>> >>>>>>>>>>> features.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>> >>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei
>> ZHOU
>> >>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you.
>> I am
>> >>>>>>>>> trying
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>> adding
>> >>>>>>>>>>>>>>>>>>>>>>>> more.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should
>> depend
>> >> on
>> >>>>>>>>> IO
>> >>>>>>>>>>>>>>>> statistics
>> >>>>>>>>>>>>>>>>>>>>>>>> rather
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> than
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
>> >>>>>>>>> disk_io_stat if
>> >>>>>>>>>>> you
>> >>>>>>>>>>>>>>>> have
>> >>>>>>>>>>>>>>>>>>>>>>>> time.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>> >>>>>>>>> bytes/iops
>> >>>>>>>>>>>>>>>>>>> read/write
>> >>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers
>> >> are
>> >>>>>>>>> billed
>> >>>>>>>>>>>>> more
>> >>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for
>> >>>> those
>> >>>>>>>>>>>>>>>> operations (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I
>> >>>> would
>> >>>>>>>>>>> imagine
>> >>>>>>>>>>>>>>>> our
>> >>>>>>>>>>>>>>>>>>>>>>>> users
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost
>> >> models.
>> >>>>>>>>> Could
>> >>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>>>>>> operator
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current
>> patch?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime
>> >> change?
>> >>>>>>>>> KVM
>> >>>>>>>>>>>>>>>> supports
>> >>>>>>>>>>>>>>>>>>>>>>>> setting
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
>> >>>>>>>>> machine
>> >>>>>>>>>>>>> through
>> >>>>>>>>>>>>>>>>>>>>>>>> command
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support
>> >> changing
>> >>>>>>>>> the
>> >>>>>>>>>>>>>>>> parameters
>> >>>>>>>>>>>>>>>>>>>>>>>> of a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
>> >>>> offering).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface
>> level.
>> >>>> I
>> >>>>>>>>>>>>> apologize
>> >>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>> being
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
>> >>>>>>>>> algorithms
>> >>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>> set
>> >>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
>> >>>>>>>>> provided by a
>> >>>>>>>>>>>>>>>>>>> resource?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question.
>> Maybe it
>> >>>> is
>> >>>>>>>>>>> better
>> >>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> commit
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike
>> needs to
>> >>>>>>>>> consider
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a
>> >>>> later
>> >>>>>>>>>>>>> response
>> >>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> Mike
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
>> >>>>>>>>> features. I
>> >>>>>>>>>>>>> think
>> >>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
>> >>>>>>>>> SolidFire, but
>> >>>>>>>>>>> I
>> >>>>>>>>>>>>>>>> think
>> >>>>>>>>>>>>>>>>>>>>>>>> we need
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>> >>>>>>>>> (possibly
>> >>>>>>>>>>>>> have
>> >>>>>>>>>>>>>>>>>>> have
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to
>> coordinate on
>> >>>>>>>>> this
>> >>>>>>>>>>>>> issue.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>> >>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand
>> >> are
>> >>>>>>>>> the
>> >>>>>>>>>>>>>>>> following:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing
>> >> IOPS
>> >>>>>>>>>>> policies
>> >>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>> captured
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for
>> billing/usage
>> >>>>>>>>>>> purposes,
>> >>>>>>>>>>>>>>>>>>>>>>>> expressing
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
>> >>>>>>>>> interface model
>> >>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>> reasoning
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting
>> provisioned
>> >>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>> configurations
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>> >>>>>>>>> reconciled?
>> >>>>>>>>>>> In
>> >>>>>>>>>>>>>>>>>>>>>>>> particular,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed)
>> >> for
>> >>>>>>>>> more
>> >>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>> configured
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been
>> configured
>> >>>> to
>> >>>>>>>>>>>>> deliver.
>> >>>>>>>>>>>>>>>>>>>>>>>> Another
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration
>> between a
>> >>>>>>>>> VM and a
>> >>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification
>> to
>> >>>>>>>>> storage
>> >>>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM,
>> Mike
>> >>>>>>>>> Tutkowski
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals
>> with
>> >>>>>>>>>>>>> controlling
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> max
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
>> >>>> controlling
>> >>>>>>>>> IOPS
>> >>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM,
>> >>>> John
>> >>>>>>>>>>> Burwell
>> >>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
>> >>>>>>>>> should be
>> >>>>>>>>>>>>>>>> merged
>> >>>>>>>>>>>>>>>>>>>>>>>> until all
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and
>> it
>> >>>> is
>> >>>>>>>>> ready
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>>>>> turn
>> >>>>>>>>>>>>>>>>>>>>>>>> over to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete
>> read/write
>> >>>>>>>>> ops issue
>> >>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>> addressed
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this
>> criteria
>> >>>>>>>>> has been
>> >>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>> >>>>>>>>>>>>> intersect/compliment
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> https://reviews.apache.org/r/11479/
>> >>>>>>>>> )?
>> >>>>>>>>>>> As I
>> >>>>>>>>>>>>>>>>>>>>>>>> understand
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned
>> IOPS. I
>> >>>>>>>>> would
>> >>>>>>>>>>>>> like
>> >>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> ensure
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned
>> IOPS
>> >> in
>> >>>>>>>>> KVM and
>> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM,
>> Wei
>> >>>>>>>>> ZHOU <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next
>> week.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>> >>>>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei
>> ZHOU
>> >>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
>> >>>> features
>> >>>>>>>>> into
>> >>>>>>>>>>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
>> >>>> technical
>> >>>>>>>>>>> reason
>> >>>>>>>>>>>>>>>> till
>> >>>>>>>>>>>>>>>>>>>>>>>> Monday,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just
>> >>>> want
>> >>>>>>>>> to
>> >>>>>>>>>>>>> mention
>> >>>>>>>>>>>>>>>>>>> again
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps
>> asap
>> >> so
>> >>>>>>>>> that we
>> >>>>>>>>>>>>>>>> never
>> >>>>>>>>>>>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for
>> the
>> >>>> I/O
>> >>>>>>>>>>>>> throttling
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> that's
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
>> >>>>>>>>> legacy from
>> >>>>>>>>>>>>> day
>> >>>>>>>>>>>>>>>>>>> #1.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >>>>>>>>>>>>> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
>> >>>> Burwell
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>> >>>>>>>>> discretely
>> >>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be
>> fixed
>> >>>>>>>>> later
>> >>>>>>>>>>>>> because
>> >>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>> don't see
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
>> >>>>>>>>> write.
>> >>>>>>>>>>>>>>>>>>> Therefore, we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided
>> to
>> >>>>>>>>> break
>> >>>>>>>>>>>>>>>> backwards
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it
>> >> into
>> >>>>>>>>> master
>> >>>>>>>>>>>>> now
>> >>>>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>>>>>>>>> that it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read /
>> Write
>> >>>> IOps
>> >>>>>>>>>>> before
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and
>> >>>> Write
>> >>>>>>>>>>> instead
>> >>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>> Total
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
>> >>>> feature
>> >>>>>>>>>>> freeze
>> >>>>>>>>>>>>>>>> date.
>> >>>>>>>>>>>>>>>>>>>>>>>> But if
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it
>> won't
>> >>>> be
>> >>>>>>>>>>>>> necessary
>> >>>>>>>>>>>>>>>>>>> to do
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that
>> >> there
>> >>>>>>>>> is no
>> >>>>>>>>>>>>>>>>>>> association
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM,
>> Wido
>> >>>>>>>>> den
>> >>>>>>>>>>>>>>>> Hollander <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei
>> ZHOU
>> >>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the
>> >> beginning.
>> >>>>>>>>>>> Finally I
>> >>>>>>>>>>>>>>>>>>>>>>>> decided to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
>> >>>> mainly
>> >>>>>>>>>>> because
>> >>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent
>> and
>> >>>>>>>>> received
>> >>>>>>>>>>>>>>>> bytes as
>> >>>>>>>>>>>>>>>>>>>>>>>> well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
>> >>>>>>>>> Networking and
>> >>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>> I/O
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more
>> expensive
>> >> in
>> >>>>>>>>> most
>> >>>>>>>>>>>>>>>>>>> situations
>> >>>>>>>>>>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
>> >>>>>>>>> copy-paste
>> >>>>>>>>>>>>> work.
>> >>>>>>>>>>>>>>>> It
>> >>>>>>>>>>>>>>>>>>>>>>>> could be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
>> >>>>>>>>> feature
>> >>>>>>>>>>>>> freeze,
>> >>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we
>> can't
>> >>>>>>>>> miss. But
>> >>>>>>>>>>>>> if
>> >>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>> goes
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we
>> >>>> don't
>> >>>>>>>>>>> release
>> >>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
>> >>>> users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >>>>>>>>>>>>> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei
>> ZHOU
>> >>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>> >>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>> branch
>> >>>>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge
>> >>>> into
>> >>>>>>>>> master
>> >>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>> 48
>> >>>>>>>>>>>>>>>>>>>>>>>> hours.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running
>> on
>> >>>> the
>> >>>>>>>>> same
>> >>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the
>> >> rate
>> >>>>>>>>>>>>> limitation
>> >>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk
>> operation,
>> >>>> it
>> >>>>>>>>> may
>> >>>>>>>>>>>>> affect
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
>> >>>>>>>>> storage
>> >>>>>>>>>>>>> device.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the
>> >> maximum
>> >>>>>>>>> rate
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>> limit
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you
>> >> make
>> >>>>>>>>> no
>> >>>>>>>>>>>>>>>> difference
>> >>>>>>>>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support
>> setting
>> >>>>>>>>> both a
>> >>>>>>>>>>>>>>>> different
>> >>>>>>>>>>>>>>>>>>>>>>>> rate
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a
>> lot of
>> >>>>>>>>> users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the
>> polling
>> >>>>>>>>> side you
>> >>>>>>>>>>>>>>>>>>> collect
>> >>>>>>>>>>>>>>>>>>>>>>>> both
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling
>> side
>> >>>>>>>>> you only
>> >>>>>>>>>>> go
>> >>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much
>> more
>> >>>>>>>>> expensive
>> >>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where
>> that
>> >>>> an
>> >>>>>>>>> admin
>> >>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>> set
>> >>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM
>> at
>> >>>> this
>> >>>>>>>>>>> point I
>> >>>>>>>>>>>>>>>>>>> think
>> >>>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
>> >>>>>>>>> mechanism in
>> >>>>>>>>>>>>> place
>> >>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of
>> >> work.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't
>> support
>> >>>>>>>>> setting
>> >>>>>>>>>>>>>>>> different
>> >>>>>>>>>>>>>>>>>>>>>>>> values
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both
>> up
>> >>>>>>>>> and set
>> >>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
>> >>>> implemented
>> >>>>>>>>> it
>> >>>>>>>>>>> this
>> >>>>>>>>>>>>>>>> way?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of
>> VMs
>> >>>> (in
>> >>>>>>>>>>>>>>>>>>> disk_offering,
>> >>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of
>> >> VMs
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total
>> >> bps
>> >>>>>>>>> and
>> >>>>>>>>>>> iops)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>> >>>>>>>>>>> https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>> >>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>> >>>> ****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's
>> RAT
>> >>>>>>>>> execution
>> >>>>>>>>>>>>>>>>>>> success?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>> >>>>>>>>> introduced?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit
>> >> and
>> >>>>>>>>>>>>>>>> integration)
>> >>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> included
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done
>> to
>> >>>>>>>>> check for
>> >>>>>>>>>>>>>>>>>>> potential
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and
>> IOPS
>> >>>>>>>>> rate on
>> >>>>>>>>>>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>>>>>>>>>> UI.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
>> >>>>>>>>> destroy,
>> >>>>>>>>>>>>> expunge.
>> >>>>>>>>>>>>>>>>>>>>>>>> migrate,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations,
>> including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can
>> try
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> >>>>>>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>> >>>> ****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>> refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>> >>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> >>>>>>>>> SolidFire
>> >>>>>>>>>>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e:
>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world
>> uses
>> >>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> >> SolidFire
>> >>>>>>>>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>> >>>>>>>>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>> >>>>>>>>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>> cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> >>>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> >> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>> cloud<
>> >> http://solidfire.com/solution/overview/?video=play
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>> *â„¢*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the
>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>> *â„¢*
>> >>>>
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I agree, John. Just wanted to point out that I have a working GUI for you
to review (in that document), if you'd like to check it out.
Thanks!
On Thu, Jun 6, 2013 at 8:34 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I would like the UIs of two features reviewed together to ensure
> consistency across the concepts of hypervisor throttled IOPs and
> storage device provisioned IOPs. I see the potential for confusion,
> and I think a side-by-side Ui review of these features will help
> minimize any potential confusion.
>
> As I mentioned, the term reconciliation issue will work itself if it
> is acceptable that a VM is only permitted utilize hypervisor throttled
> IOPs or storage provisioned IOPs.
>
> Thanks,
> -John
>
> On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
> <mi...@solidfire.com> wrote:
>
> > Hi John,
> >
> > Yeah, when you get a chance, refer to the Google doc I sent to you the
> > other day to see how the GUI looks for provisioned storage IOPS.
> >
> > Several months ago, I put this topic out on the e-mail list and we
> decided
> > to place the Min, Max, and Burst IOPS in the Add Disk Offering dialog.
> > Other storage vendors are coming out with QoS, so they should be able to
> > leverage this GUI going forward (even if they, say, only use Max IOPS).
> > These fields are optional and can be ignored for storage that does not
> > support provisioned IOPS. Just like the Disk Size field, the admin can
> > choose to allow the end user to fill in Min, Max, and Burst IOPS.
> >
> > I'm OK if we do an either/or model (either Wei's feature or mine, as is
> > decided by the admin).
> >
> > I'm not sure what we can do about these two features expressing the speed
> > in different terms. I've never seen a SAN express the IOPS for QoS in any
> > way other than total IOPS (i.e. not broken in into read/write IOPS).
> >
> > Thanks
> >
> >
> > On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com> wrote:
> >
> >> Wei,
> >>
> >> We have been down the rabbit hole a bit on the Storage/Hypervisor layer
> >> separation, but we still need to reconcile the behavior of hypervisor
> >> throttled I/O and storage provisioned IOPS. I see the following issues
> >> outstanding:
> >>
> >> 1. Hypervisor throttled IOPS are expressed as discrete read/write values
> >> where as storage provisioned IOPS are expressed as total IOPS.
> >> 2. How do we handle VMs with throttled IOPS attached to storage volumes
> >> with provisioned IOPS?
> >> 3. How should usage data be captured for throttled and provisioned IOPS
> >> that will permit providers to discriminate these guaranteed operations
> in
> >> the event they want to bill for it differently?
> >> 4. What is the user experience for throttled and provisioned IOPS that
> >> minimizes confusion of these concepts?
> >>
> >> My thinking is that a VM can have either utilize hypervisor throttled
> IOPS
> >> or storage provisioned IOPS. This policy would neatly solve items 1
> and 2.
> >> Since the two facilities would not be permitted to operate together,
> they
> >> do not need to be semantically compatible. I think item 3 can be
> resolved
> >> with an additional flag or two on the usage records. As for Item 4, I
> am
> >> not familiar with how these two enhancements are depicted in the user
> >> interface. I think we need to review the UI enhancements for both
> >> enhancements and ensure they are consistent.
> >>
> >> Do these solutions make sense?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com> wrote:
> >>
> >>> John and Mike,
> >>>
> >>> I was busy working on other issues (CLOUDSTACK-2780/2729,
> >>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this
> week.
> >>> I will start to develop on iops/bps changes tomorrow, and ask for
> second
> >>> merge request after testing.
> >>>
> >>> -Wei
> >>>
> >>>
> >>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> >>>
> >>>> I believe I understand where you're going with this, John, and have
> been
> >>>> re-working this section of code today.
> >>>>
> >>>> I should be able to run it by you tomorrow.
> >>>>
> >>>> Thanks for the comments,
> >>>> Mike
> >>>>
> >>>>
> >>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> See my responses in-line below.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>> Hi John,
> >>>>>
> >>>>> Thanks for the response.
> >>>>>
> >>>>> "I am fine with the VolumeManager determining whether or not a Volume
> >> is
> >>>>> managed (i.e. not based on the StoragePoolType, but an actual
> isManaged
> >>>>> method), and asking the device driver to allocate resources for the
> >>>> volume
> >>>>> if it is managed."
> >>>>>
> >>>>> Are you thinking you'd like to see an isManaged() method added to the
> >>>>> PrimaryDataStoreDriver interface? If it returns true, then the
> storage
> >>>>> framework could call the manage() (or whatever name) method (which
> >> would
> >>>> be
> >>>>> new to the PrimaryDataStoreDriver interface, as well) and this would
> >> call
> >>>>> into a new method in the hypervisor code to create, say on XenServer,
> >> an
> >>>> SR?
> >>>>>
> >>>>>
> >>>>> I would like to see canBeManaged() : boolean on DataStoreDriver.
> Since
> >>>>> the notion of Volumes only pertains to primary storage, I would add a
> >>>>> allocateStorage and deallocateStorage (Storage is a straw man term --
> >>>>> something other than volume) methods to
> >> allocate/create/deallocate/delete
> >>>>> underlying storage. To my mind, managed is a mutable property of
> >>>> DataStore
> >>>>> which can be enabled if/when the underlying DataStoreDriver can be
> >>>> managed.
> >>>>> This approach allows operators to override manageability of devices.
> >>>>>
> >>>>> In terms of orchestration/process flow for SR, the Xen plugin would
> be
> >>>>> responsible for composing DataStore/Volume methods to create any
> >>>>> directories or files necessary for the SR. There should be no
> >>>> dependencies
> >>>>> from the Storage to the Hypervisor layer. As I said earlier, such
> >>>> circular
> >>>>> dependencies will lead to a tangled, unmaintainable mess.
> >>>>>
> >>>>>
> >>>>> Just want to make sure I'm on the same page with you.
> >>>>>
> >>>>> Thanks again, John
> >>>>>
> >>>>>
> >>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Fundamentally, we can't end up with a Storage layer that supports n
> >>>>>> devices types with each specific behaviors of m hypervisors. Such a
> >>>>>> scenario will create an unmaintainable and untestable beast.
> >>>> Therefore, my
> >>>>>> thoughts and recommendations are driven to evolve the Storage layer
> >>>> towards
> >>>>>> this separation.
> >>>>>>
> >>>>>> I am fine with the VolumeManager determining whether or not a Volume
> >> is
> >>>>>> managed (i.e. not based on the StoragePoolType, but an actual
> >> isManaged
> >>>>>> method), and asking the device driver to allocate resources for the
> >>>> volume
> >>>>>> if it is managed. Furthermore, the device driver needs to indicate
> >>>> whether
> >>>>>> or not it supports management operations. Finally, I think we need
> to
> >>>>>> provide the ability for an administrator to elect to have something
> >>>> that is
> >>>>>> manageable be unmanaged (i.e. the driver is capable managing the
> >> device,
> >>>>>> but the administrator has elected to leave it unmanaged).
> >>>>>>
> >>>>>> Creation of a structure on the volume should be done in the Xen
> >>>>>> hypervisor module using methods exposed by the Storage layer to
> >> perform
> >>>>>> low-level operations (e.g. make directories, create a file, etc).
> >> This
> >>>>>> structure is specific to the operation of the Xen hypervisor, as
> such,
> >>>>>> should be confined to its implementation. From my perspective,
> >> nothing
> >>>> in
> >>>>>> the Storage layer should be concerned with content. From its
> >>>> perspective,
> >>>>>> structure and data are opaque. It provides the means to query the
> >> data
> >>>> to
> >>>>>> support the interpretation of the content by higher-level layers
> (e.g.
> >>>>>> Hypervisors). To my mind, attach should be a composition of
> >> operations
> >>>>>> from the Storage layer that varies based on the Volume storage
> >> protocol
> >>>>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Hi John,
> >>>>>>
> >>>>>> Alternatively to the way the attach logic is implemented in my
> patch,
> >> we
> >>>>>> could do the following:
> >>>>>>
> >>>>>> Leave the attach logic in the agent code alone. In VolumeManagerImpl
> >> we
> >>>>>> create an AttachVolumeCommand and send it to the hypervisor. Before
> >> this
> >>>>>> command is sent, we could check to see if we're dealing with Dynamic
> >> (or
> >>>>>> whatever we want to call it) storage and - if so - send a "Create
> SR"
> >>>>>> command to the hypervisor. If this returns OK, we would then proceed
> >> to
> >>>> the
> >>>>>> AttachVolumeCommand, as usual.
> >>>>>>
> >>>>>> This way the attach logic remains the same and we just add another
> >>>>>> command to the agent code that is called for this particular type of
> >>>>>> storage.
> >>>>>>
> >>>>>> What do you think?
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Hey John,
> >>>>>>>
> >>>>>>> I created a document for a customer today that outlines how the
> >> plug-in
> >>>>>>> works from a user standpoint. This will probably be of use to you,
> as
> >>>> well,
> >>>>>>> as you perform the code review.
> >>>>>>>
> >>>>>>> I have shared this document with you (you should have received that
> >>>>>>> information in a separate e-mail).
> >>>>>>>
> >>>>>>> Talk to you later!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> Oh, OK, that sounds really good, John.
> >>>>>>>>
> >>>>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jburwell@basho.com
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> I am never at a loss for an opinion. I some thoughts, but want
> to
> >>>>>>>>> confirm assumptions and ideas against the solidfire,
> >>>> disk_io_throttle,
> >>>>>>>>> and
> >>>>>>>>> object_store branches. I hope to collect them in a coherent form
> >>>>>>>>> tomorrow (5 June 2013).
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> "So, in essence, the SolidFire plugin introduces the notion of a
> >>>>>>>>> managed
> >>>>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>>>
> >>>>>>>>>> Technically, the SolidFire plug-in just introduces the notion of
> >>>>>>>>>> provisioned storage IOPS.
> >>>>>>>>>>
> >>>>>>>>>> The storage framework that leverages the plug-in was incomplete,
> >> so
> >>>>>>>>> I had
> >>>>>>>>>> to try to add in the notion of a managed iSCSI device.
> >>>>>>>>>>
> >>>>>>>>>> I appreciate all the time you've been spending on this. :) Do
> you
> >>>>>>>>> have a
> >>>>>>>>>> recommendation as to how we should accomplish what you're
> looking
> >>>>>>>>> for?
> >>>>>>>>>>
> >>>>>>>>>> Thanks!
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <
> jburwell@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> So, in essence, the SolidFire plugin introduces the notion of a
> >>>>>>>>> managed
> >>>>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>>>
> >>>>>>>>>>> I want to see a separation of the management capabilities (i.e.
> >>>> can
> >>>>>>>>> a
> >>>>>>>>>>> device be managed/does an operator want it managed by
> CloudStack)
> >>>>>>>>> from the
> >>>>>>>>>>> storage protocol. Ideally, we should end up with a semantic
> that
> >>>>>>>>> will
> >>>>>>>>>>> allow any type of storage device to be managed. I also want to
> >>>> make
> >>>>>>>>>>> progress on decoupling the storage types from the hypervisor
> >>>>>>>>> definitions.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>
> >>>>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> >> jburwell@basho.com
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Could you please answer the following questions for me with
> >>>>>>>>> regards to
> >>>>>>>>>>> the
> >>>>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN volumes?
> >>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
> >>>>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>>>
> >>>>>>>>>>>>> What is the cardinality between SAN Volumes and CloudStack
> >>>>>>>>> Volumes?
> >>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Are the LUN(s) created by the management server or externally
> >> by
> >>>>>>>>> the
> >>>>>>>>>>>>> operator?
> >>>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume (same as a
> >> SAN
> >>>>>>>>> LUN) is
> >>>>>>>>>>>> created by the management server (via the plug-in) the first
> >> time
> >>>>>>>>> the
> >>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but still want
> >> to
> >>>>>>>>> use a
> >>>>>>>>>>>> SolidFire volume (LUN), you can do this already today (prior
> to
> >>>>>>>>> 4.2). The
> >>>>>>>>>>>> admin manually creates the SAN volume and - in this case -
> >>>>>>>>> multiple VMs
> >>>>>>>>>>> and
> >>>>>>>>>>>> data disks can share this SAN volume. While you can do this
> >>>> today,
> >>>>>>>>> it is
> >>>>>>>>>>>> not useful if you want to enforce storage QoS.
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Are the SAN volumes by the management server or externally by
> >>>> the
> >>>>>>>>>>> operator?
> >>>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes are
> >>>> completely
> >>>>>>>>>>> managed
> >>>>>>>>>>>> by the management server (via the plug-in). There is no admin
> >>>>>>>>>>> interaction.
> >>>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume and a
> >>>> CloudStack
> >>>>>>>>>>> volume,
> >>>>>>>>>>>> which is necessary for any storage vendor that supports true
> >> QoS.
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I would like to clarify how these pieces are related and
> >>>> expected
> >>>>>>>>> to
> >>>>>>>>>>>>> operate.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs with
> >>>>>>>>> provisioned
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>>> won't be cut off by the underlying storage."
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In the storage QoS world, we need to map a single SAN volume
> >>>>>>>>> (LUN) to a
> >>>>>>>>>>>>>> single CloudStack volume. We cannot have multiple CloudStack
> >>>>>>>>> volumes
> >>>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If the user wants to have a single SAN volume house more
> than
> >>>> one
> >>>>>>>>>>>>>> CloudStack volume, then can do that today without any of my
> >>>>>>>>> plug-in
> >>>>>>>>>>> code.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> "The administrator will allocate a SAN volume for
> >> CloudStack's
> >>>>>>>>> use
> >>>>>>>>>>> onto
> >>>>>>>>>>>>>>> which CloudStack volumes will be created."
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> >>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> You are coming to the part which concerns me -- concepts
> >> from
> >>>>>>>>> the
> >>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only used
> >> (as
> >>>>>>>>> far as
> >>>>>>>>>>> I
> >>>>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>>>> in regards to XenServer (when you have not PreSetup an
> SR).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from VMware, it uses
> >>>> the
> >>>>>>>>> vmfs
> >>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM, it uses
> the
> >>>>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here was to
> make a
> >>>>>>>>> new type
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison added
> into
> >>>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>> framework. Maybe it should be more specificy, though:
> >>>>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or care that
> it
> >>>> is
> >>>>>>>>> being
> >>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer questions about
> >> it
> >>>>>>>>> is
> >>>>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I see...so your concern here is that the SolidFire
> plug-in
> >>>>>>>>> needs to
> >>>>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic
> >> knows
> >>>>>>>>> to
> >>>>>>>>>>> treat
> >>>>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint and just
> >>>>>>>>> calling it
> >>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>> whatever. We would just need a way for the hypervisor
> >>>> attach
> >>>>>>>>> logic
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> detect this new requirement and perform the necessary
> >>>>>>>>> activities.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
> >>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to manage a
> >>>> device,
> >>>>>>>>> an
> >>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
> >>>> device
> >>>>>>>>> is
> >>>>>>>>>>>>> shared
> >>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>> many services, and the operator has chosen to dedicate
> >>>>>>>>> only a
> >>>>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
> >>>>>>>>> plug-in
> >>>>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would work unless
> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My thinking is that, independent of CloudStack, an
> >>>> operator
> >>>>>>>>>>>>> allocates
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a
> >>>>>>>>> LUN. They
> >>>>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to CloudStack,
> but
> >>>>>>>>> not allow
> >>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my changes, we
> would
> >>>> be
> >>>>>>>>>>> passing
> >>>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>>>> SAN volume that does not have the necessary hypervisor
> >>>>>>>>> support
> >>>>>>>>>>>>> (like
> >>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the storage
> >>>> framework
> >>>>>>>>>>> itself
> >>>>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from the
> hypervisor
> >>>>>>>>> side and
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>> call into the agent code specifically to create the SR
> >>>>>>>>> before the
> >>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin should have a
> >>>> rich
> >>>>>>>>> enough
> >>>>>>>>>>>>>>>>>>> interface to storage to determine available for volume
> >>>>>>>>> storage.
> >>>>>>>>>>> For
> >>>>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>>>> this interface would allow the interrogation of the
> >> device
> >>>>>>>>> to
> >>>>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>>>> SR is present. The storage device itself shouldn't
> know
> >>>>>>>>> or care
> >>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to answer
> >>>>>>>>> questions
> >>>>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver (which is
> a
> >>>>>>>>> type of
> >>>>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
> >>>>>>>>> Storage shows
> >>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that references
> the
> >>>>>>>>> storage
> >>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack volume. This
> leads
> >>>> to
> >>>>>>>>> a new
> >>>>>>>>>>>>> row
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
> >>>>>>>>> (attach
> >>>>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the plug-in to
> >>>>>>>>> create a
> >>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN). The
> plug-in
> >>>>>>>>> also
> >>>>>>>>>>>>> updates
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN
> of
> >>>>>>>>> the SAN
> >>>>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
> >>>>>>>>> volume is in
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the volume will
> be
> >>>>>>>>> in the
> >>>>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the plug-in code
> >>>>>>>>> will not
> >>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor
> >>>> data
> >>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on terminology,
> volume,
> >>>>>>>>> in these
> >>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical allocation on
> the
> >>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN
> volume'.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can make in
> >>>>>>>>> CloudStack, I
> >>>>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What you say here may very well make sense, but I'm
> >>>>>>>>> having a
> >>>>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on this
> conversation
> >>>>>>>>> as he
> >>>>>>>>>>> was
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two distinct
> concepts
> >>>> --
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In both cases,
> we
> >>>>>>>>> are
> >>>>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
> >>>>>>>>> volumes
> >>>>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the volume to be
> >>>>>>>>> volume to be
> >>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that
> >> we
> >>>>>>>>> are in
> >>>>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these definitions, refers
> to
> >>>>>>>>> the
> >>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we
> >>>> must
> >>>>>>>>> be
> >>>>>>>>>>> able
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to move bits
> from
> >>>>>>>>> one place
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a
> storage
> >>>>>>>>> device
> >>>>>>>>>>> may
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
> >>>>>>>>> unmanaged
> >>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
> >>>> managed
> >>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable of
> allocating
> >>>>>>>>> LUNs and
> >>>>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
> >>>>>>>>> manage a
> >>>>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
> >>>> CloudStack
> >>>>>>>>> (e.g.
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the operator has
> chosen
> >>>> to
> >>>>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning
> >> make
> >>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it
> seems
> >>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from
> StoragePoolType,
> >>>>>>>>> add the
> >>>>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver that
> >> indicates
> >>>>>>>>> whether
> >>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
> >>>>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is associated
> >>>>>>>>> with a
> >>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
> >>>> declare
> >>>>>>>>> which
> >>>>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a device managed
> by
> >>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>>>> management operations exposed by the driver to
> >>>> automate
> >>>>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can
> sit
> >>>>>>>>> down and
> >>>>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the current iSCSI
> >>>> type
> >>>>>>>>> and
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
> >>>>>>>>> Primary
> >>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
> >>>>>>>>> volume must
> >>>>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
> >>>> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
> >>>>>>>>> Primary
> >>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on
> its
> >>>>>>>>> storage
> >>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
> >>>>>>>>> framework). When
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
> >>>>>>>>> framework
> >>>>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is
> done,
> >>>>>>>>> the
> >>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called. No
> hypervisor
> >>>>>>>>> data
> >>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was just
> created.
> >>>> The
> >>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should probably have
> >>>>>>>>> defined them
> >>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated
> (ex.
> >>>>>>>>> an admin
> >>>>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data
> structure
> >>>> is
> >>>>>>>>>>> created
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
> >>>> hypervisor
> >>>>>>>>> data
> >>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
> >>>> placed
> >>>>>>>>> on
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
> >>>> attach
> >>>>>>>>> logic
> >>>>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already in place
> and
> >>>>>>>>> simply
> >>>>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
> >>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
> >> pre-allocated.
> >>>>>>>>> Instead
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
> >>>>>>>>> volume on
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must
> be
> >>>>>>>>> created
> >>>>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these types of
> volumes
> >>>>>>>>> have not
> >>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an
> >> admin.
> >>>>>>>>> Once
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this
> volume,
> >>>> it
> >>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
> >>>>>>>>> attach
> >>>>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
> >>>>>>>>> difference.
> >>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
> >>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely
> >> used
> >>>>>>>>> by
> >>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is
> directly
> >>>>>>>>> based on
> >>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to
> the
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target
> as
> >>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
> >>>>>>>>> XenServer.
> >>>>>>>>>>> For
> >>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the
> hypervisor
> >>>>>>>>> and
> >>>>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the
> >> difference
> >>>>>>>>> between
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
> >>>> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a
> >> clusterfs
> >>>>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage
> Pool
> >>>>>>>>> is set
> >>>>>>>>>>>>> up
> >>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage
> >>>> (ex.
> >>>>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike
> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage
> >> company
> >>>>>>>>> wants
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
> >>>>>>>>> feature. It
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need
> >>>> only
> >>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes
> to
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
> >>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
> >>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
> >>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of
> this
> >>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's
> storage
> >>>>>>>>> framework
> >>>>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that
> >> supported
> >>>>>>>>> the
> >>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
> >>>>>>>>> visited with
> >>>>>>>>>>>>> him
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
> >>>>>>>>> complete. We
> >>>>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
> >>>> volumes
> >>>>>>>>> unless
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of
> >> storage
> >>>>>>>>> and
> >>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John
> Burwell
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any
> >> operation
> >>>>>>>>> in the
> >>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I
> expect
> >>>>>>>>> to see
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> >>>>>>>>>>> composed/driven
> >>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
> >>>>>>>>> structures
> >>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are
> going
> >>>> to
> >>>>>>>>> end
> >>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
> >>>> increasingly
> >>>>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
> >>>>>>>>> specific
> >>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
> >>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
> >>>> method
> >>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
> >>>>>>>>> system,
> >>>>>>>>>>> then
> >>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use
> of
> >>>> the
> >>>>>>>>>>> volume
> >>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore
> on
> >>>>>>>>> ESX(i)).
> >>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
> >>>> storage
> >>>>>>>>> need
> >>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
> >>>> relationship
> >>>>>>>>>>> between
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary
> >>>> for
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is
> called
> >>>> to
> >>>>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework
> did
> >>>>>>>>> not
> >>>>>>>>>>>>> address
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
> >>>>>>>>> structure
> >>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing
> >> it
> >>>>>>>>> to do
> >>>>>>>>>>> so.
> >>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got
> ESX(i)
> >>>> in
> >>>>>>>>> my
> >>>>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
> >>>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach
> logic
> >>>>>>>>> (in the
> >>>>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to
> have
> >>>> it
> >>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the
> volume
> >>>> to
> >>>>>>>>> a
> >>>>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize
> it's
> >>>>>>>>> being
> >>>>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been
> created)
> >>>> and
> >>>>>>>>>>> create,
> >>>>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a
> datastore
> >>>>>>>>> (for
> >>>>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
> >>>> recently
> >>>>>>>>> making
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
> >>>>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
> >>>> driver
> >>>>>>>>> is
> >>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
> >>>> Tutkowski
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to
> support
> >>>>>>>>> all
> >>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of
> time
> >>>>>>>>> for 4.2
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
> >>>> XenServer.
> >>>>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have
> >> it
> >>>>>>>>> working
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all
> in
> >>>>>>>>> regards
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM,
> Kelcey
> >>>>>>>>> Jamison
> >>>>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM
> in
> >>>>>>>>> the
> >>>>>>>>>>> patch
> >>>>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32
> AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
> >> disk_io_throttling
> >>>>>>>>> to
> >>>>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature
> >> first,
> >>>>>>>>> then
> >>>>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then
> >> it
> >>>>>>>>> is a
> >>>>>>>>>>> non
> >>>>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei
> >>>> ZHOU
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
> >>>>>>>>> billing
> >>>>>>>>>>> now,
> >>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can
> >>>> get
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes
> >> for
> >>>>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation
> >>>> value
> >>>>>>>>> of
> >>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I
> >>>> can
> >>>>>>>>> work
> >>>>>>>>>>> on
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
> >>>>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the
> >> two
> >>>>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei
> ZHOU
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I
> am
> >>>>>>>>> trying
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend
> >> on
> >>>>>>>>> IO
> >>>>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
> >>>>>>>>> disk_io_stat if
> >>>>>>>>>>> you
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
> >>>>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers
> >> are
> >>>>>>>>> billed
> >>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for
> >>>> those
> >>>>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I
> >>>> would
> >>>>>>>>>>> imagine
> >>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost
> >> models.
> >>>>>>>>> Could
> >>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current
> patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime
> >> change?
> >>>>>>>>> KVM
> >>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
> >>>>>>>>> machine
> >>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support
> >> changing
> >>>>>>>>> the
> >>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
> >>>> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface
> level.
> >>>> I
> >>>>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
> >>>>>>>>> algorithms
> >>>>>>>>>>> with a
> >>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
> >>>>>>>>> provided by a
> >>>>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe
> it
> >>>> is
> >>>>>>>>>>> better
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs
> to
> >>>>>>>>> consider
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a
> >>>> later
> >>>>>>>>>>>>> response
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
> >>>>>>>>> features. I
> >>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
> >>>>>>>>> SolidFire, but
> >>>>>>>>>>> I
> >>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
> >>>>>>>>> (possibly
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate
> on
> >>>>>>>>> this
> >>>>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand
> >> are
> >>>>>>>>> the
> >>>>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing
> >> IOPS
> >>>>>>>>>>> policies
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for
> billing/usage
> >>>>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
> >>>>>>>>> interface model
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting
> provisioned
> >>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
> >>>>>>>>> reconciled?
> >>>>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed)
> >> for
> >>>>>>>>> more
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been
> configured
> >>>> to
> >>>>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration
> between a
> >>>>>>>>> VM and a
> >>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification
> to
> >>>>>>>>> storage
> >>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
> >>>>>>>>> Tutkowski
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals
> with
> >>>>>>>>>>>>> controlling
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
> >>>> controlling
> >>>>>>>>> IOPS
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM,
> >>>> John
> >>>>>>>>>>> Burwell
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
> >>>>>>>>> should be
> >>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and
> it
> >>>> is
> >>>>>>>>> ready
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete
> read/write
> >>>>>>>>> ops issue
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this
> criteria
> >>>>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >>>>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://reviews.apache.org/r/11479/
> >>>>>>>>> )?
> >>>>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned
> IOPS. I
> >>>>>>>>> would
> >>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS
> >> in
> >>>>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei
> >>>>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next
> week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
> >>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei
> ZHOU
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
> >>>> features
> >>>>>>>>> into
> >>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
> >>>> technical
> >>>>>>>>>>> reason
> >>>>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just
> >>>> want
> >>>>>>>>> to
> >>>>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap
> >> so
> >>>>>>>>> that we
> >>>>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the
> >>>> I/O
> >>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
> >>>>>>>>> legacy from
> >>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
> >>>> Burwell
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
> >>>>>>>>> discretely
> >>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed
> >>>>>>>>> later
> >>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
> >>>>>>>>> write.
> >>>>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided
> to
> >>>>>>>>> break
> >>>>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it
> >> into
> >>>>>>>>> master
> >>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write
> >>>> IOps
> >>>>>>>>>>> before
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and
> >>>> Write
> >>>>>>>>>>> instead
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
> >>>> feature
> >>>>>>>>>>> freeze
> >>>>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it
> won't
> >>>> be
> >>>>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that
> >> there
> >>>>>>>>> is no
> >>>>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM,
> Wido
> >>>>>>>>> den
> >>>>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei
> ZHOU
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the
> >> beginning.
> >>>>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
> >>>> mainly
> >>>>>>>>>>> because
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
> >>>>>>>>> received
> >>>>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
> >>>>>>>>> Networking and
> >>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive
> >> in
> >>>>>>>>> most
> >>>>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
> >>>>>>>>> copy-paste
> >>>>>>>>>>>>> work.
> >>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
> >>>>>>>>> feature
> >>>>>>>>>>>>> freeze,
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we
> can't
> >>>>>>>>> miss. But
> >>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we
> >>>> don't
> >>>>>>>>>>> release
> >>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
> >>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei
> ZHOU
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
> >>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge
> >>>> into
> >>>>>>>>> master
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on
> >>>> the
> >>>>>>>>> same
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the
> >> rate
> >>>>>>>>>>>>> limitation
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk
> operation,
> >>>> it
> >>>>>>>>> may
> >>>>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
> >>>>>>>>> storage
> >>>>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the
> >> maximum
> >>>>>>>>> rate
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you
> >> make
> >>>>>>>>> no
> >>>>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support
> setting
> >>>>>>>>> both a
> >>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot
> of
> >>>>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the
> polling
> >>>>>>>>> side you
> >>>>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling
> side
> >>>>>>>>> you only
> >>>>>>>>>>> go
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much
> more
> >>>>>>>>> expensive
> >>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where
> that
> >>>> an
> >>>>>>>>> admin
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at
> >>>> this
> >>>>>>>>>>> point I
> >>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
> >>>>>>>>> mechanism in
> >>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of
> >> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
> >>>>>>>>> setting
> >>>>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both
> up
> >>>>>>>>> and set
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
> >>>> implemented
> >>>>>>>>> it
> >>>>>>>>>>> this
> >>>>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs
> >>>> (in
> >>>>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of
> >> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total
> >> bps
> >>>>>>>>> and
> >>>>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> >>>>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's
> RAT
> >>>>>>>>> execution
> >>>>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
> >>>>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit
> >> and
> >>>>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to
> >>>>>>>>> check for
> >>>>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS
> >>>>>>>>> rate on
> >>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
> >>>>>>>>> destroy,
> >>>>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations,
> including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >>>> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>>>> SolidFire
> >>>>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
I would like the UIs of two features reviewed together to ensure
consistency across the concepts of hypervisor throttled IOPs and
storage device provisioned IOPs. I see the potential for confusion,
and I think a side-by-side Ui review of these features will help
minimize any potential confusion.
As I mentioned, the term reconciliation issue will work itself if it
is acceptable that a VM is only permitted utilize hypervisor throttled
IOPs or storage provisioned IOPs.
Thanks,
-John
On Jun 6, 2013, at 10:05 PM, Mike Tutkowski
<mi...@solidfire.com> wrote:
> Hi John,
>
> Yeah, when you get a chance, refer to the Google doc I sent to you the
> other day to see how the GUI looks for provisioned storage IOPS.
>
> Several months ago, I put this topic out on the e-mail list and we decided
> to place the Min, Max, and Burst IOPS in the Add Disk Offering dialog.
> Other storage vendors are coming out with QoS, so they should be able to
> leverage this GUI going forward (even if they, say, only use Max IOPS).
> These fields are optional and can be ignored for storage that does not
> support provisioned IOPS. Just like the Disk Size field, the admin can
> choose to allow the end user to fill in Min, Max, and Burst IOPS.
>
> I'm OK if we do an either/or model (either Wei's feature or mine, as is
> decided by the admin).
>
> I'm not sure what we can do about these two features expressing the speed
> in different terms. I've never seen a SAN express the IOPS for QoS in any
> way other than total IOPS (i.e. not broken in into read/write IOPS).
>
> Thanks
>
>
> On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com> wrote:
>
>> Wei,
>>
>> We have been down the rabbit hole a bit on the Storage/Hypervisor layer
>> separation, but we still need to reconcile the behavior of hypervisor
>> throttled I/O and storage provisioned IOPS. I see the following issues
>> outstanding:
>>
>> 1. Hypervisor throttled IOPS are expressed as discrete read/write values
>> where as storage provisioned IOPS are expressed as total IOPS.
>> 2. How do we handle VMs with throttled IOPS attached to storage volumes
>> with provisioned IOPS?
>> 3. How should usage data be captured for throttled and provisioned IOPS
>> that will permit providers to discriminate these guaranteed operations in
>> the event they want to bill for it differently?
>> 4. What is the user experience for throttled and provisioned IOPS that
>> minimizes confusion of these concepts?
>>
>> My thinking is that a VM can have either utilize hypervisor throttled IOPS
>> or storage provisioned IOPS. This policy would neatly solve items 1 and 2.
>> Since the two facilities would not be permitted to operate together, they
>> do not need to be semantically compatible. I think item 3 can be resolved
>> with an additional flag or two on the usage records. As for Item 4, I am
>> not familiar with how these two enhancements are depicted in the user
>> interface. I think we need to review the UI enhancements for both
>> enhancements and ensure they are consistent.
>>
>> Do these solutions make sense?
>>
>> Thanks,
>> -John
>>
>> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> John and Mike,
>>>
>>> I was busy working on other issues (CLOUDSTACK-2780/2729,
>>> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this week.
>>> I will start to develop on iops/bps changes tomorrow, and ask for second
>>> merge request after testing.
>>>
>>> -Wei
>>>
>>>
>>> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>>>
>>>> I believe I understand where you're going with this, John, and have been
>>>> re-working this section of code today.
>>>>
>>>> I should be able to run it by you tomorrow.
>>>>
>>>> Thanks for the comments,
>>>> Mike
>>>>
>>>>
>>>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> See my responses in-line below.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>
>>>>> Hi John,
>>>>>
>>>>> Thanks for the response.
>>>>>
>>>>> "I am fine with the VolumeManager determining whether or not a Volume
>> is
>>>>> managed (i.e. not based on the StoragePoolType, but an actual isManaged
>>>>> method), and asking the device driver to allocate resources for the
>>>> volume
>>>>> if it is managed."
>>>>>
>>>>> Are you thinking you'd like to see an isManaged() method added to the
>>>>> PrimaryDataStoreDriver interface? If it returns true, then the storage
>>>>> framework could call the manage() (or whatever name) method (which
>> would
>>>> be
>>>>> new to the PrimaryDataStoreDriver interface, as well) and this would
>> call
>>>>> into a new method in the hypervisor code to create, say on XenServer,
>> an
>>>> SR?
>>>>>
>>>>>
>>>>> I would like to see canBeManaged() : boolean on DataStoreDriver. Since
>>>>> the notion of Volumes only pertains to primary storage, I would add a
>>>>> allocateStorage and deallocateStorage (Storage is a straw man term --
>>>>> something other than volume) methods to
>> allocate/create/deallocate/delete
>>>>> underlying storage. To my mind, managed is a mutable property of
>>>> DataStore
>>>>> which can be enabled if/when the underlying DataStoreDriver can be
>>>> managed.
>>>>> This approach allows operators to override manageability of devices.
>>>>>
>>>>> In terms of orchestration/process flow for SR, the Xen plugin would be
>>>>> responsible for composing DataStore/Volume methods to create any
>>>>> directories or files necessary for the SR. There should be no
>>>> dependencies
>>>>> from the Storage to the Hypervisor layer. As I said earlier, such
>>>> circular
>>>>> dependencies will lead to a tangled, unmaintainable mess.
>>>>>
>>>>>
>>>>> Just want to make sure I'm on the same page with you.
>>>>>
>>>>> Thanks again, John
>>>>>
>>>>>
>>>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com>
>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> Fundamentally, we can't end up with a Storage layer that supports n
>>>>>> devices types with each specific behaviors of m hypervisors. Such a
>>>>>> scenario will create an unmaintainable and untestable beast.
>>>> Therefore, my
>>>>>> thoughts and recommendations are driven to evolve the Storage layer
>>>> towards
>>>>>> this separation.
>>>>>>
>>>>>> I am fine with the VolumeManager determining whether or not a Volume
>> is
>>>>>> managed (i.e. not based on the StoragePoolType, but an actual
>> isManaged
>>>>>> method), and asking the device driver to allocate resources for the
>>>> volume
>>>>>> if it is managed. Furthermore, the device driver needs to indicate
>>>> whether
>>>>>> or not it supports management operations. Finally, I think we need to
>>>>>> provide the ability for an administrator to elect to have something
>>>> that is
>>>>>> manageable be unmanaged (i.e. the driver is capable managing the
>> device,
>>>>>> but the administrator has elected to leave it unmanaged).
>>>>>>
>>>>>> Creation of a structure on the volume should be done in the Xen
>>>>>> hypervisor module using methods exposed by the Storage layer to
>> perform
>>>>>> low-level operations (e.g. make directories, create a file, etc).
>> This
>>>>>> structure is specific to the operation of the Xen hypervisor, as such,
>>>>>> should be confined to its implementation. From my perspective,
>> nothing
>>>> in
>>>>>> the Storage layer should be concerned with content. From its
>>>> perspective,
>>>>>> structure and data are opaque. It provides the means to query the
>> data
>>>> to
>>>>>> support the interpretation of the content by higher-level layers (e.g.
>>>>>> Hypervisors). To my mind, attach should be a composition of
>> operations
>>>>>> from the Storage layer that varies based on the Volume storage
>> protocol
>>>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> Alternatively to the way the attach logic is implemented in my patch,
>> we
>>>>>> could do the following:
>>>>>>
>>>>>> Leave the attach logic in the agent code alone. In VolumeManagerImpl
>> we
>>>>>> create an AttachVolumeCommand and send it to the hypervisor. Before
>> this
>>>>>> command is sent, we could check to see if we're dealing with Dynamic
>> (or
>>>>>> whatever we want to call it) storage and - if so - send a "Create SR"
>>>>>> command to the hypervisor. If this returns OK, we would then proceed
>> to
>>>> the
>>>>>> AttachVolumeCommand, as usual.
>>>>>>
>>>>>> This way the attach logic remains the same and we just add another
>>>>>> command to the agent code that is called for this particular type of
>>>>>> storage.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> Hey John,
>>>>>>>
>>>>>>> I created a document for a customer today that outlines how the
>> plug-in
>>>>>>> works from a user standpoint. This will probably be of use to you, as
>>>> well,
>>>>>>> as you perform the code review.
>>>>>>>
>>>>>>> I have shared this document with you (you should have received that
>>>>>>> information in a separate e-mail).
>>>>>>>
>>>>>>> Talk to you later!
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> Oh, OK, that sounds really good, John.
>>>>>>>>
>>>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jburwell@basho.com
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> I am never at a loss for an opinion. I some thoughts, but want to
>>>>>>>>> confirm assumptions and ideas against the solidfire,
>>>> disk_io_throttle,
>>>>>>>>> and
>>>>>>>>> object_store branches. I hope to collect them in a coherent form
>>>>>>>>> tomorrow (5 June 2013).
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> "So, in essence, the SolidFire plugin introduces the notion of a
>>>>>>>>> managed
>>>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>>>
>>>>>>>>>> Technically, the SolidFire plug-in just introduces the notion of
>>>>>>>>>> provisioned storage IOPS.
>>>>>>>>>>
>>>>>>>>>> The storage framework that leverages the plug-in was incomplete,
>> so
>>>>>>>>> I had
>>>>>>>>>> to try to add in the notion of a managed iSCSI device.
>>>>>>>>>>
>>>>>>>>>> I appreciate all the time you've been spending on this. :) Do you
>>>>>>>>> have a
>>>>>>>>>> recommendation as to how we should accomplish what you're looking
>>>>>>>>> for?
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> So, in essence, the SolidFire plugin introduces the notion of a
>>>>>>>>> managed
>>>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>>>
>>>>>>>>>>> I want to see a separation of the management capabilities (i.e.
>>>> can
>>>>>>>>> a
>>>>>>>>>>> device be managed/does an operator want it managed by CloudStack)
>>>>>>>>> from the
>>>>>>>>>>> storage protocol. Ideally, we should end up with a semantic that
>>>>>>>>> will
>>>>>>>>>>> allow any type of storage device to be managed. I also want to
>>>> make
>>>>>>>>>>> progress on decoupling the storage types from the hypervisor
>>>>>>>>> definitions.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>
>>>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
>> jburwell@basho.com
>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Could you please answer the following questions for me with
>>>>>>>>> regards to
>>>>>>>>>>> the
>>>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>>>
>>>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
>>>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>>>
>>>>>>>>>>>>> What is the cardinality between SAN Volumes and CloudStack
>>>>>>>>> Volumes?
>>>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>>>
>>>>>>>>>>>>> Are the LUN(s) created by the management server or externally
>> by
>>>>>>>>> the
>>>>>>>>>>>>> operator?
>>>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume (same as a
>> SAN
>>>>>>>>> LUN) is
>>>>>>>>>>>> created by the management server (via the plug-in) the first
>> time
>>>>>>>>> the
>>>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>>>>>>>>>>>>
>>>>>>>>>>>> If you don't want to use the SolidFire plug-in, but still want
>> to
>>>>>>>>> use a
>>>>>>>>>>>> SolidFire volume (LUN), you can do this already today (prior to
>>>>>>>>> 4.2). The
>>>>>>>>>>>> admin manually creates the SAN volume and - in this case -
>>>>>>>>> multiple VMs
>>>>>>>>>>> and
>>>>>>>>>>>> data disks can share this SAN volume. While you can do this
>>>> today,
>>>>>>>>> it is
>>>>>>>>>>>> not useful if you want to enforce storage QoS.
>>>>>>>>>>>>
>>>>>>>>>>>>> Are the SAN volumes by the management server or externally by
>>>> the
>>>>>>>>>>> operator?
>>>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes are
>>>> completely
>>>>>>>>>>> managed
>>>>>>>>>>>> by the management server (via the plug-in). There is no admin
>>>>>>>>>>> interaction.
>>>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume and a
>>>> CloudStack
>>>>>>>>>>> volume,
>>>>>>>>>>>> which is necessary for any storage vendor that supports true
>> QoS.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would like to clarify how these pieces are related and
>>>> expected
>>>>>>>>> to
>>>>>>>>>>>>> operate.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs with
>>>>>>>>> provisioned
>>>>>>>>>>> IOPS
>>>>>>>>>>>>>> won't be cut off by the underlying storage."
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In the storage QoS world, we need to map a single SAN volume
>>>>>>>>> (LUN) to a
>>>>>>>>>>>>>> single CloudStack volume. We cannot have multiple CloudStack
>>>>>>>>> volumes
>>>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If the user wants to have a single SAN volume house more than
>>>> one
>>>>>>>>>>>>>> CloudStack volume, then can do that today without any of my
>>>>>>>>> plug-in
>>>>>>>>>>> code.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> "The administrator will allocate a SAN volume for
>> CloudStack's
>>>>>>>>> use
>>>>>>>>>>> onto
>>>>>>>>>>>>>>> which CloudStack volumes will be created."
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You are coming to the part which concerns me -- concepts
>> from
>>>>>>>>> the
>>>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only used
>> (as
>>>>>>>>> far as
>>>>>>>>>>> I
>>>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from VMware, it uses
>>>> the
>>>>>>>>> vmfs
>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>>>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here was to make a
>>>>>>>>> new type
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison added into
>>>> the
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> framework. Maybe it should be more specificy, though:
>>>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or care that it
>>>> is
>>>>>>>>> being
>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer questions about
>> it
>>>>>>>>> is
>>>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I see...so your concern here is that the SolidFire plug-in
>>>>>>>>> needs to
>>>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic
>> knows
>>>>>>>>> to
>>>>>>>>>>> treat
>>>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm totally open to removing that constraint and just
>>>>>>>>> calling it
>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> whatever. We would just need a way for the hypervisor
>>>> attach
>>>>>>>>> logic
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> detect this new requirement and perform the necessary
>>>>>>>>> activities.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to manage a
>>>> device,
>>>>>>>>> an
>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
>>>> device
>>>>>>>>> is
>>>>>>>>>>>>> shared
>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>> many services, and the operator has chosen to dedicate
>>>>>>>>> only a
>>>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
>>>>>>>>> plug-in
>>>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would work unless the
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> My thinking is that, independent of CloudStack, an
>>>> operator
>>>>>>>>>>>>> allocates
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a
>>>>>>>>> LUN. They
>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>> want to turn control of that LUN over to CloudStack, but
>>>>>>>>> not allow
>>>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my changes, we would
>>>> be
>>>>>>>>>>> passing
>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>>> SAN volume that does not have the necessary hypervisor
>>>>>>>>> support
>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the storage
>>>> framework
>>>>>>>>>>> itself
>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from the hypervisor
>>>>>>>>> side and
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> call into the agent code specifically to create the SR
>>>>>>>>> before the
>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I think the hypervisor management plugin should have a
>>>> rich
>>>>>>>>> enough
>>>>>>>>>>>>>>>>>>> interface to storage to determine available for volume
>>>>>>>>> storage.
>>>>>>>>>>> For
>>>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>>>> this interface would allow the interrogation of the
>> device
>>>>>>>>> to
>>>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>>>> SR is present. The storage device itself shouldn't know
>>>>>>>>> or care
>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>>>>>>>>> questions
>>>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver (which is a
>>>>>>>>> type of
>>>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
>>>>>>>>> Storage shows
>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that references the
>>>>>>>>> storage
>>>>>>>>>>>>> tag
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack volume. This leads
>>>> to
>>>>>>>>> a new
>>>>>>>>>>>>> row
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
>>>>>>>>> (attach
>>>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the plug-in to
>>>>>>>>> create a
>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in
>>>>>>>>> also
>>>>>>>>>>>>> updates
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of
>>>>>>>>> the SAN
>>>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
>>>>>>>>> volume is in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be
>>>>>>>>> in the
>>>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the plug-in code
>>>>>>>>> will not
>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor
>>>> data
>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume,
>>>>>>>>> in these
>>>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical allocation on the
>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>>>>>>>>> CloudStack, I
>>>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What you say here may very well make sense, but I'm
>>>>>>>>> having a
>>>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation
>>>>>>>>> as he
>>>>>>>>>>> was
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I took.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two distinct concepts
>>>> --
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In both cases, we
>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
>>>>>>>>> volumes
>>>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>> device while the other requires the volume to be
>>>>>>>>> volume to be
>>>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that
>> we
>>>>>>>>> are in
>>>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to
>>>>>>>>> the
>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we
>>>> must
>>>>>>>>> be
>>>>>>>>>>> able
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to move bits from
>>>>>>>>> one place
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage
>>>>>>>>> device
>>>>>>>>>>> may
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
>>>>>>>>> unmanaged
>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
>>>> managed
>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable of allocating
>>>>>>>>> LUNs and
>>>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
>>>>>>>>> manage a
>>>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
>>>> CloudStack
>>>>>>>>> (e.g.
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the operator has chosen
>>>> to
>>>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning
>> make
>>>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from StoragePoolType,
>>>>>>>>> add the
>>>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver that
>> indicates
>>>>>>>>> whether
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>>>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is associated
>>>>>>>>> with a
>>>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
>>>> declare
>>>>>>>>> which
>>>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a device managed by
>>>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>>>> management operations exposed by the driver to
>>>> automate
>>>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit
>>>>>>>>> down and
>>>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the current iSCSI
>>>> type
>>>>>>>>> and
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
>>>>>>>>> Primary
>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
>>>>>>>>> volume must
>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
>>>>>>>>> CloudStack
>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
>>>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
>>>>>>>>> Primary
>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
>>>>>>>>> storage
>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
>>>>>>>>> framework). When
>>>>>>>>>>> a
>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>>>>>>>>> framework
>>>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done,
>>>>>>>>> the
>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor
>>>>>>>>> data
>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was just created.
>>>> The
>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should probably have
>>>>>>>>> defined them
>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex.
>>>>>>>>> an admin
>>>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure
>>>> is
>>>>>>>>>>> created
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
>>>> hypervisor
>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
>>>> placed
>>>>>>>>> on
>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
>>>> attach
>>>>>>>>> logic
>>>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already in place and
>>>>>>>>> simply
>>>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
>> pre-allocated.
>>>>>>>>> Instead
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
>>>>>>>>> volume on
>>>>>>>>>>> a
>>>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
>>>>>>>>> created
>>>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these types of volumes
>>>>>>>>> have not
>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an
>> admin.
>>>>>>>>> Once
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume,
>>>> it
>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
>>>>>>>>> attach
>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>>>>>>>>> difference.
>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely
>> used
>>>>>>>>> by
>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly
>>>>>>>>> based on
>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
>>>>>>>>> XenServer.
>>>>>>>>>>> For
>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor
>>>>>>>>> and
>>>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the
>> difference
>>>>>>>>> between
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
>>>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a
>> clusterfs
>>>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool
>>>>>>>>> is set
>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage
>>>> (ex.
>>>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage
>> company
>>>>>>>>> wants
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>>>>>>>>> feature. It
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need
>>>> only
>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this
>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
>>>>>>>>> framework
>>>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that
>> supported
>>>>>>>>> the
>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
>>>>>>>>> visited with
>>>>>>>>>>>>> him
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
>>>>>>>>> complete. We
>>>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
>>>> volumes
>>>>>>>>> unless
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of
>> storage
>>>>>>>>> and
>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any
>> operation
>>>>>>>>> in the
>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect
>>>>>>>>> to see
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>>>>>>>>>>> composed/driven
>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>>>>>>>>> structures
>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going
>>>> to
>>>>>>>>> end
>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
>>>> increasingly
>>>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
>>>>>>>>> specific
>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
>>>> method
>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
>>>>>>>>> system,
>>>>>>>>>>> then
>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of
>>>> the
>>>>>>>>>>> volume
>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
>>>>>>>>> ESX(i)).
>>>>>>>>>>>>> VMs
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
>>>> storage
>>>>>>>>> need
>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
>>>> relationship
>>>>>>>>>>> between
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary
>>>> for
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called
>>>> to
>>>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did
>>>>>>>>> not
>>>>>>>>>>>>> address
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
>>>>>>>>> structure
>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing
>> it
>>>>>>>>> to do
>>>>>>>>>>> so.
>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i)
>>>> in
>>>>>>>>> my
>>>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
>>>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic
>>>>>>>>> (in the
>>>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have
>>>> it
>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume
>>>> to
>>>>>>>>> a
>>>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's
>>>>>>>>> being
>>>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created)
>>>> and
>>>>>>>>>>> create,
>>>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore
>>>>>>>>> (for
>>>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
>>>> recently
>>>>>>>>> making
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
>>>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
>>>> driver
>>>>>>>>> is
>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
>>>> Tutkowski
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support
>>>>>>>>> all
>>>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time
>>>>>>>>> for 4.2
>>>>>>>>>>> to
>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
>>>> XenServer.
>>>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have
>> it
>>>>>>>>> working
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
>>>>>>>>> regards
>>>>>>>>>>> to
>>>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
>>>>>>>>> Jamison
>>>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in
>>>>>>>>> the
>>>>>>>>>>> patch
>>>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
>> disk_io_throttling
>>>>>>>>> to
>>>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature
>> first,
>>>>>>>>> then
>>>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then
>> it
>>>>>>>>> is a
>>>>>>>>>>> non
>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei
>>>> ZHOU
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
>>>>>>>>> billing
>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can
>>>> get
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes
>> for
>>>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation
>>>> value
>>>>>>>>> of
>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I
>>>> can
>>>>>>>>> work
>>>>>>>>>>> on
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>>>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the
>> two
>>>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
>>>>>>>>> trying
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend
>> on
>>>>>>>>> IO
>>>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
>>>>>>>>> disk_io_stat if
>>>>>>>>>>> you
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>>>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers
>> are
>>>>>>>>> billed
>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for
>>>> those
>>>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I
>>>> would
>>>>>>>>>>> imagine
>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost
>> models.
>>>>>>>>> Could
>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime
>> change?
>>>>>>>>> KVM
>>>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
>>>>>>>>> machine
>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support
>> changing
>>>>>>>>> the
>>>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
>>>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level.
>>>> I
>>>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
>>>>>>>>> algorithms
>>>>>>>>>>> with a
>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
>>>>>>>>> provided by a
>>>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it
>>>> is
>>>>>>>>>>> better
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
>>>>>>>>> consider
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a
>>>> later
>>>>>>>>>>>>> response
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
>>>>>>>>> features. I
>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
>>>>>>>>> SolidFire, but
>>>>>>>>>>> I
>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>>>>>>>>> (possibly
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on
>>>>>>>>> this
>>>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand
>> are
>>>>>>>>> the
>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing
>> IOPS
>>>>>>>>>>> policies
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>>>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
>>>>>>>>> interface model
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned
>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>>>>>>>>> reconciled?
>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed)
>> for
>>>>>>>>> more
>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured
>>>> to
>>>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a
>>>>>>>>> VM and a
>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
>>>>>>>>> storage
>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
>>>>>>>>> Tutkowski
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>>>>>>>>>>>>> controlling
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
>>>> controlling
>>>>>>>>> IOPS
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM,
>>>> John
>>>>>>>>>>> Burwell
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
>>>>>>>>> should be
>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it
>>>> is
>>>>>>>>> ready
>>>>>>>>>>> to
>>>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write
>>>>>>>>> ops issue
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria
>>>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>>>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://reviews.apache.org/r/11479/
>>>>>>>>> )?
>>>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I
>>>>>>>>> would
>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS
>> in
>>>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei
>>>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
>>>> features
>>>>>>>>> into
>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
>>>> technical
>>>>>>>>>>> reason
>>>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just
>>>> want
>>>>>>>>> to
>>>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap
>> so
>>>>>>>>> that we
>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the
>>>> I/O
>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
>>>>>>>>> legacy from
>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
>>>> Burwell
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>>>>>>>>> discretely
>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed
>>>>>>>>> later
>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
>>>>>>>>> write.
>>>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to
>>>>>>>>> break
>>>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it
>> into
>>>>>>>>> master
>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write
>>>> IOps
>>>>>>>>>>> before
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and
>>>> Write
>>>>>>>>>>> instead
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
>>>> feature
>>>>>>>>>>> freeze
>>>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't
>>>> be
>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that
>> there
>>>>>>>>> is no
>>>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido
>>>>>>>>> den
>>>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the
>> beginning.
>>>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
>>>> mainly
>>>>>>>>>>> because
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
>>>>>>>>> received
>>>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
>>>>>>>>> Networking and
>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive
>> in
>>>>>>>>> most
>>>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
>>>>>>>>> copy-paste
>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
>>>>>>>>> feature
>>>>>>>>>>>>> freeze,
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't
>>>>>>>>> miss. But
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we
>>>> don't
>>>>>>>>>>> release
>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge
>>>> into
>>>>>>>>> master
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on
>>>> the
>>>>>>>>> same
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the
>> rate
>>>>>>>>>>>>> limitation
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation,
>>>> it
>>>>>>>>> may
>>>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
>>>>>>>>> storage
>>>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the
>> maximum
>>>>>>>>> rate
>>>>>>>>>>> and
>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you
>> make
>>>>>>>>> no
>>>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting
>>>>>>>>> both a
>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling
>>>>>>>>> side you
>>>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side
>>>>>>>>> you only
>>>>>>>>>>> go
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
>>>>>>>>> expensive
>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that
>>>> an
>>>>>>>>> admin
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at
>>>> this
>>>>>>>>>>> point I
>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
>>>>>>>>> mechanism in
>>>>>>>>>>>>> place
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of
>> work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
>>>>>>>>> setting
>>>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up
>>>>>>>>> and set
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
>>>> implemented
>>>>>>>>> it
>>>>>>>>>>> this
>>>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs
>>>> (in
>>>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of
>> VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total
>> bps
>>>>>>>>> and
>>>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>>>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit
>> and
>>>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to
>>>>>>>>> check for
>>>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS
>>>>>>>>> rate on
>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
>>>>>>>>> destroy,
>>>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>>>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>>>> SolidFire
>>>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
Yeah, when you get a chance, refer to the Google doc I sent to you the
other day to see how the GUI looks for provisioned storage IOPS.
Several months ago, I put this topic out on the e-mail list and we decided
to place the Min, Max, and Burst IOPS in the Add Disk Offering dialog.
Other storage vendors are coming out with QoS, so they should be able to
leverage this GUI going forward (even if they, say, only use Max IOPS).
These fields are optional and can be ignored for storage that does not
support provisioned IOPS. Just like the Disk Size field, the admin can
choose to allow the end user to fill in Min, Max, and Burst IOPS.
I'm OK if we do an either/or model (either Wei's feature or mine, as is
decided by the admin).
I'm not sure what we can do about these two features expressing the speed
in different terms. I've never seen a SAN express the IOPS for QoS in any
way other than total IOPS (i.e. not broken in into read/write IOPS).
Thanks
On Thu, Jun 6, 2013 at 7:16 PM, John Burwell <jb...@basho.com> wrote:
> Wei,
>
> We have been down the rabbit hole a bit on the Storage/Hypervisor layer
> separation, but we still need to reconcile the behavior of hypervisor
> throttled I/O and storage provisioned IOPS. I see the following issues
> outstanding:
>
> 1. Hypervisor throttled IOPS are expressed as discrete read/write values
> where as storage provisioned IOPS are expressed as total IOPS.
> 2. How do we handle VMs with throttled IOPS attached to storage volumes
> with provisioned IOPS?
> 3. How should usage data be captured for throttled and provisioned IOPS
> that will permit providers to discriminate these guaranteed operations in
> the event they want to bill for it differently?
> 4. What is the user experience for throttled and provisioned IOPS that
> minimizes confusion of these concepts?
>
> My thinking is that a VM can have either utilize hypervisor throttled IOPS
> or storage provisioned IOPS. This policy would neatly solve items 1 and 2.
> Since the two facilities would not be permitted to operate together, they
> do not need to be semantically compatible. I think item 3 can be resolved
> with an additional flag or two on the usage records. As for Item 4, I am
> not familiar with how these two enhancements are depicted in the user
> interface. I think we need to review the UI enhancements for both
> enhancements and ensure they are consistent.
>
> Do these solutions make sense?
>
> Thanks,
> -John
>
> On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com> wrote:
>
> > John and Mike,
> >
> > I was busy working on other issues (CLOUDSTACK-2780/2729,
> > CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this week.
> > I will start to develop on iops/bps changes tomorrow, and ask for second
> > merge request after testing.
> >
> > -Wei
> >
> >
> > 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> >
> >> I believe I understand where you're going with this, John, and have been
> >> re-working this section of code today.
> >>
> >> I should be able to run it by you tomorrow.
> >>
> >> Thanks for the comments,
> >> Mike
> >>
> >>
> >> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Mike,
> >>>
> >>> See my responses in-line below.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>> wrote:
> >>>
> >>> Hi John,
> >>>
> >>> Thanks for the response.
> >>>
> >>> "I am fine with the VolumeManager determining whether or not a Volume
> is
> >>> managed (i.e. not based on the StoragePoolType, but an actual isManaged
> >>> method), and asking the device driver to allocate resources for the
> >> volume
> >>> if it is managed."
> >>>
> >>> Are you thinking you'd like to see an isManaged() method added to the
> >>> PrimaryDataStoreDriver interface? If it returns true, then the storage
> >>> framework could call the manage() (or whatever name) method (which
> would
> >> be
> >>> new to the PrimaryDataStoreDriver interface, as well) and this would
> call
> >>> into a new method in the hypervisor code to create, say on XenServer,
> an
> >> SR?
> >>>
> >>>
> >>> I would like to see canBeManaged() : boolean on DataStoreDriver. Since
> >>> the notion of Volumes only pertains to primary storage, I would add a
> >>> allocateStorage and deallocateStorage (Storage is a straw man term --
> >>> something other than volume) methods to
> allocate/create/deallocate/delete
> >>> underlying storage. To my mind, managed is a mutable property of
> >> DataStore
> >>> which can be enabled if/when the underlying DataStoreDriver can be
> >> managed.
> >>> This approach allows operators to override manageability of devices.
> >>>
> >>> In terms of orchestration/process flow for SR, the Xen plugin would be
> >>> responsible for composing DataStore/Volume methods to create any
> >>> directories or files necessary for the SR. There should be no
> >> dependencies
> >>> from the Storage to the Hypervisor layer. As I said earlier, such
> >> circular
> >>> dependencies will lead to a tangled, unmaintainable mess.
> >>>
> >>>
> >>> Just want to make sure I'm on the same page with you.
> >>>
> >>> Thanks again, John
> >>>
> >>>
> >>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com>
> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Fundamentally, we can't end up with a Storage layer that supports n
> >>>> devices types with each specific behaviors of m hypervisors. Such a
> >>>> scenario will create an unmaintainable and untestable beast.
> >> Therefore, my
> >>>> thoughts and recommendations are driven to evolve the Storage layer
> >> towards
> >>>> this separation.
> >>>>
> >>>> I am fine with the VolumeManager determining whether or not a Volume
> is
> >>>> managed (i.e. not based on the StoragePoolType, but an actual
> isManaged
> >>>> method), and asking the device driver to allocate resources for the
> >> volume
> >>>> if it is managed. Furthermore, the device driver needs to indicate
> >> whether
> >>>> or not it supports management operations. Finally, I think we need to
> >>>> provide the ability for an administrator to elect to have something
> >> that is
> >>>> manageable be unmanaged (i.e. the driver is capable managing the
> device,
> >>>> but the administrator has elected to leave it unmanaged).
> >>>>
> >>>> Creation of a structure on the volume should be done in the Xen
> >>>> hypervisor module using methods exposed by the Storage layer to
> perform
> >>>> low-level operations (e.g. make directories, create a file, etc).
> This
> >>>> structure is specific to the operation of the Xen hypervisor, as such,
> >>>> should be confined to its implementation. From my perspective,
> nothing
> >> in
> >>>> the Storage layer should be concerned with content. From its
> >> perspective,
> >>>> structure and data are opaque. It provides the means to query the
> data
> >> to
> >>>> support the interpretation of the content by higher-level layers (e.g.
> >>>> Hypervisors). To my mind, attach should be a composition of
> operations
> >>>> from the Storage layer that varies based on the Volume storage
> protocol
> >>>> (iSCSI, local file system, NFS, RBD, etc).
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>> Hi John,
> >>>>
> >>>> Alternatively to the way the attach logic is implemented in my patch,
> we
> >>>> could do the following:
> >>>>
> >>>> Leave the attach logic in the agent code alone. In VolumeManagerImpl
> we
> >>>> create an AttachVolumeCommand and send it to the hypervisor. Before
> this
> >>>> command is sent, we could check to see if we're dealing with Dynamic
> (or
> >>>> whatever we want to call it) storage and - if so - send a "Create SR"
> >>>> command to the hypervisor. If this returns OK, we would then proceed
> to
> >> the
> >>>> AttachVolumeCommand, as usual.
> >>>>
> >>>> This way the attach logic remains the same and we just add another
> >>>> command to the agent code that is called for this particular type of
> >>>> storage.
> >>>>
> >>>> What do you think?
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> Hey John,
> >>>>>
> >>>>> I created a document for a customer today that outlines how the
> plug-in
> >>>>> works from a user standpoint. This will probably be of use to you, as
> >> well,
> >>>>> as you perform the code review.
> >>>>>
> >>>>> I have shared this document with you (you should have received that
> >>>>> information in a separate e-mail).
> >>>>>
> >>>>> Talk to you later!
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>>> Oh, OK, that sounds really good, John.
> >>>>>>
> >>>>>> Thanks and talk to you tomorrow! :)
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jburwell@basho.com
> >>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> I am never at a loss for an opinion. I some thoughts, but want to
> >>>>>>> confirm assumptions and ideas against the solidfire,
> >> disk_io_throttle,
> >>>>>>> and
> >>>>>>> object_store branches. I hope to collect them in a coherent form
> >>>>>>> tomorrow (5 June 2013).
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>>
> >>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> "So, in essence, the SolidFire plugin introduces the notion of a
> >>>>>>> managed
> >>>>>>>> iSCSI device and provisioned IOPS."
> >>>>>>>>
> >>>>>>>> Technically, the SolidFire plug-in just introduces the notion of
> >>>>>>>> provisioned storage IOPS.
> >>>>>>>>
> >>>>>>>> The storage framework that leverages the plug-in was incomplete,
> so
> >>>>>>> I had
> >>>>>>>> to try to add in the notion of a managed iSCSI device.
> >>>>>>>>
> >>>>>>>> I appreciate all the time you've been spending on this. :) Do you
> >>>>>>> have a
> >>>>>>>> recommendation as to how we should accomplish what you're looking
> >>>>>>> for?
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> So, in essence, the SolidFire plugin introduces the notion of a
> >>>>>>> managed
> >>>>>>>>> iSCSI device and provisioned IOPS.
> >>>>>>>>>
> >>>>>>>>> I want to see a separation of the management capabilities (i.e.
> >> can
> >>>>>>> a
> >>>>>>>>> device be managed/does an operator want it managed by CloudStack)
> >>>>>>> from the
> >>>>>>>>> storage protocol. Ideally, we should end up with a semantic that
> >>>>>>> will
> >>>>>>>>> allow any type of storage device to be managed. I also want to
> >> make
> >>>>>>>>> progress on decoupling the storage types from the hypervisor
> >>>>>>> definitions.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi John,
> >>>>>>>>>>
> >>>>>>>>>> No problem. Answers are below in red.
> >>>>>>>>>>
> >>>>>>>>>> Thanks!
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <
> jburwell@basho.com
> >>>
> >>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> Could you please answer the following questions for me with
> >>>>>>> regards to
> >>>>>>>>> the
> >>>>>>>>>>> operation of the SolidFire plugin:
> >>>>>>>>>>>
> >>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN volumes?
> >>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
> >>>>>>>>>> 1 SAN volume : 1 LUN
> >>>>>>>>>>
> >>>>>>>>>>> What is the cardinality between SAN Volumes and CloudStack
> >>>>>>> Volumes?
> >>>>>>>>>> 1 SAN volume : 1 CloudStack volume
> >>>>>>>>>>
> >>>>>>>>>>> Are the LUN(s) created by the management server or externally
> by
> >>>>>>> the
> >>>>>>>>>>> operator?
> >>>>>>>>>> When used with the SolidFire plug-in, a SAN volume (same as a
> SAN
> >>>>>>> LUN) is
> >>>>>>>>>> created by the management server (via the plug-in) the first
> time
> >>>>>>> the
> >>>>>>>>>> CloudStack volume is attached to a hypervisor.
> >>>>>>>>>>
> >>>>>>>>>> If you don't want to use the SolidFire plug-in, but still want
> to
> >>>>>>> use a
> >>>>>>>>>> SolidFire volume (LUN), you can do this already today (prior to
> >>>>>>> 4.2). The
> >>>>>>>>>> admin manually creates the SAN volume and - in this case -
> >>>>>>> multiple VMs
> >>>>>>>>> and
> >>>>>>>>>> data disks can share this SAN volume. While you can do this
> >> today,
> >>>>>>> it is
> >>>>>>>>>> not useful if you want to enforce storage QoS.
> >>>>>>>>>>
> >>>>>>>>>>> Are the SAN volumes by the management server or externally by
> >> the
> >>>>>>>>> operator?
> >>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes are
> >> completely
> >>>>>>>>> managed
> >>>>>>>>>> by the management server (via the plug-in). There is no admin
> >>>>>>>>> interaction.
> >>>>>>>>>> This allows for a 1:1 mapping between a SAN volume and a
> >> CloudStack
> >>>>>>>>> volume,
> >>>>>>>>>> which is necessary for any storage vendor that supports true
> QoS.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I would like to clarify how these pieces are related and
> >> expected
> >>>>>>> to
> >>>>>>>>>>> operate.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> "In particular, how do we ensure that multiple VMs with
> >>>>>>> provisioned
> >>>>>>>>> IOPS
> >>>>>>>>>>>> won't be cut off by the underlying storage."
> >>>>>>>>>>>>
> >>>>>>>>>>>> In the storage QoS world, we need to map a single SAN volume
> >>>>>>> (LUN) to a
> >>>>>>>>>>>> single CloudStack volume. We cannot have multiple CloudStack
> >>>>>>> volumes
> >>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If the user wants to have a single SAN volume house more than
> >> one
> >>>>>>>>>>>> CloudStack volume, then can do that today without any of my
> >>>>>>> plug-in
> >>>>>>>>> code.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> "The administrator will allocate a SAN volume for
> CloudStack's
> >>>>>>> use
> >>>>>>>>> onto
> >>>>>>>>>>>>> which CloudStack volumes will be created."
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think we crossed e-mails. :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Check out my recent e-mail on this.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> >>>>>>> jburwell@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> You are coming to the part which concerns me -- concepts
> from
> >>>>>>> the
> >>>>>>>>>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only used
> (as
> >>>>>>> far as
> >>>>>>>>> I
> >>>>>>>>>>>>>> know)
> >>>>>>>>>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If you want to use your iSCSI volume from VMware, it uses
> >> the
> >>>>>>> vmfs
> >>>>>>>>>>> type.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
> >>>>>>>>>>>>>> SharedMountPoint
> >>>>>>>>>>>>>>> type.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here was to make a
> >>>>>>> new type
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>> storage to describe this dynamic ability Edison added into
> >> the
> >>>>>>>>> storage
> >>>>>>>>>>>>>>> framework. Maybe it should be more specificy, though:
> >>>>>>> Dynamic_iSCSI
> >>>>>>>>>>>>>> versus,
> >>>>>>>>>>>>>>> say, Dynamic_FC.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> "The storage device itself shouldn't know or care that it
> >> is
> >>>>>>> being
> >>>>>>>>>>> used
> >>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer questions about
> it
> >>>>>>> is
> >>>>>>>>>>>>>> storing."
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I see...so your concern here is that the SolidFire plug-in
> >>>>>>> needs to
> >>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic
> knows
> >>>>>>> to
> >>>>>>>>> treat
> >>>>>>>>>>>>>> it as
> >>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm totally open to removing that constraint and just
> >>>>>>> calling it
> >>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>> whatever. We would just need a way for the hypervisor
> >> attach
> >>>>>>> logic
> >>>>>>>>> to
> >>>>>>>>>>>>>>>> detect this new requirement and perform the necessary
> >>>>>>> activities.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
> >>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> See my responses in-line.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to manage a
> >> device,
> >>>>>>> an
> >>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>> may
> >>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
> >> device
> >>>>>>> is
> >>>>>>>>>>> shared
> >>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>> many services, and the operator has chosen to dedicate
> >>>>>>> only a
> >>>>>>>>>>> portion
> >>>>>>>>>>>>>>>>> of it
> >>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
> >>>>>>> plug-in
> >>>>>>>>>>>>>> (before
> >>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>> attach request to the hypervisor) would work unless the
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> consumes
> >>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> My thinking is that, independent of CloudStack, an
> >> operator
> >>>>>>>>>>> allocates
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a
> >>>>>>> LUN. They
> >>>>>>>>>>>>>> simply
> >>>>>>>>>>>>>>>>> want to turn control of that LUN over to CloudStack, but
> >>>>>>> not allow
> >>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> As the attach logic stands prior to my changes, we would
> >> be
> >>>>>>>>> passing
> >>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>> SAN volume that does not have the necessary hypervisor
> >>>>>>> support
> >>>>>>>>>>> (like
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>> SR)
> >>>>>>>>>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the storage
> >> framework
> >>>>>>>>> itself
> >>>>>>>>>>>>>>>>> detect
> >>>>>>>>>>>>>>>>>> that such a SAN volume needs support from the hypervisor
> >>>>>>> side and
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> call into the agent code specifically to create the SR
> >>>>>>> before the
> >>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I think the hypervisor management plugin should have a
> >> rich
> >>>>>>> enough
> >>>>>>>>>>>>>>>>> interface to storage to determine available for volume
> >>>>>>> storage.
> >>>>>>>>> For
> >>>>>>>>>>>>>> Xen,
> >>>>>>>>>>>>>>>>> this interface would allow the interrogation of the
> device
> >>>>>>> to
> >>>>>>>>>>>>>> determine the
> >>>>>>>>>>>>>>>>> SR is present. The storage device itself shouldn't know
> >>>>>>> or care
> >>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to answer
> >>>>>>> questions
> >>>>>>>>>>>>>> about it
> >>>>>>>>>>>>>>>>> is storing.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver (which is a
> >>>>>>> type of
> >>>>>>>>>>>>>>>>> so-called
> >>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
> >>>>>>> Storage shows
> >>>>>>>>>>> up
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that references the
> >>>>>>> storage
> >>>>>>>>>>> tag
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack volume. This leads
> >> to
> >>>>>>> a new
> >>>>>>>>>>> row
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
> >>>>>>> (attach
> >>>>>>>>>>> disk).
> >>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>> leads to the storage framework calling the plug-in to
> >>>>>>> create a
> >>>>>>>>> new
> >>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in
> >>>>>>> also
> >>>>>>>>>>> updates
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of
> >>>>>>> the SAN
> >>>>>>>>>>>>>>>>> volume).
> >>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
> >>>>>>> volume is in
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be
> >>>>>>> in the
> >>>>>>>>>>>>>> 'Ready'
> >>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the plug-in code
> >>>>>>> will not
> >>>>>>>>> be
> >>>>>>>>>>>>>>>>> called
> >>>>>>>>>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
> >>>>>>> CloudStack
> >>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor
> >> data
> >>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume,
> >>>>>>> in these
> >>>>>>>>>>>>>>>>>>>> definitions, refers to the physical allocation on the
> >>>>>>> device,
> >>>>>>>>>>>>>>>>> correct?"
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can make in
> >>>>>>> CloudStack, I
> >>>>>>>>>>>>>> try to
> >>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What you say here may very well make sense, but I'm
> >>>>>>> having a
> >>>>>>>>>>> hard
> >>>>>>>>>>>>>>>>> time
> >>>>>>>>>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation
> >>>>>>> as he
> >>>>>>>>> was
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two distinct concepts
> >> --
> >>>>>>>>> storage
> >>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In both cases, we
> >>>>>>> are
> >>>>>>>>>>>>>>>>> communicating with
> >>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
> >>>>>>> volumes
> >>>>>>>>>>>>>> (Dynamic)
> >>>>>>>>>>>>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>> device while the other requires the volume to be
> >>>>>>> volume to be
> >>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that
> we
> >>>>>>> are in
> >>>>>>>>>>>>>> sync on
> >>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to
> >>>>>>> the
> >>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we
> >> must
> >>>>>>> be
> >>>>>>>>> able
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to move bits from
> >>>>>>> one place
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> another,
> >>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage
> >>>>>>> device
> >>>>>>>>> may
> >>>>>>>>>>> be
> >>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
> >>>>>>> unmanaged
> >>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
> >> managed
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable of allocating
> >>>>>>> LUNs and
> >>>>>>>>>>>>>> storing
> >>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
> >>>>>>> manage a
> >>>>>>>>>>>>>> device,
> >>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
> >> CloudStack
> >>>>>>> (e.g.
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> device is
> >>>>>>>>>>>>>>>>>>>>>> shared by many services, and the operator has chosen
> >> to
> >>>>>>>>>>> dedicate
> >>>>>>>>>>>>>>>>> only a
> >>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning
> make
> >>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
> >>>>>>>>> appropriate
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> strip the management concerns from StoragePoolType,
> >>>>>>> add the
> >>>>>>>>>>>>>> notion
> >>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver that
> indicates
> >>>>>>> whether
> >>>>>>>>>>> or
> >>>>>>>>>>>>>>>>> not is
> >>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
> >>>>>>>>>>> representing a
> >>>>>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is associated
> >>>>>>> with a
> >>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>> With
> >>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
> >> declare
> >>>>>>> which
> >>>>>>>>>>>>>>>>> protocols they
> >>>>>>>>>>>>>>>>>>>>>> support and when they encounter a device managed by
> >>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>> utilize the
> >>>>>>>>>>>>>>>>>>>>>> management operations exposed by the driver to
> >> automate
> >>>>>>>>>>>>>> allocation.
> >>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit
> >>>>>>> down and
> >>>>>>>>>>>>>> drill
> >>>>>>>>>>>>>>>>> down to
> >>>>>>>>>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the current iSCSI
> >> type
> >>>>>>> and
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
> >>>>>>> Primary
> >>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
> >>>>>>> volume must
> >>>>>>>>>>>>>> exist
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
> >>>>>>> CloudStack
> >>>>>>>>>>> volumes
> >>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
> >> scenes.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
> >>>>>>> Primary
> >>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>> based
> >>>>>>>>>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
> >>>>>>> storage
> >>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
> >>>>>>> framework). When
> >>>>>>>>> a
> >>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
> >>>>>>> framework
> >>>>>>>>>>> tells
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done,
> >>>>>>> the
> >>>>>>>>> attach
> >>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor
> >>>>>>> data
> >>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was just created.
> >> The
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should probably have
> >>>>>>> defined them
> >>>>>>>>>>> up
> >>>>>>>>>>>>>>>>> front
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex.
> >>>>>>> an admin
> >>>>>>>>>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure
> >> is
> >>>>>>>>> created
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
> >> hypervisor
> >>>>>>> data
> >>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
> >> placed
> >>>>>>> on
> >>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
> >> attach
> >>>>>>> logic
> >>>>>>>>>>>>>> assumes
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already in place and
> >>>>>>> simply
> >>>>>>>>>>>>>> attaches
> >>>>>>>>>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
> >>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not
> pre-allocated.
> >>>>>>> Instead
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
> >>>>>>> volume on
> >>>>>>>>> a
> >>>>>>>>>>>>>> SAN,
> >>>>>>>>>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
> >>>>>>> created
> >>>>>>>>>>>>>> when an
> >>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these types of volumes
> >>>>>>> have not
> >>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an
> admin.
> >>>>>>> Once
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume,
> >> it
> >>>>>>>>> attaches
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
> >>>>>>> attach
> >>>>>>>>>>> behavior
> >>>>>>>>>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
> >>>>>>> difference.
> >>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
> >>>>>>> Dynamic
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely
> used
> >>>>>>> by
> >>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly
> >>>>>>> based on
> >>>>>>>>> an
> >>>>>>>>>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
> >>>>>>>>> hypervisor
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
> >>>>>>>>> CloudStack
> >>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
> >>>>>>> XenServer.
> >>>>>>>>> For
> >>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor
> >>>>>>> and
> >>>>>>>>>>> perform
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the
> difference
> >>>>>>> between
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
> >> Dynamic?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a
> clusterfs
> >>>>>>>>> overlay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool
> >>>>>>> is set
> >>>>>>>>>>> up
> >>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage
> >> (ex.
> >>>>>>>>>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage
> company
> >>>>>>> wants
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
> >>>>>>> feature. It
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need
> >> only
> >>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
> >> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
> >>>>>>> SolidFire
> >>>>>>>>>>>>>> specific.
> >>>>>>>>>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
> >>>>>>> dynamic,
> >>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this
> >>>>>>> kind of
> >>>>>>>>>>>>>>>>> storage).
> >>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
> >>>>>>> framework
> >>>>>>>>>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that
> supported
> >>>>>>> the
> >>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
> >>>>>>> visited with
> >>>>>>>>>>> him
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
> >>>>>>> complete. We
> >>>>>>>>>>>>>> realized
> >>>>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
> >> volumes
> >>>>>>> unless
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of
> storage
> >>>>>>> and
> >>>>>>>>>>> create
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell
> >> <
> >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any
> operation
> >>>>>>> in the
> >>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect
> >>>>>>> to see
> >>>>>>>>> the
> >>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> >>>>>>>>> composed/driven
> >>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
> >>>>>>> structures
> >>>>>>>>>>> per
> >>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going
> >> to
> >>>>>>> end
> >>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
> >> increasingly
> >>>>>>>>>>>>>> difficult to
> >>>>>>>>>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
> >>>>>>> specific
> >>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
> >> method
> >>>>>>> for
> >>>>>>>>>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
> >>>>>>> system,
> >>>>>>>>> then
> >>>>>>>>>>> go
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of
> >> the
> >>>>>>>>> volume
> >>>>>>>>>>>>>> (ex.
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
> >>>>>>> ESX(i)).
> >>>>>>>>>>> VMs
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
> >> storage
> >>>>>>> need
> >>>>>>>>> no
> >>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
> >> relationship
> >>>>>>>>> between
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary
> >> for
> >>>>>>>>> storage
> >>>>>>>>>>>>>>>>> Quality
> >>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called
> >> to
> >>>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did
> >>>>>>> not
> >>>>>>>>>>> address
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
> >>>>>>> structure
> >>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing
> it
> >>>>>>> to do
> >>>>>>>>> so.
> >>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i)
> >> in
> >>>>>>> my
> >>>>>>>>>>> sandbox
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
> >>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
> >>>>>>> hypervisor
> >>>>>>>>>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic
> >>>>>>> (in the
> >>>>>>>>>>>>>> agent
> >>>>>>>>>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have
> >> it
> >>>>>>>>> create a
> >>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume
> >> to
> >>>>>>> a
> >>>>>>>>>>>>>> hypervisor,
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's
> >>>>>>> being
> >>>>>>>>>>>>>> invoked on
> >>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created)
> >> and
> >>>>>>>>> create,
> >>>>>>>>>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore
> >>>>>>> (for
> >>>>>>>>>>>>>> VMware) to
> >>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
> >> recently
> >>>>>>> making
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
> >>>>>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
> >> driver
> >>>>>>> is
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
> >> Tutkowski
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support
> >>>>>>> all
> >>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time
> >>>>>>> for 4.2
> >>>>>>>>> to
> >>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>> KVM
> >>>>>>>>>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
> >> XenServer.
> >>>>>>>>>>>>>> Depending on
> >>>>>>>>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have
> it
> >>>>>>> working
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
> >>>>>>> regards
> >>>>>>>>> to
> >>>>>>>>>>>>>> Disk
> >>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
> >>>>>>> Jamison
> >>>>>>>>>>>>>> Damage
> >>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in
> >>>>>>> the
> >>>>>>>>> patch
> >>>>>>>>>>>>>> cycle
> >>>>>>>>>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE]
> disk_io_throttling
> >>>>>>> to
> >>>>>>>>> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature
> first,
> >>>>>>> then
> >>>>>>>>>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then
> it
> >>>>>>> is a
> >>>>>>>>> non
> >>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei
> >> ZHOU
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
> >>>>>>> billing
> >>>>>>>>> now,
> >>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can
> >> get
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes
> for
> >>>>>>>>>>>>>> calculation.
> >>>>>>>>>>>>>>>>> Of
> >>>>>>>>>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation
> >> value
> >>>>>>> of
> >>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I
> >> can
> >>>>>>> work
> >>>>>>>>> on
> >>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
> >>>>>>> question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the
> two
> >>>>>>>>> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU
> >> <
> >>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
> >>>>>>> trying
> >>>>>>>>> to
> >>>>>>>>>>>>>>>>> adding
> >>>>>>>>>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend
> on
> >>>>>>> IO
> >>>>>>>>>>>>>> statistics
> >>>>>>>>>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
> >>>>>>> disk_io_stat if
> >>>>>>>>> you
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
> >>>>>>> bytes/iops
> >>>>>>>>>>>>>>>>> read/write
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers
> are
> >>>>>>> billed
> >>>>>>>>>>> more
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for
> >> those
> >>>>>>>>>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I
> >> would
> >>>>>>>>> imagine
> >>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost
> models.
> >>>>>>> Could
> >>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime
> change?
> >>>>>>> KVM
> >>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
> >>>>>>> machine
> >>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support
> changing
> >>>>>>> the
> >>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
> >> offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level.
> >> I
> >>>>>>>>>>> apologize
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
> >>>>>>> algorithms
> >>>>>>>>> with a
> >>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
> >>>>>>> provided by a
> >>>>>>>>>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it
> >> is
> >>>>>>>>> better
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
> >>>>>>> consider
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a
> >> later
> >>>>>>>>>>> response
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
> >>>>>>> features. I
> >>>>>>>>>>> think
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
> >>>>>>> SolidFire, but
> >>>>>>>>> I
> >>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
> >>>>>>> (possibly
> >>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on
> >>>>>>> this
> >>>>>>>>>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand
> are
> >>>>>>> the
> >>>>>>>>>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing
> IOPS
> >>>>>>>>> policies
> >>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
> >>>>>>>>> purposes,
> >>>>>>>>>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
> >>>>>>> interface model
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned
> >>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
> >>>>>>> reconciled?
> >>>>>>>>> In
> >>>>>>>>>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed)
> for
> >>>>>>> more
> >>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured
> >> to
> >>>>>>>>>>> deliver.
> >>>>>>>>>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a
> >>>>>>> VM and a
> >>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
> >>>>>>> storage
> >>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
> >>>>>>> Tutkowski
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
> >>>>>>>>>>> controlling
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
> >> controlling
> >>>>>>> IOPS
> >>>>>>>>>>> from
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM,
> >> John
> >>>>>>>>> Burwell
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
> >>>>>>> should be
> >>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it
> >> is
> >>>>>>> ready
> >>>>>>>>> to
> >>>>>>>>>>>>>> turn
> >>>>>>>>>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write
> >>>>>>> ops issue
> >>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria
> >>>>>>> has been
> >>>>>>>>>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >>>>>>>>>>> intersect/compliment
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> https://reviews.apache.org/r/11479/
> >>>>>>> )?
> >>>>>>>>> As I
> >>>>>>>>>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I
> >>>>>>> would
> >>>>>>>>>>> like
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS
> in
> >>>>>>> KVM and
> >>>>>>>>>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei
> >>>>>>> ZHOU <
> >>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
> >>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
> >> features
> >>>>>>> into
> >>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
> >> technical
> >>>>>>>>> reason
> >>>>>>>>>>>>>> till
> >>>>>>>>>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just
> >> want
> >>>>>>> to
> >>>>>>>>>>> mention
> >>>>>>>>>>>>>>>>> again
> >>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap
> so
> >>>>>>> that we
> >>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the
> >> I/O
> >>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
> >>>>>>> legacy from
> >>>>>>>>>>> day
> >>>>>>>>>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
> >> Burwell
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
> >>>>>>> discretely
> >>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed
> >>>>>>> later
> >>>>>>>>>>> because
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
> >>>>>>> write.
> >>>>>>>>>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to
> >>>>>>> break
> >>>>>>>>>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it
> into
> >>>>>>> master
> >>>>>>>>>>> now
> >>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write
> >> IOps
> >>>>>>>>> before
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and
> >> Write
> >>>>>>>>> instead
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
> >> feature
> >>>>>>>>> freeze
> >>>>>>>>>>>>>> date.
> >>>>>>>>>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't
> >> be
> >>>>>>>>>>> necessary
> >>>>>>>>>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that
> there
> >>>>>>> is no
> >>>>>>>>>>>>>>>>> association
> >>>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido
> >>>>>>> den
> >>>>>>>>>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the
> beginning.
> >>>>>>>>> Finally I
> >>>>>>>>>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
> >> mainly
> >>>>>>>>> because
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
> >>>>>>> received
> >>>>>>>>>>>>>> bytes as
> >>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
> >>>>>>> Networking and
> >>>>>>>>>>> disk
> >>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive
> in
> >>>>>>> most
> >>>>>>>>>>>>>>>>> situations
> >>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
> >>>>>>> copy-paste
> >>>>>>>>>>> work.
> >>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
> >>>>>>> feature
> >>>>>>>>>>> freeze,
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't
> >>>>>>> miss. But
> >>>>>>>>>>> if
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we
> >> don't
> >>>>>>>>> release
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
> >> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>>>>>>>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
> >>>>>>> disk_io_throttling
> >>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge
> >> into
> >>>>>>> master
> >>>>>>>>>>> in
> >>>>>>>>>>>>>> 48
> >>>>>>>>>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on
> >> the
> >>>>>>> same
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the
> rate
> >>>>>>>>>>> limitation
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation,
> >> it
> >>>>>>> may
> >>>>>>>>>>> affect
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
> >>>>>>> storage
> >>>>>>>>>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the
> maximum
> >>>>>>> rate
> >>>>>>>>> and
> >>>>>>>>>>>>>> limit
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you
> make
> >>>>>>> no
> >>>>>>>>>>>>>> difference
> >>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting
> >>>>>>> both a
> >>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
> >>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling
> >>>>>>> side you
> >>>>>>>>>>>>>>>>> collect
> >>>>>>>>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side
> >>>>>>> you only
> >>>>>>>>> go
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
> >>>>>>> expensive
> >>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that
> >> an
> >>>>>>> admin
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at
> >> this
> >>>>>>>>> point I
> >>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
> >>>>>>> mechanism in
> >>>>>>>>>>> place
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of
> work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
> >>>>>>> setting
> >>>>>>>>>>>>>> different
> >>>>>>>>>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up
> >>>>>>> and set
> >>>>>>>>>>> that
> >>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
> >> implemented
> >>>>>>> it
> >>>>>>>>> this
> >>>>>>>>>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs
> >> (in
> >>>>>>>>>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of
> VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total
> bps
> >>>>>>> and
> >>>>>>>>> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> >>>>>>>>> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
> >>>>>>> execution
> >>>>>>>>>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
> >>>>>>> introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit
> and
> >>>>>>>>>>>>>> integration)
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to
> >>>>>>> check for
> >>>>>>>>>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS
> >>>>>>> rate on
> >>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
> >>>>>>> destroy,
> >>>>>>>>>>> expunge.
> >>>>>>>>>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> >> ****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>>>> SolidFire
> >>>>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> >> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Wei,
We have been down the rabbit hole a bit on the Storage/Hypervisor layer separation, but we still need to reconcile the behavior of hypervisor throttled I/O and storage provisioned IOPS. I see the following issues outstanding:
1. Hypervisor throttled IOPS are expressed as discrete read/write values where as storage provisioned IOPS are expressed as total IOPS.
2. How do we handle VMs with throttled IOPS attached to storage volumes with provisioned IOPS?
3. How should usage data be captured for throttled and provisioned IOPS that will permit providers to discriminate these guaranteed operations in the event they want to bill for it differently?
4. What is the user experience for throttled and provisioned IOPS that minimizes confusion of these concepts?
My thinking is that a VM can have either utilize hypervisor throttled IOPS or storage provisioned IOPS. This policy would neatly solve items 1 and 2. Since the two facilities would not be permitted to operate together, they do not need to be semantically compatible. I think item 3 can be resolved with an additional flag or two on the usage records. As for Item 4, I am not familiar with how these two enhancements are depicted in the user interface. I think we need to review the UI enhancements for both enhancements and ensure they are consistent.
Do these solutions make sense?
Thanks,
-John
On Jun 6, 2013, at 5:22 PM, Wei ZHOU <us...@gmail.com> wrote:
> John and Mike,
>
> I was busy working on other issues (CLOUDSTACK-2780/2729,
> CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this week.
> I will start to develop on iops/bps changes tomorrow, and ask for second
> merge request after testing.
>
> -Wei
>
>
> 2013/6/6 Mike Tutkowski <mi...@solidfire.com>
>
>> I believe I understand where you're going with this, John, and have been
>> re-working this section of code today.
>>
>> I should be able to run it by you tomorrow.
>>
>> Thanks for the comments,
>> Mike
>>
>>
>> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> See my responses in-line below.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>> wrote:
>>>
>>> Hi John,
>>>
>>> Thanks for the response.
>>>
>>> "I am fine with the VolumeManager determining whether or not a Volume is
>>> managed (i.e. not based on the StoragePoolType, but an actual isManaged
>>> method), and asking the device driver to allocate resources for the
>> volume
>>> if it is managed."
>>>
>>> Are you thinking you'd like to see an isManaged() method added to the
>>> PrimaryDataStoreDriver interface? If it returns true, then the storage
>>> framework could call the manage() (or whatever name) method (which would
>> be
>>> new to the PrimaryDataStoreDriver interface, as well) and this would call
>>> into a new method in the hypervisor code to create, say on XenServer, an
>> SR?
>>>
>>>
>>> I would like to see canBeManaged() : boolean on DataStoreDriver. Since
>>> the notion of Volumes only pertains to primary storage, I would add a
>>> allocateStorage and deallocateStorage (Storage is a straw man term --
>>> something other than volume) methods to allocate/create/deallocate/delete
>>> underlying storage. To my mind, managed is a mutable property of
>> DataStore
>>> which can be enabled if/when the underlying DataStoreDriver can be
>> managed.
>>> This approach allows operators to override manageability of devices.
>>>
>>> In terms of orchestration/process flow for SR, the Xen plugin would be
>>> responsible for composing DataStore/Volume methods to create any
>>> directories or files necessary for the SR. There should be no
>> dependencies
>>> from the Storage to the Hypervisor layer. As I said earlier, such
>> circular
>>> dependencies will lead to a tangled, unmaintainable mess.
>>>
>>>
>>> Just want to make sure I'm on the same page with you.
>>>
>>> Thanks again, John
>>>
>>>
>>> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com> wrote:
>>>
>>>> Mike,
>>>>
>>>> Fundamentally, we can't end up with a Storage layer that supports n
>>>> devices types with each specific behaviors of m hypervisors. Such a
>>>> scenario will create an unmaintainable and untestable beast.
>> Therefore, my
>>>> thoughts and recommendations are driven to evolve the Storage layer
>> towards
>>>> this separation.
>>>>
>>>> I am fine with the VolumeManager determining whether or not a Volume is
>>>> managed (i.e. not based on the StoragePoolType, but an actual isManaged
>>>> method), and asking the device driver to allocate resources for the
>> volume
>>>> if it is managed. Furthermore, the device driver needs to indicate
>> whether
>>>> or not it supports management operations. Finally, I think we need to
>>>> provide the ability for an administrator to elect to have something
>> that is
>>>> manageable be unmanaged (i.e. the driver is capable managing the device,
>>>> but the administrator has elected to leave it unmanaged).
>>>>
>>>> Creation of a structure on the volume should be done in the Xen
>>>> hypervisor module using methods exposed by the Storage layer to perform
>>>> low-level operations (e.g. make directories, create a file, etc). This
>>>> structure is specific to the operation of the Xen hypervisor, as such,
>>>> should be confined to its implementation. From my perspective, nothing
>> in
>>>> the Storage layer should be concerned with content. From its
>> perspective,
>>>> structure and data are opaque. It provides the means to query the data
>> to
>>>> support the interpretation of the content by higher-level layers (e.g.
>>>> Hypervisors). To my mind, attach should be a composition of operations
>>>> from the Storage layer that varies based on the Volume storage protocol
>>>> (iSCSI, local file system, NFS, RBD, etc).
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>> Hi John,
>>>>
>>>> Alternatively to the way the attach logic is implemented in my patch, we
>>>> could do the following:
>>>>
>>>> Leave the attach logic in the agent code alone. In VolumeManagerImpl we
>>>> create an AttachVolumeCommand and send it to the hypervisor. Before this
>>>> command is sent, we could check to see if we're dealing with Dynamic (or
>>>> whatever we want to call it) storage and - if so - send a "Create SR"
>>>> command to the hypervisor. If this returns OK, we would then proceed to
>> the
>>>> AttachVolumeCommand, as usual.
>>>>
>>>> This way the attach logic remains the same and we just add another
>>>> command to the agent code that is called for this particular type of
>>>> storage.
>>>>
>>>> What do you think?
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Hey John,
>>>>>
>>>>> I created a document for a customer today that outlines how the plug-in
>>>>> works from a user standpoint. This will probably be of use to you, as
>> well,
>>>>> as you perform the code review.
>>>>>
>>>>> I have shared this document with you (you should have received that
>>>>> information in a separate e-mail).
>>>>>
>>>>> Talk to you later!
>>>>>
>>>>>
>>>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> Oh, OK, that sounds really good, John.
>>>>>>
>>>>>> Thanks and talk to you tomorrow! :)
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jburwell@basho.com
>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> I am never at a loss for an opinion. I some thoughts, but want to
>>>>>>> confirm assumptions and ideas against the solidfire,
>> disk_io_throttle,
>>>>>>> and
>>>>>>> object_store branches. I hope to collect them in a coherent form
>>>>>>> tomorrow (5 June 2013).
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>>
>>>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> "So, in essence, the SolidFire plugin introduces the notion of a
>>>>>>> managed
>>>>>>>> iSCSI device and provisioned IOPS."
>>>>>>>>
>>>>>>>> Technically, the SolidFire plug-in just introduces the notion of
>>>>>>>> provisioned storage IOPS.
>>>>>>>>
>>>>>>>> The storage framework that leverages the plug-in was incomplete, so
>>>>>>> I had
>>>>>>>> to try to add in the notion of a managed iSCSI device.
>>>>>>>>
>>>>>>>> I appreciate all the time you've been spending on this. :) Do you
>>>>>>> have a
>>>>>>>> recommendation as to how we should accomplish what you're looking
>>>>>>> for?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> So, in essence, the SolidFire plugin introduces the notion of a
>>>>>>> managed
>>>>>>>>> iSCSI device and provisioned IOPS.
>>>>>>>>>
>>>>>>>>> I want to see a separation of the management capabilities (i.e.
>> can
>>>>>>> a
>>>>>>>>> device be managed/does an operator want it managed by CloudStack)
>>>>>>> from the
>>>>>>>>> storage protocol. Ideally, we should end up with a semantic that
>>>>>>> will
>>>>>>>>> allow any type of storage device to be managed. I also want to
>> make
>>>>>>>>> progress on decoupling the storage types from the hypervisor
>>>>>>> definitions.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi John,
>>>>>>>>>>
>>>>>>>>>> No problem. Answers are below in red.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jburwell@basho.com
>>>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> Could you please answer the following questions for me with
>>>>>>> regards to
>>>>>>>>> the
>>>>>>>>>>> operation of the SolidFire plugin:
>>>>>>>>>>>
>>>>>>>>>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>>>>>>>>> Each SAN volume is equivalent to a single LUN (LUN 0).
>>>>>>>>>> 1 SAN volume : 1 LUN
>>>>>>>>>>
>>>>>>>>>>> What is the cardinality between SAN Volumes and CloudStack
>>>>>>> Volumes?
>>>>>>>>>> 1 SAN volume : 1 CloudStack volume
>>>>>>>>>>
>>>>>>>>>>> Are the LUN(s) created by the management server or externally by
>>>>>>> the
>>>>>>>>>>> operator?
>>>>>>>>>> When used with the SolidFire plug-in, a SAN volume (same as a SAN
>>>>>>> LUN) is
>>>>>>>>>> created by the management server (via the plug-in) the first time
>>>>>>> the
>>>>>>>>>> CloudStack volume is attached to a hypervisor.
>>>>>>>>>>
>>>>>>>>>> If you don't want to use the SolidFire plug-in, but still want to
>>>>>>> use a
>>>>>>>>>> SolidFire volume (LUN), you can do this already today (prior to
>>>>>>> 4.2). The
>>>>>>>>>> admin manually creates the SAN volume and - in this case -
>>>>>>> multiple VMs
>>>>>>>>> and
>>>>>>>>>> data disks can share this SAN volume. While you can do this
>> today,
>>>>>>> it is
>>>>>>>>>> not useful if you want to enforce storage QoS.
>>>>>>>>>>
>>>>>>>>>>> Are the SAN volumes by the management server or externally by
>> the
>>>>>>>>> operator?
>>>>>>>>>> When the SolidFire plug-in is used, the SAN volumes are
>> completely
>>>>>>>>> managed
>>>>>>>>>> by the management server (via the plug-in). There is no admin
>>>>>>>>> interaction.
>>>>>>>>>> This allows for a 1:1 mapping between a SAN volume and a
>> CloudStack
>>>>>>>>> volume,
>>>>>>>>>> which is necessary for any storage vendor that supports true QoS.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I would like to clarify how these pieces are related and
>> expected
>>>>>>> to
>>>>>>>>>>> operate.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> "In particular, how do we ensure that multiple VMs with
>>>>>>> provisioned
>>>>>>>>> IOPS
>>>>>>>>>>>> won't be cut off by the underlying storage."
>>>>>>>>>>>>
>>>>>>>>>>>> In the storage QoS world, we need to map a single SAN volume
>>>>>>> (LUN) to a
>>>>>>>>>>>> single CloudStack volume. We cannot have multiple CloudStack
>>>>>>> volumes
>>>>>>>>>>>> sharing a single SAN volume and still guarantee QoS.
>>>>>>>>>>>>
>>>>>>>>>>>> If the user wants to have a single SAN volume house more than
>> one
>>>>>>>>>>>> CloudStack volume, then can do that today without any of my
>>>>>>> plug-in
>>>>>>>>> code.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> "The administrator will allocate a SAN volume for CloudStack's
>>>>>>> use
>>>>>>>>> onto
>>>>>>>>>>>>> which CloudStack volumes will be created."
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think we crossed e-mails. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Check out my recent e-mail on this.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You are coming to the part which concerns me -- concepts from
>>>>>>> the
>>>>>>>>>>>>>> hypervisor are leaking into storage layer.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The weird part is that the iSCSI type is today only used (as
>>>>>>> far as
>>>>>>>>> I
>>>>>>>>>>>>>> know)
>>>>>>>>>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you want to use your iSCSI volume from VMware, it uses
>> the
>>>>>>> vmfs
>>>>>>>>>>> type.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>>>>>>>>>>>>>> SharedMountPoint
>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So, I suppose mine and Edison's thinking here was to make a
>>>>>>> new type
>>>>>>>>>>> of
>>>>>>>>>>>>>>> storage to describe this dynamic ability Edison added into
>> the
>>>>>>>>> storage
>>>>>>>>>>>>>>> framework. Maybe it should be more specificy, though:
>>>>>>> Dynamic_iSCSI
>>>>>>>>>>>>>> versus,
>>>>>>>>>>>>>>> say, Dynamic_FC.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> "The storage device itself shouldn't know or care that it
>> is
>>>>>>> being
>>>>>>>>>>> used
>>>>>>>>>>>>>>>> for a Xen SR -- simply be able to answer questions about it
>>>>>>> is
>>>>>>>>>>>>>> storing."
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I see...so your concern here is that the SolidFire plug-in
>>>>>>> needs to
>>>>>>>>>>>>>> call
>>>>>>>>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows
>>>>>>> to
>>>>>>>>> treat
>>>>>>>>>>>>>> it as
>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm totally open to removing that constraint and just
>>>>>>> calling it
>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> whatever. We would just need a way for the hypervisor
>> attach
>>>>>>> logic
>>>>>>>>> to
>>>>>>>>>>>>>>>> detect this new requirement and perform the necessary
>>>>>>> activities.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> See my responses in-line.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "Finally, while CloudStack may be able to manage a
>> device,
>>>>>>> an
>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
>> device
>>>>>>> is
>>>>>>>>>>> shared
>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>> many services, and the operator has chosen to dedicate
>>>>>>> only a
>>>>>>>>>>> portion
>>>>>>>>>>>>>>>>> of it
>>>>>>>>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
>>>>>>> plug-in
>>>>>>>>>>>>>> (before
>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>> attach request to the hypervisor) would work unless the
>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>> the SAN volume in the form of, say, an SR.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> My thinking is that, independent of CloudStack, an
>> operator
>>>>>>>>>>> allocates
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a
>>>>>>> LUN. They
>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>> want to turn control of that LUN over to CloudStack, but
>>>>>>> not allow
>>>>>>>>>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As the attach logic stands prior to my changes, we would
>> be
>>>>>>>>> passing
>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>> SAN volume that does not have the necessary hypervisor
>>>>>>> support
>>>>>>>>>>> (like
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>> SR)
>>>>>>>>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Are you thinking we should maybe have the storage
>> framework
>>>>>>>>> itself
>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>> that such a SAN volume needs support from the hypervisor
>>>>>>> side and
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> call into the agent code specifically to create the SR
>>>>>>> before the
>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think the hypervisor management plugin should have a
>> rich
>>>>>>> enough
>>>>>>>>>>>>>>>>> interface to storage to determine available for volume
>>>>>>> storage.
>>>>>>>>> For
>>>>>>>>>>>>>> Xen,
>>>>>>>>>>>>>>>>> this interface would allow the interrogation of the device
>>>>>>> to
>>>>>>>>>>>>>> determine the
>>>>>>>>>>>>>>>>> SR is present. The storage device itself shouldn't know
>>>>>>> or care
>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>>>>>>> questions
>>>>>>>>>>>>>> about it
>>>>>>>>>>>>>>>>> is storing.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * The admin registers the SolidFire driver (which is a
>>>>>>> type of
>>>>>>>>>>>>>>>>> so-called
>>>>>>>>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
>>>>>>> Storage shows
>>>>>>>>>>> up
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * The admin creates a Disk Offering that references the
>>>>>>> storage
>>>>>>>>>>> tag
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * The end user creates a CloudStack volume. This leads
>> to
>>>>>>> a new
>>>>>>>>>>> row
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
>>>>>>> (attach
>>>>>>>>>>> disk).
>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>> leads to the storage framework calling the plug-in to
>>>>>>> create a
>>>>>>>>> new
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in
>>>>>>> also
>>>>>>>>>>> updates
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of
>>>>>>> the SAN
>>>>>>>>>>>>>>>>> volume).
>>>>>>>>>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
>>>>>>> volume is in
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be
>>>>>>> in the
>>>>>>>>>>>>>> 'Ready'
>>>>>>>>>>>>>>>>>>> state (even after a detach disk) and the plug-in code
>>>>>>> will not
>>>>>>>>> be
>>>>>>>>>>>>>>>>> called
>>>>>>>>>>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>>>>>>> CloudStack
>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor
>> data
>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume,
>>>>>>> in these
>>>>>>>>>>>>>>>>>>>> definitions, refers to the physical allocation on the
>>>>>>> device,
>>>>>>>>>>>>>>>>> correct?"
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>>>>>>> CloudStack, I
>>>>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What you say here may very well make sense, but I'm
>>>>>>> having a
>>>>>>>>>>> hard
>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation
>>>>>>> as he
>>>>>>>>> was
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> initial person to suggest the approach I took.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> It feels like we are combining two distinct concepts
>> --
>>>>>>>>> storage
>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>> management and storage protocols. In both cases, we
>>>>>>> are
>>>>>>>>>>>>>>>>> communicating with
>>>>>>>>>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
>>>>>>> volumes
>>>>>>>>>>>>>> (Dynamic)
>>>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>> device while the other requires the volume to be
>>>>>>> volume to be
>>>>>>>>>>>>>>>>> managed
>>>>>>>>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we
>>>>>>> are in
>>>>>>>>>>>>>> sync on
>>>>>>>>>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to
>>>>>>> the
>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we
>> must
>>>>>>> be
>>>>>>>>> able
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> communicate with a storage device to move bits from
>>>>>>> one place
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> another,
>>>>>>>>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage
>>>>>>> device
>>>>>>>>> may
>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
>>>>>>> unmanaged
>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
>> managed
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>> device on which CloudStack is capable of allocating
>>>>>>> LUNs and
>>>>>>>>>>>>>> storing
>>>>>>>>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
>>>>>>> manage a
>>>>>>>>>>>>>> device,
>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
>> CloudStack
>>>>>>> (e.g.
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> device is
>>>>>>>>>>>>>>>>>>>>>> shared by many services, and the operator has chosen
>> to
>>>>>>>>>>> dedicate
>>>>>>>>>>>>>>>>> only a
>>>>>>>>>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make
>>>>>>> sense?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>>>>>>>>> appropriate
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> strip the management concerns from StoragePoolType,
>>>>>>> add the
>>>>>>>>>>>>>> notion
>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>> storage device with an attached driver that indicates
>>>>>>> whether
>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> not is
>>>>>>>>>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>>>>>>>>>>> representing a
>>>>>>>>>>>>>>>>> physical
>>>>>>>>>>>>>>>>>>>>>> allocation on a device separate that is associated
>>>>>>> with a
>>>>>>>>>>> volume.
>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
>> declare
>>>>>>> which
>>>>>>>>>>>>>>>>> protocols they
>>>>>>>>>>>>>>>>>>>>>> support and when they encounter a device managed by
>>>>>>>>> CloudStack,
>>>>>>>>>>>>>>>>> utilize the
>>>>>>>>>>>>>>>>>>>>>> management operations exposed by the driver to
>> automate
>>>>>>>>>>>>>> allocation.
>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit
>>>>>>> down and
>>>>>>>>>>>>>> drill
>>>>>>>>>>>>>>>>> down to
>>>>>>>>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Here is the difference between the current iSCSI
>> type
>>>>>>> and
>>>>>>>>> the
>>>>>>>>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
>>>>>>> Primary
>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
>>>>>>> volume must
>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
>>>>>>> CloudStack
>>>>>>>>>>> volumes
>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
>> scenes.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
>>>>>>> Primary
>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
>>>>>>> storage
>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
>>>>>>> framework). When
>>>>>>>>> a
>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>>>>>>> framework
>>>>>>>>>>> tells
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done,
>>>>>>> the
>>>>>>>>> attach
>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor
>>>>>>> data
>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>>>>>>>>> at this point because the volume was just created.
>> The
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> These are new terms, so I should probably have
>>>>>>> defined them
>>>>>>>>>>> up
>>>>>>>>>>>>>>>>> front
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex.
>>>>>>> an admin
>>>>>>>>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure
>> is
>>>>>>>>> created
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
>> hypervisor
>>>>>>> data
>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
>> placed
>>>>>>> on
>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
>> attach
>>>>>>> logic
>>>>>>>>>>>>>> assumes
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure is already in place and
>>>>>>> simply
>>>>>>>>>>>>>> attaches
>>>>>>>>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
>>>>>>> Instead
>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
>>>>>>> volume on
>>>>>>>>> a
>>>>>>>>>>>>>> SAN,
>>>>>>>>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
>>>>>>> created
>>>>>>>>>>>>>> when an
>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>> volume is performed because these types of volumes
>>>>>>> have not
>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin.
>>>>>>> Once
>>>>>>>>> the
>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume,
>> it
>>>>>>>>> attaches
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
>>>>>>> attach
>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>>>>>>> difference.
>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
>>>>>>> Dynamic
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used
>>>>>>> by
>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly
>>>>>>> based on
>>>>>>>>> an
>>>>>>>>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>>>>>>>>> hypervisor
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>>>>>>>>> CloudStack
>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
>>>>>>> XenServer.
>>>>>>>>> For
>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor
>>>>>>> and
>>>>>>>>>>> perform
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
>>>>>>> between
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
>> Dynamic?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>>>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>>>>>>>>> overlay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool
>>>>>>> is set
>>>>>>>>>>> up
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage
>> (ex.
>>>>>>>>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company
>>>>>>> wants
>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>>>>>>> feature. It
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need
>> only
>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to
>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
>>>>>>> SolidFire
>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
>>>>>>> dynamic,
>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this
>>>>>>> kind of
>>>>>>>>>>>>>>>>> storage).
>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
>>>>>>> framework
>>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported
>>>>>>> the
>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
>>>>>>> visited with
>>>>>>>>>>> him
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
>>>>>>> complete. We
>>>>>>>>>>>>>> realized
>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
>> volumes
>>>>>>> unless
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage
>>>>>>> and
>>>>>>>>>>> create
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell
>> <
>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation
>>>>>>> in the
>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect
>>>>>>> to see
>>>>>>>>> the
>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>>>>>>>>> composed/driven
>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>>>>>>> structures
>>>>>>>>>>> per
>>>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going
>> to
>>>>>>> end
>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
>> increasingly
>>>>>>>>>>>>>> difficult to
>>>>>>>>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
>>>>>>> specific
>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
>> method
>>>>>>> for
>>>>>>>>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
>>>>>>> system,
>>>>>>>>> then
>>>>>>>>>>> go
>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of
>> the
>>>>>>>>> volume
>>>>>>>>>>>>>> (ex.
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
>>>>>>> ESX(i)).
>>>>>>>>>>> VMs
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
>> storage
>>>>>>> need
>>>>>>>>> no
>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
>> relationship
>>>>>>>>> between
>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary
>> for
>>>>>>>>> storage
>>>>>>>>>>>>>>>>> Quality
>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called
>> to
>>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did
>>>>>>> not
>>>>>>>>>>> address
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
>>>>>>> structure
>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it
>>>>>>> to do
>>>>>>>>> so.
>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i)
>> in
>>>>>>> my
>>>>>>>>>>> sandbox
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
>>>>>>> Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
>>>>>>> hypervisor
>>>>>>>>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic
>>>>>>> (in the
>>>>>>>>>>>>>> agent
>>>>>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have
>> it
>>>>>>>>> create a
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume
>> to
>>>>>>> a
>>>>>>>>>>>>>> hypervisor,
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's
>>>>>>> being
>>>>>>>>>>>>>> invoked on
>>>>>>>>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created)
>> and
>>>>>>>>> create,
>>>>>>>>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore
>>>>>>> (for
>>>>>>>>>>>>>> VMware) to
>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
>> recently
>>>>>>> making
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
>>>>>>> Burwell <
>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
>> driver
>>>>>>> is
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
>> Tutkowski
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support
>>>>>>> all
>>>>>>>>>>>>>> hypervisors
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time
>>>>>>> for 4.2
>>>>>>>>> to
>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>> KVM
>>>>>>>>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
>> XenServer.
>>>>>>>>>>>>>> Depending on
>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
>>>>>>> working
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
>>>>>>> regards
>>>>>>>>> to
>>>>>>>>>>>>>> Disk
>>>>>>>>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
>>>>>>> Jamison
>>>>>>>>>>>>>> Damage
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in
>>>>>>> the
>>>>>>>>> patch
>>>>>>>>>>>>>> cycle
>>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling
>>>>>>> to
>>>>>>>>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first,
>>>>>>> then
>>>>>>>>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it
>>>>>>> is a
>>>>>>>>> non
>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei
>> ZHOU
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
>>>>>>> billing
>>>>>>>>> now,
>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can
>> get
>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>>>>>>>>>>>>>> calculation.
>>>>>>>>>>>>>>>>> Of
>>>>>>>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation
>> value
>>>>>>> of
>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I
>> can
>>>>>>> work
>>>>>>>>> on
>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>>>>>>> question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>>>>>>>>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU
>> <
>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
>>>>>>> trying
>>>>>>>>> to
>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on
>>>>>>> IO
>>>>>>>>>>>>>> statistics
>>>>>>>>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
>>>>>>> disk_io_stat if
>>>>>>>>> you
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>>>>>>> bytes/iops
>>>>>>>>>>>>>>>>> read/write
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are
>>>>>>> billed
>>>>>>>>>>> more
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for
>> those
>>>>>>>>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I
>> would
>>>>>>>>> imagine
>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
>>>>>>> Could
>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change?
>>>>>>> KVM
>>>>>>>>>>>>>> supports
>>>>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
>>>>>>> machine
>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing
>>>>>>> the
>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
>> offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level.
>> I
>>>>>>>>>>> apologize
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
>>>>>>> algorithms
>>>>>>>>> with a
>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
>>>>>>> provided by a
>>>>>>>>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it
>> is
>>>>>>>>> better
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
>>>>>>> consider
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a
>> later
>>>>>>>>>>> response
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
>>>>>>> features. I
>>>>>>>>>>> think
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
>>>>>>> SolidFire, but
>>>>>>>>> I
>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>>>>>>> (possibly
>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on
>>>>>>> this
>>>>>>>>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are
>>>>>>> the
>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>>>>>>>>> policies
>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>>>>>>>>> purposes,
>>>>>>>>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
>>>>>>> interface model
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned
>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>>>>>>> reconciled?
>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for
>>>>>>> more
>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured
>> to
>>>>>>>>>>> deliver.
>>>>>>>>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a
>>>>>>> VM and a
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
>>>>>>> storage
>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
>>>>>>> Tutkowski
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>>>>>>>>>>> controlling
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
>> controlling
>>>>>>> IOPS
>>>>>>>>>>> from
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM,
>> John
>>>>>>>>> Burwell
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
>>>>>>> should be
>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it
>> is
>>>>>>> ready
>>>>>>>>> to
>>>>>>>>>>>>>> turn
>>>>>>>>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write
>>>>>>> ops issue
>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria
>>>>>>> has been
>>>>>>>>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>>>>>>>>>>> intersect/compliment
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> https://reviews.apache.org/r/11479/
>>>>>>> )?
>>>>>>>>> As I
>>>>>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I
>>>>>>> would
>>>>>>>>>>> like
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in
>>>>>>> KVM and
>>>>>>>>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei
>>>>>>> ZHOU <
>>>>>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU
>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
>> features
>>>>>>> into
>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
>> technical
>>>>>>>>> reason
>>>>>>>>>>>>>> till
>>>>>>>>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just
>> want
>>>>>>> to
>>>>>>>>>>> mention
>>>>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so
>>>>>>> that we
>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the
>> I/O
>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
>>>>>>> legacy from
>>>>>>>>>>> day
>>>>>>>>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
>> Burwell
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>>>>>>> discretely
>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed
>>>>>>> later
>>>>>>>>>>> because
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
>>>>>>> write.
>>>>>>>>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to
>>>>>>> break
>>>>>>>>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into
>>>>>>> master
>>>>>>>>>>> now
>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write
>> IOps
>>>>>>>>> before
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and
>> Write
>>>>>>>>> instead
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
>> feature
>>>>>>>>> freeze
>>>>>>>>>>>>>> date.
>>>>>>>>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't
>> be
>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there
>>>>>>> is no
>>>>>>>>>>>>>>>>> association
>>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido
>>>>>>> den
>>>>>>>>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU
>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>>>>>>>>> Finally I
>>>>>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
>> mainly
>>>>>>>>> because
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
>>>>>>> received
>>>>>>>>>>>>>> bytes as
>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
>>>>>>> Networking and
>>>>>>>>>>> disk
>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in
>>>>>>> most
>>>>>>>>>>>>>>>>> situations
>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
>>>>>>> copy-paste
>>>>>>>>>>> work.
>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
>>>>>>> feature
>>>>>>>>>>> freeze,
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't
>>>>>>> miss. But
>>>>>>>>>>> if
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we
>> don't
>>>>>>>>> release
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>>>>>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU
>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>>>>>>> disk_io_throttling
>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge
>> into
>>>>>>> master
>>>>>>>>>>> in
>>>>>>>>>>>>>> 48
>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on
>> the
>>>>>>> same
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>>>>>>>>>>> limitation
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation,
>> it
>>>>>>> may
>>>>>>>>>>> affect
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
>>>>>>> storage
>>>>>>>>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum
>>>>>>> rate
>>>>>>>>> and
>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make
>>>>>>> no
>>>>>>>>>>>>>> difference
>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting
>>>>>>> both a
>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
>>>>>>> users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling
>>>>>>> side you
>>>>>>>>>>>>>>>>> collect
>>>>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side
>>>>>>> you only
>>>>>>>>> go
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
>>>>>>> expensive
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that
>> an
>>>>>>> admin
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at
>> this
>>>>>>>>> point I
>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
>>>>>>> mechanism in
>>>>>>>>>>> place
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
>>>>>>> setting
>>>>>>>>>>>>>> different
>>>>>>>>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up
>>>>>>> and set
>>>>>>>>>>> that
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
>> implemented
>>>>>>> it
>>>>>>>>> this
>>>>>>>>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs
>> (in
>>>>>>>>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps
>>>>>>> and
>>>>>>>>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
>>>>>>> execution
>>>>>>>>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>>>>>>> introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>>>>>>>>>>>>>> integration)
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to
>>>>>>> check for
>>>>>>>>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS
>>>>>>> rate on
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
>>>>>>> destroy,
>>>>>>>>>>> expunge.
>>>>>>>>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
>> ****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>>>> SolidFire
>>>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>>>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
John and Mike,
I was busy working on other issues (CLOUDSTACK-2780/2729,
CLOUDSTACK-2856/2857/2865, CLOUDSTACK-2823 , CLOUDSTACK-2875 ) this week.
I will start to develop on iops/bps changes tomorrow, and ask for second
merge request after testing.
-Wei
2013/6/6 Mike Tutkowski <mi...@solidfire.com>
> I believe I understand where you're going with this, John, and have been
> re-working this section of code today.
>
> I should be able to run it by you tomorrow.
>
> Thanks for the comments,
> Mike
>
>
> On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com> wrote:
>
> > Mike,
> >
> > See my responses in-line below.
> >
> > Thanks,
> > -John
> >
> > On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> > wrote:
> >
> > Hi John,
> >
> > Thanks for the response.
> >
> > "I am fine with the VolumeManager determining whether or not a Volume is
> > managed (i.e. not based on the StoragePoolType, but an actual isManaged
> > method), and asking the device driver to allocate resources for the
> volume
> > if it is managed."
> >
> > Are you thinking you'd like to see an isManaged() method added to the
> > PrimaryDataStoreDriver interface? If it returns true, then the storage
> > framework could call the manage() (or whatever name) method (which would
> be
> > new to the PrimaryDataStoreDriver interface, as well) and this would call
> > into a new method in the hypervisor code to create, say on XenServer, an
> SR?
> >
> >
> > I would like to see canBeManaged() : boolean on DataStoreDriver. Since
> > the notion of Volumes only pertains to primary storage, I would add a
> > allocateStorage and deallocateStorage (Storage is a straw man term --
> > something other than volume) methods to allocate/create/deallocate/delete
> > underlying storage. To my mind, managed is a mutable property of
> DataStore
> > which can be enabled if/when the underlying DataStoreDriver can be
> managed.
> > This approach allows operators to override manageability of devices.
> >
> > In terms of orchestration/process flow for SR, the Xen plugin would be
> > responsible for composing DataStore/Volume methods to create any
> > directories or files necessary for the SR. There should be no
> dependencies
> > from the Storage to the Hypervisor layer. As I said earlier, such
> circular
> > dependencies will lead to a tangled, unmaintainable mess.
> >
> >
> > Just want to make sure I'm on the same page with you.
> >
> > Thanks again, John
> >
> >
> > On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com> wrote:
> >
> >> Mike,
> >>
> >> Fundamentally, we can't end up with a Storage layer that supports n
> >> devices types with each specific behaviors of m hypervisors. Such a
> >> scenario will create an unmaintainable and untestable beast.
> Therefore, my
> >> thoughts and recommendations are driven to evolve the Storage layer
> towards
> >> this separation.
> >>
> >> I am fine with the VolumeManager determining whether or not a Volume is
> >> managed (i.e. not based on the StoragePoolType, but an actual isManaged
> >> method), and asking the device driver to allocate resources for the
> volume
> >> if it is managed. Furthermore, the device driver needs to indicate
> whether
> >> or not it supports management operations. Finally, I think we need to
> >> provide the ability for an administrator to elect to have something
> that is
> >> manageable be unmanaged (i.e. the driver is capable managing the device,
> >> but the administrator has elected to leave it unmanaged).
> >>
> >> Creation of a structure on the volume should be done in the Xen
> >> hypervisor module using methods exposed by the Storage layer to perform
> >> low-level operations (e.g. make directories, create a file, etc). This
> >> structure is specific to the operation of the Xen hypervisor, as such,
> >> should be confined to its implementation. From my perspective, nothing
> in
> >> the Storage layer should be concerned with content. From its
> perspective,
> >> structure and data are opaque. It provides the means to query the data
> to
> >> support the interpretation of the content by higher-level layers (e.g.
> >> Hypervisors). To my mind, attach should be a composition of operations
> >> from the Storage layer that varies based on the Volume storage protocol
> >> (iSCSI, local file system, NFS, RBD, etc).
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >> Hi John,
> >>
> >> Alternatively to the way the attach logic is implemented in my patch, we
> >> could do the following:
> >>
> >> Leave the attach logic in the agent code alone. In VolumeManagerImpl we
> >> create an AttachVolumeCommand and send it to the hypervisor. Before this
> >> command is sent, we could check to see if we're dealing with Dynamic (or
> >> whatever we want to call it) storage and - if so - send a "Create SR"
> >> command to the hypervisor. If this returns OK, we would then proceed to
> the
> >> AttachVolumeCommand, as usual.
> >>
> >> This way the attach logic remains the same and we just add another
> >> command to the agent code that is called for this particular type of
> >> storage.
> >>
> >> What do you think?
> >>
> >>
> >> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> Hey John,
> >>>
> >>> I created a document for a customer today that outlines how the plug-in
> >>> works from a user standpoint. This will probably be of use to you, as
> well,
> >>> as you perform the code review.
> >>>
> >>> I have shared this document with you (you should have received that
> >>> information in a separate e-mail).
> >>>
> >>> Talk to you later!
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> Oh, OK, that sounds really good, John.
> >>>>
> >>>> Thanks and talk to you tomorrow! :)
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jburwell@basho.com
> >wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> I am never at a loss for an opinion. I some thoughts, but want to
> >>>>> confirm assumptions and ideas against the solidfire,
> disk_io_throttle,
> >>>>> and
> >>>>> object_store branches. I hope to collect them in a coherent form
> >>>>> tomorrow (5 June 2013).
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>>
> >>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>> > "So, in essence, the SolidFire plugin introduces the notion of a
> >>>>> managed
> >>>>> > iSCSI device and provisioned IOPS."
> >>>>> >
> >>>>> > Technically, the SolidFire plug-in just introduces the notion of
> >>>>> > provisioned storage IOPS.
> >>>>> >
> >>>>> > The storage framework that leverages the plug-in was incomplete, so
> >>>>> I had
> >>>>> > to try to add in the notion of a managed iSCSI device.
> >>>>> >
> >>>>> > I appreciate all the time you've been spending on this. :) Do you
> >>>>> have a
> >>>>> > recommendation as to how we should accomplish what you're looking
> >>>>> for?
> >>>>> >
> >>>>> > Thanks!
> >>>>> >
> >>>>> >
> >>>>> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>> >
> >>>>> >> Mike,
> >>>>> >>
> >>>>> >> So, in essence, the SolidFire plugin introduces the notion of a
> >>>>> managed
> >>>>> >> iSCSI device and provisioned IOPS.
> >>>>> >>
> >>>>> >> I want to see a separation of the management capabilities (i.e.
> can
> >>>>> a
> >>>>> >> device be managed/does an operator want it managed by CloudStack)
> >>>>> from the
> >>>>> >> storage protocol. Ideally, we should end up with a semantic that
> >>>>> will
> >>>>> >> allow any type of storage device to be managed. I also want to
> make
> >>>>> >> progress on decoupling the storage types from the hypervisor
> >>>>> definitions.
> >>>>> >>
> >>>>> >> Thanks,
> >>>>> >> -John
> >>>>> >>
> >>>>> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>> >> wrote:
> >>>>> >>
> >>>>> >>> Hi John,
> >>>>> >>>
> >>>>> >>> No problem. Answers are below in red.
> >>>>> >>>
> >>>>> >>> Thanks!
> >>>>> >>>
> >>>>> >>>
> >>>>> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jburwell@basho.com
> >
> >>>>> wrote:
> >>>>> >>>
> >>>>> >>>> Mike,
> >>>>> >>>>
> >>>>> >>>> Could you please answer the following questions for me with
> >>>>> regards to
> >>>>> >> the
> >>>>> >>>> operation of the SolidFire plugin:
> >>>>> >>>>
> >>>>> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
> >>>>> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
> >>>>> >>> 1 SAN volume : 1 LUN
> >>>>> >>>
> >>>>> >>>> What is the cardinality between SAN Volumes and CloudStack
> >>>>> Volumes?
> >>>>> >>> 1 SAN volume : 1 CloudStack volume
> >>>>> >>>
> >>>>> >>>> Are the LUN(s) created by the management server or externally by
> >>>>> the
> >>>>> >>>> operator?
> >>>>> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN
> >>>>> LUN) is
> >>>>> >>> created by the management server (via the plug-in) the first time
> >>>>> the
> >>>>> >>> CloudStack volume is attached to a hypervisor.
> >>>>> >>>
> >>>>> >>> If you don't want to use the SolidFire plug-in, but still want to
> >>>>> use a
> >>>>> >>> SolidFire volume (LUN), you can do this already today (prior to
> >>>>> 4.2). The
> >>>>> >>> admin manually creates the SAN volume and - in this case -
> >>>>> multiple VMs
> >>>>> >> and
> >>>>> >>> data disks can share this SAN volume. While you can do this
> today,
> >>>>> it is
> >>>>> >>> not useful if you want to enforce storage QoS.
> >>>>> >>>
> >>>>> >>>> Are the SAN volumes by the management server or externally by
> the
> >>>>> >> operator?
> >>>>> >>> When the SolidFire plug-in is used, the SAN volumes are
> completely
> >>>>> >> managed
> >>>>> >>> by the management server (via the plug-in). There is no admin
> >>>>> >> interaction.
> >>>>> >>> This allows for a 1:1 mapping between a SAN volume and a
> CloudStack
> >>>>> >> volume,
> >>>>> >>> which is necessary for any storage vendor that supports true QoS.
> >>>>> >>>
> >>>>> >>>>
> >>>>> >>>> I would like to clarify how these pieces are related and
> expected
> >>>>> to
> >>>>> >>>> operate.
> >>>>> >>>>
> >>>>> >>>> Thanks,
> >>>>> >>>> -John
> >>>>> >>>>
> >>>>> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>> >> mike.tutkowski@solidfire.com>
> >>>>> >>>> wrote:
> >>>>> >>>>
> >>>>> >>>>> "In particular, how do we ensure that multiple VMs with
> >>>>> provisioned
> >>>>> >> IOPS
> >>>>> >>>>> won't be cut off by the underlying storage."
> >>>>> >>>>>
> >>>>> >>>>> In the storage QoS world, we need to map a single SAN volume
> >>>>> (LUN) to a
> >>>>> >>>>> single CloudStack volume. We cannot have multiple CloudStack
> >>>>> volumes
> >>>>> >>>>> sharing a single SAN volume and still guarantee QoS.
> >>>>> >>>>>
> >>>>> >>>>> If the user wants to have a single SAN volume house more than
> one
> >>>>> >>>>> CloudStack volume, then can do that today without any of my
> >>>>> plug-in
> >>>>> >> code.
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >>>>> >>>> mike.tutkowski@solidfire.com
> >>>>> >>>>>> wrote:
> >>>>> >>>>>
> >>>>> >>>>>> "The administrator will allocate a SAN volume for CloudStack's
> >>>>> use
> >>>>> >> onto
> >>>>> >>>>>> which CloudStack volumes will be created."
> >>>>> >>>>>>
> >>>>> >>>>>> I think we crossed e-mails. :)
> >>>>> >>>>>>
> >>>>> >>>>>> Check out my recent e-mail on this.
> >>>>> >>>>>>
> >>>>> >>>>>>
> >>>>> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>> >>>> wrote:
> >>>>> >>>>>>
> >>>>> >>>>>>> Mike,
> >>>>> >>>>>>>
> >>>>> >>>>>>> You are coming to the part which concerns me -- concepts from
> >>>>> the
> >>>>> >>>>>>> hypervisor are leaking into storage layer.
> >>>>> >>>>>>>
> >>>>> >>>>>>> Thanks,
> >>>>> >>>>>>> -John
> >>>>> >>>>>>>
> >>>>> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>>> >>>> mike.tutkowski@solidfire.com>
> >>>>> >>>>>>> wrote:
> >>>>> >>>>>>>
> >>>>> >>>>>>>> The weird part is that the iSCSI type is today only used (as
> >>>>> far as
> >>>>> >> I
> >>>>> >>>>>>> know)
> >>>>> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
> >>>>> >>>>>>>>
> >>>>> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses
> the
> >>>>> vmfs
> >>>>> >>>> type.
> >>>>> >>>>>>>>
> >>>>> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
> >>>>> >>>>>>> SharedMountPoint
> >>>>> >>>>>>>> type.
> >>>>> >>>>>>>>
> >>>>> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a
> >>>>> new type
> >>>>> >>>> of
> >>>>> >>>>>>>> storage to describe this dynamic ability Edison added into
> the
> >>>>> >> storage
> >>>>> >>>>>>>> framework. Maybe it should be more specificy, though:
> >>>>> Dynamic_iSCSI
> >>>>> >>>>>>> versus,
> >>>>> >>>>>>>> say, Dynamic_FC.
> >>>>> >>>>>>>>
> >>>>> >>>>>>>>
> >>>>> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>> >>>>>>> mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>> wrote:
> >>>>> >>>>>>>>
> >>>>> >>>>>>>>> "The storage device itself shouldn't know or care that it
> is
> >>>>> being
> >>>>> >>>> used
> >>>>> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it
> >>>>> is
> >>>>> >>>>>>> storing."
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in
> >>>>> needs to
> >>>>> >>>>>>> call
> >>>>> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows
> >>>>> to
> >>>>> >> treat
> >>>>> >>>>>>> it as
> >>>>> >>>>>>>>> such.
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>> I'm totally open to removing that constraint and just
> >>>>> calling it
> >>>>> >>>> iSCSI
> >>>>> >>>>>>> or
> >>>>> >>>>>>>>> whatever. We would just need a way for the hypervisor
> attach
> >>>>> logic
> >>>>> >> to
> >>>>> >>>>>>>>> detect this new requirement and perform the necessary
> >>>>> activities.
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>> >>>>>>> wrote:
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>>> Mike,
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>> See my responses in-line.
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>> -John
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>> >>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>>> I'm trying to picture this:
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a
> device,
> >>>>> an
> >>>>> >>>>>>> operator
> >>>>> >>>>>>>>>> may
> >>>>> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the
> device
> >>>>> is
> >>>>> >>>> shared
> >>>>> >>>>>>> by
> >>>>> >>>>>>>>>>> many services, and the operator has chosen to dedicate
> >>>>> only a
> >>>>> >>>> portion
> >>>>> >>>>>>>>>> of it
> >>>>> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
> >>>>> plug-in
> >>>>> >>>>>>> (before
> >>>>> >>>>>>>>>> an
> >>>>> >>>>>>>>>>> attach request to the hypervisor) would work unless the
> >>>>> >> hypervisor
> >>>>> >>>>>>>>>> consumes
> >>>>> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>> My thinking is that, independent of CloudStack, an
> operator
> >>>>> >>>> allocates
> >>>>> >>>>>>> a
> >>>>> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a
> >>>>> LUN. They
> >>>>> >>>>>>> simply
> >>>>> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but
> >>>>> not allow
> >>>>> >>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>> As the attach logic stands prior to my changes, we would
> be
> >>>>> >> passing
> >>>>> >>>>>>> in a
> >>>>> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor
> >>>>> support
> >>>>> >>>> (like
> >>>>> >>>>>>> an
> >>>>> >>>>>>>>>> SR)
> >>>>> >>>>>>>>>>> and the logic will fail.
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>> Are you thinking we should maybe have the storage
> framework
> >>>>> >> itself
> >>>>> >>>>>>>>>> detect
> >>>>> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor
> >>>>> side and
> >>>>> >>>>>>> have
> >>>>> >>>>>>>>>> it
> >>>>> >>>>>>>>>>> call into the agent code specifically to create the SR
> >>>>> before the
> >>>>> >>>>>>> attach
> >>>>> >>>>>>>>>>> logic runs in the agent code?
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>> I think the hypervisor management plugin should have a
> rich
> >>>>> enough
> >>>>> >>>>>>>>>> interface to storage to determine available for volume
> >>>>> storage.
> >>>>> >> For
> >>>>> >>>>>>> Xen,
> >>>>> >>>>>>>>>> this interface would allow the interrogation of the device
> >>>>> to
> >>>>> >>>>>>> determine the
> >>>>> >>>>>>>>>> SR is present. The storage device itself shouldn't know
> >>>>> or care
> >>>>> >>>>>>> that it
> >>>>> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer
> >>>>> questions
> >>>>> >>>>>>> about it
> >>>>> >>>>>>>>>> is storing.
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>>> So, the flow is as follows:
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a
> >>>>> type of
> >>>>> >>>>>>>>>> so-called
> >>>>> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
> >>>>> Storage shows
> >>>>> >>>> up
> >>>>> >>>>>>> in
> >>>>> >>>>>>>>>> the
> >>>>> >>>>>>>>>>>> applicable zone.
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>> * The admin creates a Disk Offering that references the
> >>>>> storage
> >>>>> >>>> tag
> >>>>> >>>>>>> of
> >>>>> >>>>>>>>>> the
> >>>>> >>>>>>>>>>>> newly created Primary Storage.
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads
> to
> >>>>> a new
> >>>>> >>>> row
> >>>>> >>>>>>> in
> >>>>> >>>>>>>>>> the
> >>>>> >>>>>>>>>>>> cloud.volumes table.
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
> >>>>> (attach
> >>>>> >>>> disk).
> >>>>> >>>>>>>>>> This
> >>>>> >>>>>>>>>>>> leads to the storage framework calling the plug-in to
> >>>>> create a
> >>>>> >> new
> >>>>> >>>>>>>>>> volume
> >>>>> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in
> >>>>> also
> >>>>> >>>> updates
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of
> >>>>> the SAN
> >>>>> >>>>>>>>>> volume).
> >>>>> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
> >>>>> volume is in
> >>>>> >>>> the
> >>>>> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be
> >>>>> in the
> >>>>> >>>>>>> 'Ready'
> >>>>> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code
> >>>>> will not
> >>>>> >> be
> >>>>> >>>>>>>>>> called
> >>>>> >>>>>>>>>>>> again to create this SAN volume.
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
> >>>>> CloudStack
> >>>>> >>>>>>> volume
> >>>>> >>>>>>>>>> to
> >>>>> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor
> data
> >>>>> >>>> structure
> >>>>> >>>>>>>>>> (ex.
> >>>>> >>>>>>>>>>>> an SR on XenServer).
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume,
> >>>>> in these
> >>>>> >>>>>>>>>>>>> definitions, refers to the physical allocation on the
> >>>>> device,
> >>>>> >>>>>>>>>> correct?"
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in
> >>>>> CloudStack, I
> >>>>> >>>>>>> try to
> >>>>> >>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>> Hi John,
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm
> >>>>> having a
> >>>>> >>>> hard
> >>>>> >>>>>>>>>> time
> >>>>> >>>>>>>>>>>>>> envisioning it.
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation
> >>>>> as he
> >>>>> >> was
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>> What do you think?
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>> Thanks!
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >>>>> >>>> jburwell@basho.com
> >>>>> >>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>> Mike,
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts
> --
> >>>>> >> storage
> >>>>> >>>>>>>>>> device
> >>>>> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we
> >>>>> are
> >>>>> >>>>>>>>>> communicating with
> >>>>> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
> >>>>> volumes
> >>>>> >>>>>>> (Dynamic)
> >>>>> >>>>>>>>>> on the
> >>>>> >>>>>>>>>>>>>>> device while the other requires the volume to be
> >>>>> volume to be
> >>>>> >>>>>>>>>> managed
> >>>>> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we
> >>>>> are in
> >>>>> >>>>>>> sync on
> >>>>> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to
> >>>>> the
> >>>>> >>>> physical
> >>>>> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we
> must
> >>>>> be
> >>>>> >> able
> >>>>> >>>> to
> >>>>> >>>>>>>>>>>>>>> communicate with a storage device to move bits from
> >>>>> one place
> >>>>> >>>> to
> >>>>> >>>>>>>>>> another,
> >>>>> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage
> >>>>> device
> >>>>> >> may
> >>>>> >>>> be
> >>>>> >>>>>>>>>> able to
> >>>>> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
> >>>>> unmanaged
> >>>>> >>>> iSCSI
> >>>>> >>>>>>>>>> device
> >>>>> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a
> managed
> >>>>> >>>> SolidFire
> >>>>> >>>>>>>>>> iSCSI
> >>>>> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating
> >>>>> LUNs and
> >>>>> >>>>>>> storing
> >>>>> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
> >>>>> manage a
> >>>>> >>>>>>> device,
> >>>>> >>>>>>>>>> an
> >>>>> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by
> CloudStack
> >>>>> (e.g.
> >>>>> >>>> the
> >>>>> >>>>>>>>>> device is
> >>>>> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen
> to
> >>>>> >>>> dedicate
> >>>>> >>>>>>>>>> only a
> >>>>> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make
> >>>>> sense?
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
> >>>>> >> appropriate
> >>>>> >>>>>>> to
> >>>>> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType,
> >>>>> add the
> >>>>> >>>>>>> notion
> >>>>> >>>>>>>>>> of a
> >>>>> >>>>>>>>>>>>>>> storage device with an attached driver that indicates
> >>>>> whether
> >>>>> >>>> or
> >>>>> >>>>>>>>>> not is
> >>>>> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
> >>>>> >>>> representing a
> >>>>> >>>>>>>>>> physical
> >>>>> >>>>>>>>>>>>>>> allocation on a device separate that is associated
> >>>>> with a
> >>>>> >>>> volume.
> >>>>> >>>>>>>>>> With
> >>>>> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can
> declare
> >>>>> which
> >>>>> >>>>>>>>>> protocols they
> >>>>> >>>>>>>>>>>>>>> support and when they encounter a device managed by
> >>>>> >> CloudStack,
> >>>>> >>>>>>>>>> utilize the
> >>>>> >>>>>>>>>>>>>>> management operations exposed by the driver to
> automate
> >>>>> >>>>>>> allocation.
> >>>>> >>>>>>>>>> If
> >>>>> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit
> >>>>> down and
> >>>>> >>>>>>> drill
> >>>>> >>>>>>>>>> down to
> >>>>> >>>>>>>>>>>>>>> a more detailed design.
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI
> type
> >>>>> and
> >>>>> >> the
> >>>>> >>>>>>>>>> Dynamic
> >>>>> >>>>>>>>>>>>>>> type:
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
> >>>>> Primary
> >>>>> >>>> Storage
> >>>>> >>>>>>>>>> based
> >>>>> >>>>>>>>>>>>>>> on
> >>>>> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
> >>>>> volume must
> >>>>> >>>>>>> exist
> >>>>> >>>>>>>>>> on
> >>>>> >>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
> >>>>> CloudStack
> >>>>> >>>> volumes
> >>>>> >>>>>>> are
> >>>>> >>>>>>>>>>>>>>> created
> >>>>> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the
> scenes.
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
> >>>>> Primary
> >>>>> >>>> Storage
> >>>>> >>>>>>>>>> based
> >>>>> >>>>>>>>>>>>>>> on a
> >>>>> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
> >>>>> storage
> >>>>> >>>>>>> system
> >>>>> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
> >>>>> framework). When
> >>>>> >> a
> >>>>> >>>>>>> user
> >>>>> >>>>>>>>>>>>>>> wants to
> >>>>> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
> >>>>> framework
> >>>>> >>>> tells
> >>>>> >>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done,
> >>>>> the
> >>>>> >> attach
> >>>>> >>>>>>>>>> logic
> >>>>> >>>>>>>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor
> >>>>> data
> >>>>> >>>>>>> structure
> >>>>> >>>>>>>>>>>>>>> exists
> >>>>> >>>>>>>>>>>>>>>> at this point because the volume was just created.
> The
> >>>>> >>>>>>> hypervisor
> >>>>> >>>>>>>>>> data
> >>>>> >>>>>>>>>>>>>>>> structure must be created.
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have
> >>>>> defined them
> >>>>> >>>> up
> >>>>> >>>>>>>>>> front
> >>>>> >>>>>>>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>>> you. :)
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex.
> >>>>> an admin
> >>>>> >>>>>>>>>> creates a
> >>>>> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure
> is
> >>>>> >> created
> >>>>> >>>>>>> to
> >>>>> >>>>>>>>>>>>>>> consume
> >>>>> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that
> hypervisor
> >>>>> data
> >>>>> >>>>>>>>>> structure
> >>>>> >>>>>>>>>>>>>>> is
> >>>>> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later
> placed
> >>>>> on
> >>>>> >> this
> >>>>> >>>>>>>>>>>>>>> hypervisor
> >>>>> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the
> attach
> >>>>> logic
> >>>>> >>>>>>> assumes
> >>>>> >>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and
> >>>>> simply
> >>>>> >>>>>>> attaches
> >>>>> >>>>>>>>>>>>>>> the VDI
> >>>>> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
> >>>>> question.
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
> >>>>> Instead
> >>>>> >>>> of
> >>>>> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
> >>>>> volume on
> >>>>> >> a
> >>>>> >>>>>>> SAN,
> >>>>> >>>>>>>>>>>>>>> but the
> >>>>> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
> >>>>> created
> >>>>> >>>>>>> when an
> >>>>> >>>>>>>>>>>>>>> attach
> >>>>> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes
> >>>>> have not
> >>>>> >>>>>>> been
> >>>>> >>>>>>>>>>>>>>> pre-hooked
> >>>>> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin.
> >>>>> Once
> >>>>> >> the
> >>>>> >>>>>>>>>> attach
> >>>>> >>>>>>>>>>>>>>> logic
> >>>>> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume,
> it
> >>>>> >> attaches
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>>>>> one and
> >>>>> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>>>> >>>>>>> jburwell@basho.com>
> >>>>> >>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>> Mike,
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
> >>>>> attach
> >>>>> >>>> behavior
> >>>>> >>>>>>>>>>>>>>> works in
> >>>>> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
> >>>>> difference.
> >>>>> >>>>>>> Another
> >>>>> >>>>>>>>>>>>>>> way to
> >>>>> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
> >>>>> Dynamic
> >>>>> >>>>>>> storage
> >>>>> >>>>>>>>>>>>>>> pool type?
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>> >>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used
> >>>>> by
> >>>>> >>>>>>> XenServer
> >>>>> >>>>>>>>>>>>>>> when you
> >>>>> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly
> >>>>> based on
> >>>>> >> an
> >>>>> >>>>>>>>>> iSCSI
> >>>>> >>>>>>>>>>>>>>>>>> target.
> >>>>> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
> >>>>> >> hypervisor
> >>>>> >>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>> creating a
> >>>>> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
> >>>>> >> CloudStack
> >>>>> >>>>>>> does
> >>>>> >>>>>>>>>>>>>>> that
> >>>>> >>>>>>>>>>>>>>>>>> part
> >>>>> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
> >>>>> XenServer.
> >>>>> >> For
> >>>>> >>>>>>> all
> >>>>> >>>>>>>>>>>>>>> other
> >>>>> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor
> >>>>> and
> >>>>> >>>> perform
> >>>>> >>>>>>>>>> this
> >>>>> >>>>>>>>>>>>>>> step
> >>>>> >>>>>>>>>>>>>>>>>>> manually.
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>> >>>>>>>>>> jburwell@basho.com>
> >>>>> >>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
> >>>>> between
> >>>>> >>>> the
> >>>>> >>>>>>>>>>>>>>> ISCSI and
> >>>>> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered
> Dynamic?
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>> >>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>> >>>>>>>>>>>>>>>>>> class
> >>>>> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
> >>>>> >> overlay
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool
> >>>>> is set
> >>>>> >>>> up
> >>>>> >>>>>>>>>> by
> >>>>> >>>>>>>>>>>>>>>>>>>>> customers.
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage
> (ex.
> >>>>> >>>>>>>>>> SolidFire)
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> }
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> }
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> }
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski
> <
> >>>>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company
> >>>>> wants
> >>>>> >> to
> >>>>> >>>>>>>>>>>>>>> implement a
> >>>>> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
> >>>>> feature. It
> >>>>> >>>>>>> would
> >>>>> >>>>>>>>>> be
> >>>>> >>>>>>>>>>>>>>>>>> dynamic,
> >>>>> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need
> only
> >>>>> >>>>>>> implement a
> >>>>> >>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to
> >>>>> the
> >>>>> >>>>>>>>>>>>>>> hypervisor-attach
> >>>>> >>>>>>>>>>>>>>>>>>>> logic
> >>>>> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike
> Tutkowski <
> >>>>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
> >>>>> SolidFire
> >>>>> >>>>>>> specific.
> >>>>> >>>>>>>>>>>>>>> It is
> >>>>> >>>>>>>>>>>>>>>>>> not.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
> >>>>> dynamic,
> >>>>> >>>>>>>>>> zone-wide
> >>>>> >>>>>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this
> >>>>> kind of
> >>>>> >>>>>>>>>> storage).
> >>>>> >>>>>>>>>>>>>>> This
> >>>>> >>>>>>>>>>>>>>>>>>>> kind of
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
> >>>>> framework
> >>>>> >>>>>>>>>> changes.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported
> >>>>> the
> >>>>> >>>>>>> creation
> >>>>> >>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>> deletion
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
> >>>>> visited with
> >>>>> >>>> him
> >>>>> >>>>>>> in
> >>>>> >>>>>>>>>>>>>>> Portland
> >>>>> >>>>>>>>>>>>>>>>>>>> back
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
> >>>>> complete. We
> >>>>> >>>>>>> realized
> >>>>> >>>>>>>>>>>>>>> there
> >>>>> >>>>>>>>>>>>>>>>>> was
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these
> volumes
> >>>>> unless
> >>>>> >>>> the
> >>>>> >>>>>>>>>>>>>>> attach
> >>>>> >>>>>>>>>>>>>>>>>> logic
> >>>>> >>>>>>>>>>>>>>>>>>>> was
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage
> >>>>> and
> >>>>> >>>> create
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>> appropriate
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell
> <
> >>>>> >>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation
> >>>>> in the
> >>>>> >>>>>>> Storage
> >>>>> >>>>>>>>>>>>>>> layer
> >>>>> >>>>>>>>>>>>>>>>>>>> would
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect
> >>>>> to see
> >>>>> >> the
> >>>>> >>>>>>>>>> Storage
> >>>>> >>>>>>>>>>>>>>>>>>>> services
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> >>>>> >> composed/driven
> >>>>> >>>>>>> by
> >>>>> >>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
> >>>>> structures
> >>>>> >>>> per
> >>>>> >>>>>>>>>> their
> >>>>> >>>>>>>>>>>>>>>>>> needs.
> >>>>> >>>>>>>>>>>>>>>>>>>> If
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going
> to
> >>>>> end
> >>>>> >>>> with a
> >>>>> >>>>>>>>>>>>>>> massive
> >>>>> >>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system
> increasingly
> >>>>> >>>>>>> difficult to
> >>>>> >>>>>>>>>>>>>>>>>> maintain
> >>>>> >>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
> >>>>> specific
> >>>>> >>>>>>>>>> SolidFire
> >>>>> >>>>>>>>>>>>>>> code
> >>>>> >>>>>>>>>>>>>>>>>> is
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
> >>>>> Tutkowski <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup
> method
> >>>>> for
> >>>>> >>>>>>>>>> XenServer,
> >>>>> >>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>> admin
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
> >>>>> system,
> >>>>> >> then
> >>>>> >>>> go
> >>>>> >>>>>>>>>> into
> >>>>> >>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of
> the
> >>>>> >> volume
> >>>>> >>>>>>> (ex.
> >>>>> >>>>>>>>>> a
> >>>>> >>>>>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
> >>>>> ESX(i)).
> >>>>> >>>> VMs
> >>>>> >>>>>>> and
> >>>>> >>>>>>>>>>>>>>> data
> >>>>> >>>>>>>>>>>>>>>>>> disks
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework,
> storage
> >>>>> need
> >>>>> >> no
> >>>>> >>>>>>>>>> longer
> >>>>> >>>>>>>>>>>>>>> be so
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1
> relationship
> >>>>> >> between
> >>>>> >>>> a
> >>>>> >>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>> system's
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary
> for
> >>>>> >> storage
> >>>>> >>>>>>>>>> Quality
> >>>>> >>>>>>>>>>>>>>> of
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called
> to
> >>>>> >>>>>>> dynamically
> >>>>> >>>>>>>>>>>>>>> create
> >>>>> >>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did
> >>>>> not
> >>>>> >>>> address
> >>>>> >>>>>>> is
> >>>>> >>>>>>>>>> in
> >>>>> >>>>>>>>>>>>>>>>>> creating
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
> >>>>> structure
> >>>>> >> when
> >>>>> >>>>>>>>>>>>>>> performing an
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it
> >>>>> to do
> >>>>> >> so.
> >>>>> >>>>>>> I've
> >>>>> >>>>>>>>>>>>>>> got
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i)
> in
> >>>>> my
> >>>>> >>>> sandbox
> >>>>> >>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>> can
> >>>>> >>>>>>>>>>>>>>>>>>>> submit
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
> >>>>> Tutkowski <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
> >>>>> hypervisor
> >>>>> >>>>>>> agnostic.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic
> >>>>> (in the
> >>>>> >>>>>>> agent
> >>>>> >>>>>>>>>>>>>>> code).
> >>>>> >>>>>>>>>>>>>>>>>> The
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have
> it
> >>>>> >> create a
> >>>>> >>>>>>>>>> volume
> >>>>> >>>>>>>>>>>>>>> as
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume
> to
> >>>>> a
> >>>>> >>>>>>> hypervisor,
> >>>>> >>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>> attach
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's
> >>>>> being
> >>>>> >>>>>>> invoked on
> >>>>> >>>>>>>>>>>>>>>>>> zone-wide
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created)
> and
> >>>>> >> create,
> >>>>> >>>>>>>>>> say, a
> >>>>> >>>>>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore
> >>>>> (for
> >>>>> >>>>>>> VMware) to
> >>>>> >>>>>>>>>>>>>>> make
> >>>>> >>>>>>>>>>>>>>>>>> use
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time
> recently
> >>>>> making
> >>>>> >>>> the
> >>>>> >>>>>>>>>>>>>>> attach
> >>>>> >>>>>>>>>>>>>>>>>> logic
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
> >>>>> Burwell <
> >>>>> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage
> driver
> >>>>> is
> >>>>> >>>>>>> hypervisor
> >>>>> >>>>>>>>>>>>>>>>>> specific?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike
> Tutkowski
> >>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support
> >>>>> all
> >>>>> >>>>>>> hypervisors
> >>>>> >>>>>>>>>>>>>>> that
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time
> >>>>> for 4.2
> >>>>> >> to
> >>>>> >>>>>>> get
> >>>>> >>>>>>>>>> KVM
> >>>>> >>>>>>>>>>>>>>> in.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports
> XenServer.
> >>>>> >>>>>>> Depending on
> >>>>> >>>>>>>>>>>>>>> what
> >>>>> >>>>>>>>>>>>>>>>>> we do
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
> >>>>> working
> >>>>> >>>> for
> >>>>> >>>>>>>>>>>>>>> VMware in
> >>>>> >>>>>>>>>>>>>>>>>> my
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
> >>>>> regards
> >>>>> >> to
> >>>>> >>>>>>> Disk
> >>>>> >>>>>>>>>>>>>>>>>> Offerings.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
> >>>>> Jamison
> >>>>> >>>>>>> Damage
> >>>>> >>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in
> >>>>> the
> >>>>> >> patch
> >>>>> >>>>>>> cycle
> >>>>> >>>>>>>>>>>>>>> post
> >>>>> >>>>>>>>>>>>>>>>>> 4.2?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling
> >>>>> to
> >>>>> >> MASTER
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first,
> >>>>> then
> >>>>> >>>> mine.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it
> >>>>> is a
> >>>>> >> non
> >>>>> >>>>>>> issue
> >>>>> >>>>>>>>>>>>>>> as I
> >>>>> >>>>>>>>>>>>>>>>>> don't
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei
> ZHOU
> >>>>> <
> >>>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
> >>>>> billing
> >>>>> >> now,
> >>>>> >>>>>>>>>> users
> >>>>> >>>>>>>>>>>>>>> need
> >>>>> >>>>>>>>>>>>>>>>>> to
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can
> get
> >>>>> the
> >>>>> >>>>>>>>>>>>>>> service_offering
> >>>>> >>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
> >>>>> >>>>>>> calculation.
> >>>>> >>>>>>>>>> Of
> >>>>> >>>>>>>>>>>>>>> course
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation
> value
> >>>>> of
> >>>>> >>>>>>> individual
> >>>>> >>>>>>>>>>>>>>> volume,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I
> can
> >>>>> work
> >>>>> >> on
> >>>>> >>>> it
> >>>>> >>>>>>>>>>>>>>> later. Do
> >>>>> >>>>>>>>>>>>>>>>>> you
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
> >>>>> question.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
> >>>>> >> features.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>> jburwell@basho.com>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU
> <
> >>>>> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
> >>>>> trying
> >>>>> >> to
> >>>>> >>>>>>>>>> adding
> >>>>> >>>>>>>>>>>>>>> more.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on
> >>>>> IO
> >>>>> >>>>>>> statistics
> >>>>> >>>>>>>>>>>>>>> rather
> >>>>> >>>>>>>>>>>>>>>>>> than
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
> >>>>> disk_io_stat if
> >>>>> >> you
> >>>>> >>>>>>> have
> >>>>> >>>>>>>>>>>>>>> time.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
> >>>>> bytes/iops
> >>>>> >>>>>>>>>> read/write
> >>>>> >>>>>>>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>>>> an
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are
> >>>>> billed
> >>>>> >>>> more
> >>>>> >>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>>>> volumes
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for
> those
> >>>>> >>>>>>> operations (
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I
> would
> >>>>> >> imagine
> >>>>> >>>>>>> our
> >>>>> >>>>>>>>>>>>>>> users
> >>>>> >>>>>>>>>>>>>>>>>> would
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
> >>>>> Could
> >>>>> >> an
> >>>>> >>>>>>>>>>>>>>> operator
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change?
> >>>>> KVM
> >>>>> >>>>>>> supports
> >>>>> >>>>>>>>>>>>>>> setting
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
> >>>>> machine
> >>>>> >>>> through
> >>>>> >>>>>>>>>>>>>>> command
> >>>>> >>>>>>>>>>>>>>>>>> line.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing
> >>>>> the
> >>>>> >>>>>>> parameters
> >>>>> >>>>>>>>>>>>>>> of a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk
> offering).
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level.
> I
> >>>>> >>>> apologize
> >>>>> >>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>> being
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
> >>>>> algorithms
> >>>>> >> with a
> >>>>> >>>>>>> set
> >>>>> >>>>>>>>>> of
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
> >>>>> provided by a
> >>>>> >>>>>>>>>> resource?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it
> is
> >>>>> >> better
> >>>>> >>>> to
> >>>>> >>>>>>>>>>>>>>> commit
> >>>>> >>>>>>>>>>>>>>>>>> Mike's
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
> >>>>> consider
> >>>>> >>>> the
> >>>>> >>>>>>>>>>>>>>>>>> limitation in
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a
> later
> >>>>> >>>> response
> >>>>> >>>>>>> to
> >>>>> >>>>>>>>>>>>>>> Mike
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
> >>>>> features. I
> >>>>> >>>> think
> >>>>> >>>>>>>>>> that
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
> >>>>> SolidFire, but
> >>>>> >> I
> >>>>> >>>>>>> think
> >>>>> >>>>>>>>>>>>>>> we need
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
> >>>>> (possibly
> >>>>> >>>> have
> >>>>> >>>>>>>>>> have
> >>>>> >>>>>>>>>>>>>>>>>> solidfire
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on
> >>>>> this
> >>>>> >>>> issue.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
> >>>>> jburwell@basho.com>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are
> >>>>> the
> >>>>> >>>>>>> following:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
> >>>>> >> policies
> >>>>> >>>> be
> >>>>> >>>>>>>>>>>>>>> captured
> >>>>> >>>>>>>>>>>>>>>>>> in
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
> >>>>> >> purposes,
> >>>>> >>>>>>>>>>>>>>> expressing
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
> >>>>> interface model
> >>>>> >>>> for
> >>>>> >>>>>>>>>>>>>>> reasoning
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned
> >>>>> IOPS
> >>>>> >>>>>>>>>>>>>>> configurations
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
> >>>>> reconciled?
> >>>>> >> In
> >>>>> >>>>>>>>>>>>>>> particular,
> >>>>> >>>>>>>>>>>>>>>>>> a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for
> >>>>> more
> >>>>> >> IOPS
> >>>>> >>>>>>>>>>>>>>> configured
> >>>>> >>>>>>>>>>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured
> to
> >>>>> >>>> deliver.
> >>>>> >>>>>>>>>>>>>>> Another
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a
> >>>>> VM and a
> >>>>> >>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>> device at
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
> >>>>> storage
> >>>>> >>>>>>> device
> >>>>> >>>>>>>>>>>>>>>>>> introduces
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
> >>>>> Tutkowski
> >>>>> >> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
> >>>>> >>>> controlling
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>>>>> max
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on
> controlling
> >>>>> IOPS
> >>>>> >>>> from
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>>>>> storage
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM,
> John
> >>>>> >> Burwell
> >>>>> >>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
> >>>>> should be
> >>>>> >>>>>>> merged
> >>>>> >>>>>>>>>>>>>>> until all
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it
> is
> >>>>> ready
> >>>>> >> to
> >>>>> >>>>>>> turn
> >>>>> >>>>>>>>>>>>>>> over to
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write
> >>>>> ops issue
> >>>>> >>>> is
> >>>>> >>>>>>>>>>>>>>> addressed
> >>>>> >>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria
> >>>>> has been
> >>>>> >>>>>>>>>>>>>>> satisfied.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >>>>> >>>> intersect/compliment
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> https://reviews.apache.org/r/11479/
> >>>>> )?
> >>>>> >> As I
> >>>>> >>>>>>>>>>>>>>> understand
> >>>>> >>>>>>>>>>>>>>>>>> it
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I
> >>>>> would
> >>>>> >>>> like
> >>>>> >>>>>>> to
> >>>>> >>>>>>>>>>>>>>> ensure
> >>>>> >>>>>>>>>>>>>>>>>> we
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in
> >>>>> KVM and
> >>>>> >>>>>>>>>>>>>>> SolidFire are
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei
> >>>>> ZHOU <
> >>>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
> >>>>> >> wido@widodh.nl
> >>>>> >>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU
> >>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the
> features
> >>>>> into
> >>>>> >>>>>>> master.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for
> technical
> >>>>> >> reason
> >>>>> >>>>>>> till
> >>>>> >>>>>>>>>>>>>>> Monday,
> >>>>> >>>>>>>>>>>>>>>>>> I
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just
> want
> >>>>> to
> >>>>> >>>> mention
> >>>>> >>>>>>>>>> again
> >>>>> >>>>>>>>>>>>>>>>>> that we
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so
> >>>>> that we
> >>>>> >>>>>>> never
> >>>>> >>>>>>>>>>>>>>> release
> >>>>> >>>>>>>>>>>>>>>>>> a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the
> I/O
> >>>>> >>>> throttling
> >>>>> >>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>> that's
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
> >>>>> legacy from
> >>>>> >>>> day
> >>>>> >>>>>>>>>> #1.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>>> >>>> wido@widodh.nl>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John
> Burwell
> >>>>> >> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
> >>>>> discretely
> >>>>> >>>>>>> support
> >>>>> >>>>>>>>>>>>>>> read
> >>>>> >>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed
> >>>>> later
> >>>>> >>>> because
> >>>>> >>>>>>> I
> >>>>> >>>>>>>>>>>>>>> don't see
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
> >>>>> write.
> >>>>> >>>>>>>>>> Therefore, we
> >>>>> >>>>>>>>>>>>>>>>>> would
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to
> >>>>> break
> >>>>> >>>>>>> backwards
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into
> >>>>> master
> >>>>> >>>> now
> >>>>> >>>>>>> so
> >>>>> >>>>>>>>>>>>>>> that it
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write
> IOps
> >>>>> >> before
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>> 4.2
> >>>>> >>>>>>>>>>>>>>>>>> release
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and
> Write
> >>>>> >> instead
> >>>>> >>>>>>> of
> >>>>> >>>>>>>>>>>>>>> Total
> >>>>> >>>>>>>>>>>>>>>>>> IOps.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st
> feature
> >>>>> >> freeze
> >>>>> >>>>>>> date.
> >>>>> >>>>>>>>>>>>>>> But if
> >>>>> >>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't
> be
> >>>>> >>>> necessary
> >>>>> >>>>>>>>>> to do
> >>>>> >>>>>>>>>>>>>>>>>> that.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there
> >>>>> is no
> >>>>> >>>>>>>>>> association
> >>>>> >>>>>>>>>>>>>>>>>> between
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido
> >>>>> den
> >>>>> >>>>>>> Hollander <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU
> >>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
> >>>>> >> Finally I
> >>>>> >>>>>>>>>>>>>>> decided to
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write
> mainly
> >>>>> >> because
> >>>>> >>>>>>> the
> >>>>> >>>>>>>>>>>>>>> network
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
> >>>>> received
> >>>>> >>>>>>> bytes as
> >>>>> >>>>>>>>>>>>>>> well.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
> >>>>> Networking and
> >>>>> >>>> disk
> >>>>> >>>>>>> I/O
> >>>>> >>>>>>>>>>>>>>>>>> completely
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in
> >>>>> most
> >>>>> >>>>>>>>>> situations
> >>>>> >>>>>>>>>>>>>>> then
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
> >>>>> copy-paste
> >>>>> >>>> work.
> >>>>> >>>>>>> It
> >>>>> >>>>>>>>>>>>>>> could be
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
> >>>>> feature
> >>>>> >>>> freeze,
> >>>>> >>>>>>> I
> >>>>> >>>>>>>>>> will
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't
> >>>>> miss. But
> >>>>> >>>> if
> >>>>> >>>>>>> it
> >>>>> >>>>>>>>>>>>>>> goes
> >>>>> >>>>>>>>>>>>>>>>>> into
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we
> don't
> >>>>> >> release
> >>>>> >>>>>>> with
> >>>>> >>>>>>>>>>>>>>> only
> >>>>> >>>>>>>>>>>>>>>>>> total
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse
> users.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>>> >>>> wido@widodh.nl>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU
> >>>>> wrote:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
> >>>>> disk_io_throttling
> >>>>> >>>>>>> branch
> >>>>> >>>>>>>>>> into
> >>>>> >>>>>>>>>>>>>>>>>> master.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge
> into
> >>>>> master
> >>>>> >>>> in
> >>>>> >>>>>>> 48
> >>>>> >>>>>>>>>>>>>>> hours.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on
> the
> >>>>> same
> >>>>> >>>>>>> storage
> >>>>> >>>>>>>>>>>>>>> device
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
> >>>>> >>>> limitation
> >>>>> >>>>>>> of
> >>>>> >>>>>>>>>>>>>>> device
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation,
> it
> >>>>> may
> >>>>> >>>> affect
> >>>>> >>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>> disk
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
> >>>>> storage
> >>>>> >>>> device.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum
> >>>>> rate
> >>>>> >> and
> >>>>> >>>>>>> limit
> >>>>> >>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>> disk
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make
> >>>>> no
> >>>>> >>>>>>> difference
> >>>>> >>>>>>>>>>>>>>> between
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting
> >>>>> both a
> >>>>> >>>>>>> different
> >>>>> >>>>>>>>>>>>>>> rate
> >>>>> >>>>>>>>>>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
> >>>>> users.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling
> >>>>> side you
> >>>>> >>>>>>>>>> collect
> >>>>> >>>>>>>>>>>>>>> both
> >>>>> >>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side
> >>>>> you only
> >>>>> >> go
> >>>>> >>>>>>> for
> >>>>> >>>>>>>>>> a
> >>>>> >>>>>>>>>>>>>>> global
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
> >>>>> expensive
> >>>>> >>>>>>> then
> >>>>> >>>>>>>>>> Read
> >>>>> >>>>>>>>>>>>>>>>>> IOps,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that
> an
> >>>>> admin
> >>>>> >>>>>>> would
> >>>>> >>>>>>>>>> set
> >>>>> >>>>>>>>>>>>>>> a
> >>>>> >>>>>>>>>>>>>>>>>> lower
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at
> this
> >>>>> >> point I
> >>>>> >>>>>>>>>> think
> >>>>> >>>>>>>>>>>>>>> it
> >>>>> >>>>>>>>>>>>>>>>>> would
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
> >>>>> mechanism in
> >>>>> >>>> place
> >>>>> >>>>>>> to
> >>>>> >>>>>>>>>>>>>>> support
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
> >>>>> setting
> >>>>> >>>>>>> different
> >>>>> >>>>>>>>>>>>>>> values
> >>>>> >>>>>>>>>>>>>>>>>> for
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up
> >>>>> and set
> >>>>> >>>> that
> >>>>> >>>>>>> as
> >>>>> >>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>> total
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you
> implemented
> >>>>> it
> >>>>> >> this
> >>>>> >>>>>>> way?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs
> (in
> >>>>> >>>>>>>>>> disk_offering,
> >>>>> >>>>>>>>>>>>>>> and
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps
> >>>>> and
> >>>>> >> iops)
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> >>>>> >> https://issues.apache.org/****
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >>>>> >
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>
> >>>>> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>> >>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>> >>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> ****
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://cwiki.apache.org/confluence/**>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>> >>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>
> >>>>> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>
> >>>>> >>>>
> >>>>> >>
> >>>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
> >>>>> execution
> >>>>> >>>>>>>>>> success?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
> >>>>> introduced?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
> >>>>> >>>>>>> integration)
> >>>>> >>>>>>>>>> is
> >>>>> >>>>>>>>>>>>>>>>>> included
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to
> >>>>> check for
> >>>>> >>>>>>>>>> potential
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS
> >>>>> rate on
> >>>>> >>>>>>>>>> CloudStack
> >>>>> >>>>>>>>>>>>>>> UI.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
> >>>>> destroy,
> >>>>> >>>> expunge.
> >>>>> >>>>>>>>>>>>>>> migrate,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>
> >>>>> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>> >>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
> >
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>> >>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://
> ****
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://cwiki.apache.org/confluence/**>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>> >>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>
> >>>>> >>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>
> >>>>> >>>>
> >>>>> >>
> >>>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >>>>> >
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> >>>>> >> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>> >>>>>>>>>>>>>>>>>> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
> >>>>> >
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> >>>>> >> <
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
> >>>>> SolidFire
> >>>>> >> Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses
> the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>> Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>> Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>>> Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> cloud<
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>> >>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> >> http://solidfire.com/solution/overview/?video=play
> >>>>> >>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play
> >>>>> >>>
> >>>>> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>> --
> >>>>> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>>>
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>>
> >>>>> >>>>>>>>>>> --
> >>>>> >>>>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>> >>>>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>>
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>> --
> >>>>> >>>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>>> *â„¢*
> >>>>> >>>>>>>>>
> >>>>> >>>>>>>>
> >>>>> >>>>>>>>
> >>>>> >>>>>>>>
> >>>>> >>>>>>>> --
> >>>>> >>>>>>>> *Mike Tutkowski*
> >>>>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>>>> o: 303.746.7302
> >>>>> >>>>>>>> Advancing the way the world uses the
> >>>>> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>>>> *â„¢*
> >>>>> >>>>>>>
> >>>>> >>>>>>>
> >>>>> >>>>>>
> >>>>> >>>>>>
> >>>>> >>>>>> --
> >>>>> >>>>>> *Mike Tutkowski*
> >>>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>>> o: 303.746.7302
> >>>>> >>>>>> Advancing the way the world uses the cloud<
> >>>>> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>>> *â„¢*
> >>>>> >>>>>>
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>> --
> >>>>> >>>>> *Mike Tutkowski*
> >>>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> >>>>> o: 303.746.7302
> >>>>> >>>>> Advancing the way the world uses the
> >>>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> >>>>> *â„¢*
> >>>>> >>>>
> >>>>> >>>>
> >>>>> >>>
> >>>>> >>>
> >>>>> >>> --
> >>>>> >>> *Mike Tutkowski*
> >>>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> >>> e: mike.tutkowski@solidfire.com
> >>>>> >>> o: 303.746.7302
> >>>>> >>> Advancing the way the world uses the
> >>>>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> >>> *â„¢*
> >>>>> >>
> >>>>> >>
> >>>>> >
> >>>>> >
> >>>>> > --
> >>>>> > *Mike Tutkowski*
> >>>>> > *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> > e: mike.tutkowski@solidfire.com
> >>>>> > o: 303.746.7302
> >>>>> > Advancing the way the world uses the
> >>>>> > cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> > *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
> >
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I believe I understand where you're going with this, John, and have been
re-working this section of code today.
I should be able to run it by you tomorrow.
Thanks for the comments,
Mike
On Thu, Jun 6, 2013 at 3:12 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> See my responses in-line below.
>
> Thanks,
> -John
>
> On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> Hi John,
>
> Thanks for the response.
>
> "I am fine with the VolumeManager determining whether or not a Volume is
> managed (i.e. not based on the StoragePoolType, but an actual isManaged
> method), and asking the device driver to allocate resources for the volume
> if it is managed."
>
> Are you thinking you'd like to see an isManaged() method added to the
> PrimaryDataStoreDriver interface? If it returns true, then the storage
> framework could call the manage() (or whatever name) method (which would be
> new to the PrimaryDataStoreDriver interface, as well) and this would call
> into a new method in the hypervisor code to create, say on XenServer, an SR?
>
>
> I would like to see canBeManaged() : boolean on DataStoreDriver. Since
> the notion of Volumes only pertains to primary storage, I would add a
> allocateStorage and deallocateStorage (Storage is a straw man term --
> something other than volume) methods to allocate/create/deallocate/delete
> underlying storage. To my mind, managed is a mutable property of DataStore
> which can be enabled if/when the underlying DataStoreDriver can be managed.
> This approach allows operators to override manageability of devices.
>
> In terms of orchestration/process flow for SR, the Xen plugin would be
> responsible for composing DataStore/Volume methods to create any
> directories or files necessary for the SR. There should be no dependencies
> from the Storage to the Hypervisor layer. As I said earlier, such circular
> dependencies will lead to a tangled, unmaintainable mess.
>
>
> Just want to make sure I'm on the same page with you.
>
> Thanks again, John
>
>
> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Fundamentally, we can't end up with a Storage layer that supports n
>> devices types with each specific behaviors of m hypervisors. Such a
>> scenario will create an unmaintainable and untestable beast. Therefore, my
>> thoughts and recommendations are driven to evolve the Storage layer towards
>> this separation.
>>
>> I am fine with the VolumeManager determining whether or not a Volume is
>> managed (i.e. not based on the StoragePoolType, but an actual isManaged
>> method), and asking the device driver to allocate resources for the volume
>> if it is managed. Furthermore, the device driver needs to indicate whether
>> or not it supports management operations. Finally, I think we need to
>> provide the ability for an administrator to elect to have something that is
>> manageable be unmanaged (i.e. the driver is capable managing the device,
>> but the administrator has elected to leave it unmanaged).
>>
>> Creation of a structure on the volume should be done in the Xen
>> hypervisor module using methods exposed by the Storage layer to perform
>> low-level operations (e.g. make directories, create a file, etc). This
>> structure is specific to the operation of the Xen hypervisor, as such,
>> should be confined to its implementation. From my perspective, nothing in
>> the Storage layer should be concerned with content. From its perspective,
>> structure and data are opaque. It provides the means to query the data to
>> support the interpretation of the content by higher-level layers (e.g.
>> Hypervisors). To my mind, attach should be a composition of operations
>> from the Storage layer that varies based on the Volume storage protocol
>> (iSCSI, local file system, NFS, RBD, etc).
>>
>> Thanks,
>> -John
>>
>> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> Hi John,
>>
>> Alternatively to the way the attach logic is implemented in my patch, we
>> could do the following:
>>
>> Leave the attach logic in the agent code alone. In VolumeManagerImpl we
>> create an AttachVolumeCommand and send it to the hypervisor. Before this
>> command is sent, we could check to see if we're dealing with Dynamic (or
>> whatever we want to call it) storage and - if so - send a "Create SR"
>> command to the hypervisor. If this returns OK, we would then proceed to the
>> AttachVolumeCommand, as usual.
>>
>> This way the attach logic remains the same and we just add another
>> command to the agent code that is called for this particular type of
>> storage.
>>
>> What do you think?
>>
>>
>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Hey John,
>>>
>>> I created a document for a customer today that outlines how the plug-in
>>> works from a user standpoint. This will probably be of use to you, as well,
>>> as you perform the code review.
>>>
>>> I have shared this document with you (you should have received that
>>> information in a separate e-mail).
>>>
>>> Talk to you later!
>>>
>>>
>>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> Oh, OK, that sounds really good, John.
>>>>
>>>> Thanks and talk to you tomorrow! :)
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jb...@basho.com>wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> I am never at a loss for an opinion. I some thoughts, but want to
>>>>> confirm assumptions and ideas against the solidfire, disk_io_throttle,
>>>>> and
>>>>> object_store branches. I hope to collect them in a coherent form
>>>>> tomorrow (5 June 2013).
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>>
>>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>> > "So, in essence, the SolidFire plugin introduces the notion of a
>>>>> managed
>>>>> > iSCSI device and provisioned IOPS."
>>>>> >
>>>>> > Technically, the SolidFire plug-in just introduces the notion of
>>>>> > provisioned storage IOPS.
>>>>> >
>>>>> > The storage framework that leverages the plug-in was incomplete, so
>>>>> I had
>>>>> > to try to add in the notion of a managed iSCSI device.
>>>>> >
>>>>> > I appreciate all the time you've been spending on this. :) Do you
>>>>> have a
>>>>> > recommendation as to how we should accomplish what you're looking
>>>>> for?
>>>>> >
>>>>> > Thanks!
>>>>> >
>>>>> >
>>>>> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>> >
>>>>> >> Mike,
>>>>> >>
>>>>> >> So, in essence, the SolidFire plugin introduces the notion of a
>>>>> managed
>>>>> >> iSCSI device and provisioned IOPS.
>>>>> >>
>>>>> >> I want to see a separation of the management capabilities (i.e. can
>>>>> a
>>>>> >> device be managed/does an operator want it managed by CloudStack)
>>>>> from the
>>>>> >> storage protocol. Ideally, we should end up with a semantic that
>>>>> will
>>>>> >> allow any type of storage device to be managed. I also want to make
>>>>> >> progress on decoupling the storage types from the hypervisor
>>>>> definitions.
>>>>> >>
>>>>> >> Thanks,
>>>>> >> -John
>>>>> >>
>>>>> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>> >> wrote:
>>>>> >>
>>>>> >>> Hi John,
>>>>> >>>
>>>>> >>> No problem. Answers are below in red.
>>>>> >>>
>>>>> >>> Thanks!
>>>>> >>>
>>>>> >>>
>>>>> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>>> Mike,
>>>>> >>>>
>>>>> >>>> Could you please answer the following questions for me with
>>>>> regards to
>>>>> >> the
>>>>> >>>> operation of the SolidFire plugin:
>>>>> >>>>
>>>>> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>>>> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
>>>>> >>> 1 SAN volume : 1 LUN
>>>>> >>>
>>>>> >>>> What is the cardinality between SAN Volumes and CloudStack
>>>>> Volumes?
>>>>> >>> 1 SAN volume : 1 CloudStack volume
>>>>> >>>
>>>>> >>>> Are the LUN(s) created by the management server or externally by
>>>>> the
>>>>> >>>> operator?
>>>>> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN
>>>>> LUN) is
>>>>> >>> created by the management server (via the plug-in) the first time
>>>>> the
>>>>> >>> CloudStack volume is attached to a hypervisor.
>>>>> >>>
>>>>> >>> If you don't want to use the SolidFire plug-in, but still want to
>>>>> use a
>>>>> >>> SolidFire volume (LUN), you can do this already today (prior to
>>>>> 4.2). The
>>>>> >>> admin manually creates the SAN volume and - in this case -
>>>>> multiple VMs
>>>>> >> and
>>>>> >>> data disks can share this SAN volume. While you can do this today,
>>>>> it is
>>>>> >>> not useful if you want to enforce storage QoS.
>>>>> >>>
>>>>> >>>> Are the SAN volumes by the management server or externally by the
>>>>> >> operator?
>>>>> >>> When the SolidFire plug-in is used, the SAN volumes are completely
>>>>> >> managed
>>>>> >>> by the management server (via the plug-in). There is no admin
>>>>> >> interaction.
>>>>> >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
>>>>> >> volume,
>>>>> >>> which is necessary for any storage vendor that supports true QoS.
>>>>> >>>
>>>>> >>>>
>>>>> >>>> I would like to clarify how these pieces are related and expected
>>>>> to
>>>>> >>>> operate.
>>>>> >>>>
>>>>> >>>> Thanks,
>>>>> >>>> -John
>>>>> >>>>
>>>>> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>>> >> mike.tutkowski@solidfire.com>
>>>>> >>>> wrote:
>>>>> >>>>
>>>>> >>>>> "In particular, how do we ensure that multiple VMs with
>>>>> provisioned
>>>>> >> IOPS
>>>>> >>>>> won't be cut off by the underlying storage."
>>>>> >>>>>
>>>>> >>>>> In the storage QoS world, we need to map a single SAN volume
>>>>> (LUN) to a
>>>>> >>>>> single CloudStack volume. We cannot have multiple CloudStack
>>>>> volumes
>>>>> >>>>> sharing a single SAN volume and still guarantee QoS.
>>>>> >>>>>
>>>>> >>>>> If the user wants to have a single SAN volume house more than one
>>>>> >>>>> CloudStack volume, then can do that today without any of my
>>>>> plug-in
>>>>> >> code.
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>>> >>>> mike.tutkowski@solidfire.com
>>>>> >>>>>> wrote:
>>>>> >>>>>
>>>>> >>>>>> "The administrator will allocate a SAN volume for CloudStack's
>>>>> use
>>>>> >> onto
>>>>> >>>>>> which CloudStack volumes will be created."
>>>>> >>>>>>
>>>>> >>>>>> I think we crossed e-mails. :)
>>>>> >>>>>>
>>>>> >>>>>> Check out my recent e-mail on this.
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>> >>>> wrote:
>>>>> >>>>>>
>>>>> >>>>>>> Mike,
>>>>> >>>>>>>
>>>>> >>>>>>> You are coming to the part which concerns me -- concepts from
>>>>> the
>>>>> >>>>>>> hypervisor are leaking into storage layer.
>>>>> >>>>>>>
>>>>> >>>>>>> Thanks,
>>>>> >>>>>>> -John
>>>>> >>>>>>>
>>>>> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>>> >>>> mike.tutkowski@solidfire.com>
>>>>> >>>>>>> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>>> The weird part is that the iSCSI type is today only used (as
>>>>> far as
>>>>> >> I
>>>>> >>>>>>> know)
>>>>> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>>>>> >>>>>>>>
>>>>> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the
>>>>> vmfs
>>>>> >>>> type.
>>>>> >>>>>>>>
>>>>> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>>>>> >>>>>>> SharedMountPoint
>>>>> >>>>>>>> type.
>>>>> >>>>>>>>
>>>>> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a
>>>>> new type
>>>>> >>>> of
>>>>> >>>>>>>> storage to describe this dynamic ability Edison added into the
>>>>> >> storage
>>>>> >>>>>>>> framework. Maybe it should be more specificy, though:
>>>>> Dynamic_iSCSI
>>>>> >>>>>>> versus,
>>>>> >>>>>>>> say, Dynamic_FC.
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>>>> >>>>>>> mike.tutkowski@solidfire.com
>>>>> >>>>>>>>> wrote:
>>>>> >>>>>>>>
>>>>> >>>>>>>>> "The storage device itself shouldn't know or care that it is
>>>>> being
>>>>> >>>> used
>>>>> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it
>>>>> is
>>>>> >>>>>>> storing."
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in
>>>>> needs to
>>>>> >>>>>>> call
>>>>> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows
>>>>> to
>>>>> >> treat
>>>>> >>>>>>> it as
>>>>> >>>>>>>>> such.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> I'm totally open to removing that constraint and just
>>>>> calling it
>>>>> >>>> iSCSI
>>>>> >>>>>>> or
>>>>> >>>>>>>>> whatever. We would just need a way for the hypervisor attach
>>>>> logic
>>>>> >> to
>>>>> >>>>>>>>> detect this new requirement and perform the necessary
>>>>> activities.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>> >>>>>>> wrote:
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>> Mike,
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> See my responses in-line.
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> Thanks,
>>>>> >>>>>>>>>> -John
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>>>> >>>>>>> mike.tutkowski@solidfire.com>
>>>>> >>>>>>>>>> wrote:
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>>> I'm trying to picture this:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device,
>>>>> an
>>>>> >>>>>>> operator
>>>>> >>>>>>>>>> may
>>>>> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device
>>>>> is
>>>>> >>>> shared
>>>>> >>>>>>> by
>>>>> >>>>>>>>>>> many services, and the operator has chosen to dedicate
>>>>> only a
>>>>> >>>> portion
>>>>> >>>>>>>>>> of it
>>>>> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
>>>>> plug-in
>>>>> >>>>>>> (before
>>>>> >>>>>>>>>> an
>>>>> >>>>>>>>>>> attach request to the hypervisor) would work unless the
>>>>> >> hypervisor
>>>>> >>>>>>>>>> consumes
>>>>> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator
>>>>> >>>> allocates
>>>>> >>>>>>> a
>>>>> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a
>>>>> LUN. They
>>>>> >>>>>>> simply
>>>>> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but
>>>>> not allow
>>>>> >>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be
>>>>> >> passing
>>>>> >>>>>>> in a
>>>>> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor
>>>>> support
>>>>> >>>> (like
>>>>> >>>>>>> an
>>>>> >>>>>>>>>> SR)
>>>>> >>>>>>>>>>> and the logic will fail.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Are you thinking we should maybe have the storage framework
>>>>> >> itself
>>>>> >>>>>>>>>> detect
>>>>> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor
>>>>> side and
>>>>> >>>>>>> have
>>>>> >>>>>>>>>> it
>>>>> >>>>>>>>>>> call into the agent code specifically to create the SR
>>>>> before the
>>>>> >>>>>>> attach
>>>>> >>>>>>>>>>> logic runs in the agent code?
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> I think the hypervisor management plugin should have a rich
>>>>> enough
>>>>> >>>>>>>>>> interface to storage to determine available for volume
>>>>> storage.
>>>>> >> For
>>>>> >>>>>>> Xen,
>>>>> >>>>>>>>>> this interface would allow the interrogation of the device
>>>>> to
>>>>> >>>>>>> determine the
>>>>> >>>>>>>>>> SR is present. The storage device itself shouldn't know
>>>>> or care
>>>>> >>>>>>> that it
>>>>> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>>>>> questions
>>>>> >>>>>>> about it
>>>>> >>>>>>>>>> is storing.
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>>> >>>>>>>>>> mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>> So, the flow is as follows:
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a
>>>>> type of
>>>>> >>>>>>>>>> so-called
>>>>> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary
>>>>> Storage shows
>>>>> >>>> up
>>>>> >>>>>>> in
>>>>> >>>>>>>>>> the
>>>>> >>>>>>>>>>>> applicable zone.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> * The admin creates a Disk Offering that references the
>>>>> storage
>>>>> >>>> tag
>>>>> >>>>>>> of
>>>>> >>>>>>>>>> the
>>>>> >>>>>>>>>>>> newly created Primary Storage.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to
>>>>> a new
>>>>> >>>> row
>>>>> >>>>>>> in
>>>>> >>>>>>>>>> the
>>>>> >>>>>>>>>>>> cloud.volumes table.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
>>>>> (attach
>>>>> >>>> disk).
>>>>> >>>>>>>>>> This
>>>>> >>>>>>>>>>>> leads to the storage framework calling the plug-in to
>>>>> create a
>>>>> >> new
>>>>> >>>>>>>>>> volume
>>>>> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in
>>>>> also
>>>>> >>>> updates
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of
>>>>> the SAN
>>>>> >>>>>>>>>> volume).
>>>>> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack
>>>>> volume is in
>>>>> >>>> the
>>>>> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be
>>>>> in the
>>>>> >>>>>>> 'Ready'
>>>>> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code
>>>>> will not
>>>>> >> be
>>>>> >>>>>>>>>> called
>>>>> >>>>>>>>>>>> again to create this SAN volume.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>>>>> CloudStack
>>>>> >>>>>>> volume
>>>>> >>>>>>>>>> to
>>>>> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>>>>> >>>> structure
>>>>> >>>>>>>>>> (ex.
>>>>> >>>>>>>>>>>> an SR on XenServer).
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume,
>>>>> in these
>>>>> >>>>>>>>>>>>> definitions, refers to the physical allocation on the
>>>>> device,
>>>>> >>>>>>>>>> correct?"
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>>>>> CloudStack, I
>>>>> >>>>>>> try to
>>>>> >>>>>>>>>>>>> use 'CloudStack volume'.
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> Hi John,
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm
>>>>> having a
>>>>> >>>> hard
>>>>> >>>>>>>>>> time
>>>>> >>>>>>>>>>>>>> envisioning it.
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation
>>>>> as he
>>>>> >> was
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>>>> initial person to suggest the approach I took.
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> What do you think?
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> Thanks!
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>>>>> >>>> jburwell@basho.com
>>>>> >>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> Mike,
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
>>>>> >> storage
>>>>> >>>>>>>>>> device
>>>>> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we
>>>>> are
>>>>> >>>>>>>>>> communicating with
>>>>> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
>>>>> volumes
>>>>> >>>>>>> (Dynamic)
>>>>> >>>>>>>>>> on the
>>>>> >>>>>>>>>>>>>>> device while the other requires the volume to be
>>>>> volume to be
>>>>> >>>>>>>>>> managed
>>>>> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we
>>>>> are in
>>>>> >>>>>>> sync on
>>>>> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to
>>>>> the
>>>>> >>>> physical
>>>>> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must
>>>>> be
>>>>> >> able
>>>>> >>>> to
>>>>> >>>>>>>>>>>>>>> communicate with a storage device to move bits from
>>>>> one place
>>>>> >>>> to
>>>>> >>>>>>>>>> another,
>>>>> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage
>>>>> device
>>>>> >> may
>>>>> >>>> be
>>>>> >>>>>>>>>> able to
>>>>> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
>>>>> unmanaged
>>>>> >>>> iSCSI
>>>>> >>>>>>>>>> device
>>>>> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>>>>> >>>> SolidFire
>>>>> >>>>>>>>>> iSCSI
>>>>> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating
>>>>> LUNs and
>>>>> >>>>>>> storing
>>>>> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
>>>>> manage a
>>>>> >>>>>>> device,
>>>>> >>>>>>>>>> an
>>>>> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack
>>>>> (e.g.
>>>>> >>>> the
>>>>> >>>>>>>>>> device is
>>>>> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
>>>>> >>>> dedicate
>>>>> >>>>>>>>>> only a
>>>>> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make
>>>>> sense?
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>>>>> >> appropriate
>>>>> >>>>>>> to
>>>>> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType,
>>>>> add the
>>>>> >>>>>>> notion
>>>>> >>>>>>>>>> of a
>>>>> >>>>>>>>>>>>>>> storage device with an attached driver that indicates
>>>>> whether
>>>>> >>>> or
>>>>> >>>>>>>>>> not is
>>>>> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>>>>> >>>> representing a
>>>>> >>>>>>>>>> physical
>>>>> >>>>>>>>>>>>>>> allocation on a device separate that is associated
>>>>> with a
>>>>> >>>> volume.
>>>>> >>>>>>>>>> With
>>>>> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare
>>>>> which
>>>>> >>>>>>>>>> protocols they
>>>>> >>>>>>>>>>>>>>> support and when they encounter a device managed by
>>>>> >> CloudStack,
>>>>> >>>>>>>>>> utilize the
>>>>> >>>>>>>>>>>>>>> management operations exposed by the driver to automate
>>>>> >>>>>>> allocation.
>>>>> >>>>>>>>>> If
>>>>> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit
>>>>> down and
>>>>> >>>>>>> drill
>>>>> >>>>>>>>>> down to
>>>>> >>>>>>>>>>>>>>> a more detailed design.
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type
>>>>> and
>>>>> >> the
>>>>> >>>>>>>>>> Dynamic
>>>>> >>>>>>>>>>>>>>> type:
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a
>>>>> Primary
>>>>> >>>> Storage
>>>>> >>>>>>>>>> based
>>>>> >>>>>>>>>>>>>>> on
>>>>> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
>>>>> volume must
>>>>> >>>>>>> exist
>>>>> >>>>>>>>>> on
>>>>> >>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future
>>>>> CloudStack
>>>>> >>>> volumes
>>>>> >>>>>>> are
>>>>> >>>>>>>>>>>>>>> created
>>>>> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create
>>>>> Primary
>>>>> >>>> Storage
>>>>> >>>>>>>>>> based
>>>>> >>>>>>>>>>>>>>> on a
>>>>> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
>>>>> storage
>>>>> >>>>>>> system
>>>>> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage
>>>>> framework). When
>>>>> >> a
>>>>> >>>>>>> user
>>>>> >>>>>>>>>>>>>>> wants to
>>>>> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>>>>> framework
>>>>> >>>> tells
>>>>> >>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done,
>>>>> the
>>>>> >> attach
>>>>> >>>>>>>>>> logic
>>>>> >>>>>>>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor
>>>>> data
>>>>> >>>>>>> structure
>>>>> >>>>>>>>>>>>>>> exists
>>>>> >>>>>>>>>>>>>>>> at this point because the volume was just created. The
>>>>> >>>>>>> hypervisor
>>>>> >>>>>>>>>> data
>>>>> >>>>>>>>>>>>>>>> structure must be created.
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have
>>>>> defined them
>>>>> >>>> up
>>>>> >>>>>>>>>> front
>>>>> >>>>>>>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>>> you. :)
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex.
>>>>> an admin
>>>>> >>>>>>>>>> creates a
>>>>> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
>>>>> >> created
>>>>> >>>>>>> to
>>>>> >>>>>>>>>>>>>>> consume
>>>>> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor
>>>>> data
>>>>> >>>>>>>>>> structure
>>>>> >>>>>>>>>>>>>>> is
>>>>> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed
>>>>> on
>>>>> >> this
>>>>> >>>>>>>>>>>>>>> hypervisor
>>>>> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach
>>>>> logic
>>>>> >>>>>>> assumes
>>>>> >>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and
>>>>> simply
>>>>> >>>>>>> attaches
>>>>> >>>>>>>>>>>>>>> the VDI
>>>>> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
>>>>> question.
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
>>>>> Instead
>>>>> >>>> of
>>>>> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
>>>>> volume on
>>>>> >> a
>>>>> >>>>>>> SAN,
>>>>> >>>>>>>>>>>>>>> but the
>>>>> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
>>>>> created
>>>>> >>>>>>> when an
>>>>> >>>>>>>>>>>>>>> attach
>>>>> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes
>>>>> have not
>>>>> >>>>>>> been
>>>>> >>>>>>>>>>>>>>> pre-hooked
>>>>> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin.
>>>>> Once
>>>>> >> the
>>>>> >>>>>>>>>> attach
>>>>> >>>>>>>>>>>>>>> logic
>>>>> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
>>>>> >> attaches
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>>>>> one and
>>>>> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>>>> >>>>>>> jburwell@basho.com>
>>>>> >>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> Mike,
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type
>>>>> attach
>>>>> >>>> behavior
>>>>> >>>>>>>>>>>>>>> works in
>>>>> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>>>>> difference.
>>>>> >>>>>>> Another
>>>>> >>>>>>>>>>>>>>> way to
>>>>> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
>>>>> Dynamic
>>>>> >>>>>>> storage
>>>>> >>>>>>>>>>>>>>> pool type?
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>> >>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used
>>>>> by
>>>>> >>>>>>> XenServer
>>>>> >>>>>>>>>>>>>>> when you
>>>>> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly
>>>>> based on
>>>>> >> an
>>>>> >>>>>>>>>> iSCSI
>>>>> >>>>>>>>>>>>>>>>>> target.
>>>>> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>>>>> >> hypervisor
>>>>> >>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>> creating a
>>>>> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>>>>> >> CloudStack
>>>>> >>>>>>> does
>>>>> >>>>>>>>>>>>>>> that
>>>>> >>>>>>>>>>>>>>>>>> part
>>>>> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
>>>>> XenServer.
>>>>> >> For
>>>>> >>>>>>> all
>>>>> >>>>>>>>>>>>>>> other
>>>>> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor
>>>>> and
>>>>> >>>> perform
>>>>> >>>>>>>>>> this
>>>>> >>>>>>>>>>>>>>> step
>>>>> >>>>>>>>>>>>>>>>>>> manually.
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>>>> >>>>>>>>>> jburwell@basho.com>
>>>>> >>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>> Mike,
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
>>>>> between
>>>>> >>>> the
>>>>> >>>>>>>>>>>>>>> ISCSI and
>>>>> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>> >>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>> >>>>>>>>>>>>>>>>>> class
>>>>> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>>>>> >> overlay
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool
>>>>> is set
>>>>> >>>> up
>>>>> >>>>>>>>>> by
>>>>> >>>>>>>>>>>>>>>>>>>>> customers.
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>>>> >>>>>>>>>> SolidFire)
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> }
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> return shared;
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> }
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> }
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company
>>>>> wants
>>>>> >> to
>>>>> >>>>>>>>>>>>>>> implement a
>>>>> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>>>>> feature. It
>>>>> >>>>>>> would
>>>>> >>>>>>>>>> be
>>>>> >>>>>>>>>>>>>>>>>> dynamic,
>>>>> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>>>>> >>>>>>> implement a
>>>>> >>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to
>>>>> the
>>>>> >>>>>>>>>>>>>>> hypervisor-attach
>>>>> >>>>>>>>>>>>>>>>>>>> logic
>>>>> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
>>>>> SolidFire
>>>>> >>>>>>> specific.
>>>>> >>>>>>>>>>>>>>> It is
>>>>> >>>>>>>>>>>>>>>>>> not.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
>>>>> dynamic,
>>>>> >>>>>>>>>> zone-wide
>>>>> >>>>>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this
>>>>> kind of
>>>>> >>>>>>>>>> storage).
>>>>> >>>>>>>>>>>>>>> This
>>>>> >>>>>>>>>>>>>>>>>>>> kind of
>>>>> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
>>>>> framework
>>>>> >>>>>>>>>> changes.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported
>>>>> the
>>>>> >>>>>>> creation
>>>>> >>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>> deletion
>>>>> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I
>>>>> visited with
>>>>> >>>> him
>>>>> >>>>>>> in
>>>>> >>>>>>>>>>>>>>> Portland
>>>>> >>>>>>>>>>>>>>>>>>>> back
>>>>> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not
>>>>> complete. We
>>>>> >>>>>>> realized
>>>>> >>>>>>>>>>>>>>> there
>>>>> >>>>>>>>>>>>>>>>>> was
>>>>> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes
>>>>> unless
>>>>> >>>> the
>>>>> >>>>>>>>>>>>>>> attach
>>>>> >>>>>>>>>>>>>>>>>> logic
>>>>> >>>>>>>>>>>>>>>>>>>> was
>>>>> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage
>>>>> and
>>>>> >>>> create
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>> appropriate
>>>>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>>> >>>>>>>>>>>>>>> jburwell@basho.com>
>>>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation
>>>>> in the
>>>>> >>>>>>> Storage
>>>>> >>>>>>>>>>>>>>> layer
>>>>> >>>>>>>>>>>>>>>>>>>> would
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect
>>>>> to see
>>>>> >> the
>>>>> >>>>>>>>>> Storage
>>>>> >>>>>>>>>>>>>>>>>>>> services
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>>>>> >> composed/driven
>>>>> >>>>>>> by
>>>>> >>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>>>>> structures
>>>>> >>>> per
>>>>> >>>>>>>>>> their
>>>>> >>>>>>>>>>>>>>>>>> needs.
>>>>> >>>>>>>>>>>>>>>>>>>> If
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> we
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to
>>>>> end
>>>>> >>>> with a
>>>>> >>>>>>>>>>>>>>> massive
>>>>> >>>>>>>>>>>>>>>>>>>> n-to-n
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>>>>> >>>>>>> difficult to
>>>>> >>>>>>>>>>>>>>>>>> maintain
>>>>> >>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
>>>>> specific
>>>>> >>>>>>>>>> SolidFire
>>>>> >>>>>>>>>>>>>>> code
>>>>> >>>>>>>>>>>>>>>>>> is
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike
>>>>> Tutkowski <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method
>>>>> for
>>>>> >>>>>>>>>> XenServer,
>>>>> >>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>> admin
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> had
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
>>>>> system,
>>>>> >> then
>>>>> >>>> go
>>>>> >>>>>>>>>> into
>>>>> >>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
>>>>> >> volume
>>>>> >>>>>>> (ex.
>>>>> >>>>>>>>>> a
>>>>> >>>>>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
>>>>> ESX(i)).
>>>>> >>>> VMs
>>>>> >>>>>>> and
>>>>> >>>>>>>>>>>>>>> data
>>>>> >>>>>>>>>>>>>>>>>> disks
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> then
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage
>>>>> need
>>>>> >> no
>>>>> >>>>>>>>>> longer
>>>>> >>>>>>>>>>>>>>> be so
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> static
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
>>>>> >> between
>>>>> >>>> a
>>>>> >>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>> system's
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
>>>>> >> storage
>>>>> >>>>>>>>>> Quality
>>>>> >>>>>>>>>>>>>>> of
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>>>>> >>>>>>> dynamically
>>>>> >>>>>>>>>>>>>>> create
>>>>> >>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did
>>>>> not
>>>>> >>>> address
>>>>> >>>>>>> is
>>>>> >>>>>>>>>> in
>>>>> >>>>>>>>>>>>>>>>>> creating
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
>>>>> structure
>>>>> >> when
>>>>> >>>>>>>>>>>>>>> performing an
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it
>>>>> to do
>>>>> >> so.
>>>>> >>>>>>> I've
>>>>> >>>>>>>>>>>>>>> got
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in
>>>>> my
>>>>> >>>> sandbox
>>>>> >>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>> can
>>>>> >>>>>>>>>>>>>>>>>>>> submit
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> this
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
>>>>> Tutkowski <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
>>>>> hypervisor
>>>>> >>>>>>> agnostic.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic
>>>>> (in the
>>>>> >>>>>>> agent
>>>>> >>>>>>>>>>>>>>> code).
>>>>> >>>>>>>>>>>>>>>>>> The
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
>>>>> >> create a
>>>>> >>>>>>>>>> volume
>>>>> >>>>>>>>>>>>>>> as
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to
>>>>> a
>>>>> >>>>>>> hypervisor,
>>>>> >>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>> attach
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's
>>>>> being
>>>>> >>>>>>> invoked on
>>>>> >>>>>>>>>>>>>>>>>> zone-wide
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
>>>>> >> create,
>>>>> >>>>>>>>>> say, a
>>>>> >>>>>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore
>>>>> (for
>>>>> >>>>>>> VMware) to
>>>>> >>>>>>>>>>>>>>> make
>>>>> >>>>>>>>>>>>>>>>>> use
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently
>>>>> making
>>>>> >>>> the
>>>>> >>>>>>>>>>>>>>> attach
>>>>> >>>>>>>>>>>>>>>>>> logic
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> work
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
>>>>> Burwell <
>>>>> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver
>>>>> is
>>>>> >>>>>>> hypervisor
>>>>> >>>>>>>>>>>>>>>>>> specific?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski
>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support
>>>>> all
>>>>> >>>>>>> hypervisors
>>>>> >>>>>>>>>>>>>>> that
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time
>>>>> for 4.2
>>>>> >> to
>>>>> >>>>>>> get
>>>>> >>>>>>>>>> KVM
>>>>> >>>>>>>>>>>>>>> in.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>>>>> >>>>>>> Depending on
>>>>> >>>>>>>>>>>>>>> what
>>>>> >>>>>>>>>>>>>>>>>> we do
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
>>>>> working
>>>>> >>>> for
>>>>> >>>>>>>>>>>>>>> VMware in
>>>>> >>>>>>>>>>>>>>>>>> my
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
>>>>> regards
>>>>> >> to
>>>>> >>>>>>> Disk
>>>>> >>>>>>>>>>>>>>>>>> Offerings.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
>>>>> Jamison
>>>>> >>>>>>> Damage
>>>>> >>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in
>>>>> the
>>>>> >> patch
>>>>> >>>>>>> cycle
>>>>> >>>>>>>>>>>>>>> post
>>>>> >>>>>>>>>>>>>>>>>> 4.2?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>>>> >>>>>>> mike.tutkowski@solidfire.com>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling
>>>>> to
>>>>> >> MASTER
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first,
>>>>> then
>>>>> >>>> mine.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it
>>>>> is a
>>>>> >> non
>>>>> >>>>>>> issue
>>>>> >>>>>>>>>>>>>>> as I
>>>>> >>>>>>>>>>>>>>>>>> don't
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU
>>>>> <
>>>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
>>>>> billing
>>>>> >> now,
>>>>> >>>>>>>>>> users
>>>>> >>>>>>>>>>>>>>> need
>>>>> >>>>>>>>>>>>>>>>>> to
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get
>>>>> the
>>>>> >>>>>>>>>>>>>>> service_offering
>>>>> >>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>>>>> >>>>>>> calculation.
>>>>> >>>>>>>>>> Of
>>>>> >>>>>>>>>>>>>>> course
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value
>>>>> of
>>>>> >>>>>>> individual
>>>>> >>>>>>>>>>>>>>> volume,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can
>>>>> work
>>>>> >> on
>>>>> >>>> it
>>>>> >>>>>>>>>>>>>>> later. Do
>>>>> >>>>>>>>>>>>>>>>>> you
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>>>>> question.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>>>>> >> features.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>> jburwell@basho.com>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>>> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
>>>>> trying
>>>>> >> to
>>>>> >>>>>>>>>> adding
>>>>> >>>>>>>>>>>>>>> more.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on
>>>>> IO
>>>>> >>>>>>> statistics
>>>>> >>>>>>>>>>>>>>> rather
>>>>> >>>>>>>>>>>>>>>>>> than
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review
>>>>> disk_io_stat if
>>>>> >> you
>>>>> >>>>>>> have
>>>>> >>>>>>>>>>>>>>> time.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>>>>> bytes/iops
>>>>> >>>>>>>>>> read/write
>>>>> >>>>>>>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>>>> an
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are
>>>>> billed
>>>>> >>>> more
>>>>> >>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>>>> volumes
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> with
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>>>>> >>>>>>> operations (
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
>>>>> >> imagine
>>>>> >>>>>>> our
>>>>> >>>>>>>>>>>>>>> users
>>>>> >>>>>>>>>>>>>>>>>> would
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> like
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
>>>>> Could
>>>>> >> an
>>>>> >>>>>>>>>>>>>>> operator
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change?
>>>>> KVM
>>>>> >>>>>>> supports
>>>>> >>>>>>>>>>>>>>> setting
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
>>>>> machine
>>>>> >>>> through
>>>>> >>>>>>>>>>>>>>> command
>>>>> >>>>>>>>>>>>>>>>>> line.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing
>>>>> the
>>>>> >>>>>>> parameters
>>>>> >>>>>>>>>>>>>>> of a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> created
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>>>>> >>>> apologize
>>>>> >>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>> being
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation
>>>>> algorithms
>>>>> >> with a
>>>>> >>>>>>> set
>>>>> >>>>>>>>>> of
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
>>>>> provided by a
>>>>> >>>>>>>>>> resource?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
>>>>> >> better
>>>>> >>>> to
>>>>> >>>>>>>>>>>>>>> commit
>>>>> >>>>>>>>>>>>>>>>>> Mike's
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
>>>>> consider
>>>>> >>>> the
>>>>> >>>>>>>>>>>>>>>>>> limitation in
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>>>>> >>>> response
>>>>> >>>>>>> to
>>>>> >>>>>>>>>>>>>>> Mike
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
>>>>> features. I
>>>>> >>>> think
>>>>> >>>>>>>>>> that
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
>>>>> SolidFire, but
>>>>> >> I
>>>>> >>>>>>> think
>>>>> >>>>>>>>>>>>>>> we need
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>>>>> (possibly
>>>>> >>>> have
>>>>> >>>>>>>>>> have
>>>>> >>>>>>>>>>>>>>>>>> solidfire
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on
>>>>> this
>>>>> >>>> issue.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>>> jburwell@basho.com>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are
>>>>> the
>>>>> >>>>>>> following:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>>>>> >> policies
>>>>> >>>> be
>>>>> >>>>>>>>>>>>>>> captured
>>>>> >>>>>>>>>>>>>>>>>> in
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>>>>> >> purposes,
>>>>> >>>>>>>>>>>>>>> expressing
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common
>>>>> interface model
>>>>> >>>> for
>>>>> >>>>>>>>>>>>>>> reasoning
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> about
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned
>>>>> IOPS
>>>>> >>>>>>>>>>>>>>> configurations
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> between
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>>>>> reconciled?
>>>>> >> In
>>>>> >>>>>>>>>>>>>>> particular,
>>>>> >>>>>>>>>>>>>>>>>> a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for
>>>>> more
>>>>> >> IOPS
>>>>> >>>>>>>>>>>>>>> configured
>>>>> >>>>>>>>>>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>>>>> >>>> deliver.
>>>>> >>>>>>>>>>>>>>> Another
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a
>>>>> VM and a
>>>>> >>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>> device at
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
>>>>> storage
>>>>> >>>>>>> device
>>>>> >>>>>>>>>>>>>>>>>> introduces
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
>>>>> Tutkowski
>>>>> >> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>>>>> >>>> controlling
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>>>>> max
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling
>>>>> IOPS
>>>>> >>>> from
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>>>>> storage
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> system
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
>>>>> >> Burwell
>>>>> >>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
>>>>> should be
>>>>> >>>>>>> merged
>>>>> >>>>>>>>>>>>>>> until all
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is
>>>>> ready
>>>>> >> to
>>>>> >>>>>>> turn
>>>>> >>>>>>>>>>>>>>> over to
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write
>>>>> ops issue
>>>>> >>>> is
>>>>> >>>>>>>>>>>>>>> addressed
>>>>> >>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria
>>>>> has been
>>>>> >>>>>>>>>>>>>>> satisfied.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>>>>> >>>> intersect/compliment
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>> SolidFire
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>>> )?
>>>>> >> As I
>>>>> >>>>>>>>>>>>>>> understand
>>>>> >>>>>>>>>>>>>>>>>> it
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> that
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I
>>>>> would
>>>>> >>>> like
>>>>> >>>>>>> to
>>>>> >>>>>>>>>>>>>>> ensure
>>>>> >>>>>>>>>>>>>>>>>> we
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in
>>>>> KVM and
>>>>> >>>>>>>>>>>>>>> SolidFire are
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei
>>>>> ZHOU <
>>>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>>>>> >> wido@widodh.nl
>>>>> >>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU
>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features
>>>>> into
>>>>> >>>>>>> master.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
>>>>> >> reason
>>>>> >>>>>>> till
>>>>> >>>>>>>>>>>>>>> Monday,
>>>>> >>>>>>>>>>>>>>>>>> I
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> will
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want
>>>>> to
>>>>> >>>> mention
>>>>> >>>>>>>>>> again
>>>>> >>>>>>>>>>>>>>>>>> that we
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so
>>>>> that we
>>>>> >>>>>>> never
>>>>> >>>>>>>>>>>>>>> release
>>>>> >>>>>>>>>>>>>>>>>> a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>>>>> >>>> throttling
>>>>> >>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>> that's
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
>>>>> legacy from
>>>>> >>>> day
>>>>> >>>>>>>>>> #1.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>> >>>> wido@widodh.nl>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
>>>>> >> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>>>>> discretely
>>>>> >>>>>>> support
>>>>> >>>>>>>>>>>>>>> read
>>>>> >>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> write
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed
>>>>> later
>>>>> >>>> because
>>>>> >>>>>>> I
>>>>> >>>>>>>>>>>>>>> don't see
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and
>>>>> write.
>>>>> >>>>>>>>>> Therefore, we
>>>>> >>>>>>>>>>>>>>>>>> would
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> be
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to
>>>>> break
>>>>> >>>>>>> backwards
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into
>>>>> master
>>>>> >>>> now
>>>>> >>>>>>> so
>>>>> >>>>>>>>>>>>>>> that it
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
>>>>> >> before
>>>>> >>>>>>> the
>>>>> >>>>>>>>>> 4.2
>>>>> >>>>>>>>>>>>>>>>>> release
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> so
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
>>>>> >> instead
>>>>> >>>>>>> of
>>>>> >>>>>>>>>>>>>>> Total
>>>>> >>>>>>>>>>>>>>>>>> IOps.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
>>>>> >> freeze
>>>>> >>>>>>> date.
>>>>> >>>>>>>>>>>>>>> But if
>>>>> >>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>>>>> >>>> necessary
>>>>> >>>>>>>>>> to do
>>>>> >>>>>>>>>>>>>>>>>> that.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there
>>>>> is no
>>>>> >>>>>>>>>> association
>>>>> >>>>>>>>>>>>>>>>>> between
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido
>>>>> den
>>>>> >>>>>>> Hollander <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU
>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>>>>> >> Finally I
>>>>> >>>>>>>>>>>>>>> decided to
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
>>>>> >> because
>>>>> >>>>>>> the
>>>>> >>>>>>>>>>>>>>> network
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
>>>>> received
>>>>> >>>>>>> bytes as
>>>>> >>>>>>>>>>>>>>> well.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd.
>>>>> Networking and
>>>>> >>>> disk
>>>>> >>>>>>> I/O
>>>>> >>>>>>>>>>>>>>>>>> completely
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in
>>>>> most
>>>>> >>>>>>>>>> situations
>>>>> >>>>>>>>>>>>>>> then
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> network
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
>>>>> copy-paste
>>>>> >>>> work.
>>>>> >>>>>>> It
>>>>> >>>>>>>>>>>>>>> could be
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of
>>>>> feature
>>>>> >>>> freeze,
>>>>> >>>>>>> I
>>>>> >>>>>>>>>> will
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't
>>>>> miss. But
>>>>> >>>> if
>>>>> >>>>>>> it
>>>>> >>>>>>>>>>>>>>> goes
>>>>> >>>>>>>>>>>>>>>>>> into
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
>>>>> >> release
>>>>> >>>>>>> with
>>>>> >>>>>>>>>>>>>>> only
>>>>> >>>>>>>>>>>>>>>>>> total
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>>> >>>> wido@widodh.nl>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU
>>>>> wrote:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>>>>> disk_io_throttling
>>>>> >>>>>>> branch
>>>>> >>>>>>>>>> into
>>>>> >>>>>>>>>>>>>>>>>> master.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into
>>>>> master
>>>>> >>>> in
>>>>> >>>>>>> 48
>>>>> >>>>>>>>>>>>>>> hours.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the
>>>>> same
>>>>> >>>>>>> storage
>>>>> >>>>>>>>>>>>>>> device
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>>>>> >>>> limitation
>>>>> >>>>>>> of
>>>>> >>>>>>>>>>>>>>> device
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it
>>>>> may
>>>>> >>>> affect
>>>>> >>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>> disk
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same
>>>>> storage
>>>>> >>>> device.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum
>>>>> rate
>>>>> >> and
>>>>> >>>>>>> limit
>>>>> >>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>> disk
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make
>>>>> no
>>>>> >>>>>>> difference
>>>>> >>>>>>>>>>>>>>> between
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting
>>>>> both a
>>>>> >>>>>>> different
>>>>> >>>>>>>>>>>>>>> rate
>>>>> >>>>>>>>>>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
>>>>> users.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling
>>>>> side you
>>>>> >>>>>>>>>> collect
>>>>> >>>>>>>>>>>>>>> both
>>>>> >>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side
>>>>> you only
>>>>> >> go
>>>>> >>>>>>> for
>>>>> >>>>>>>>>> a
>>>>> >>>>>>>>>>>>>>> global
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
>>>>> expensive
>>>>> >>>>>>> then
>>>>> >>>>>>>>>> Read
>>>>> >>>>>>>>>>>>>>>>>> IOps,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an
>>>>> admin
>>>>> >>>>>>> would
>>>>> >>>>>>>>>> set
>>>>> >>>>>>>>>>>>>>> a
>>>>> >>>>>>>>>>>>>>>>>> lower
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
>>>>> >> point I
>>>>> >>>>>>>>>> think
>>>>> >>>>>>>>>>>>>>> it
>>>>> >>>>>>>>>>>>>>>>>> would
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> be
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the
>>>>> mechanism in
>>>>> >>>> place
>>>>> >>>>>>> to
>>>>> >>>>>>>>>>>>>>> support
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
>>>>> setting
>>>>> >>>>>>> different
>>>>> >>>>>>>>>>>>>>> values
>>>>> >>>>>>>>>>>>>>>>>> for
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up
>>>>> and set
>>>>> >>>> that
>>>>> >>>>>>> as
>>>>> >>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>> total
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented
>>>>> it
>>>>> >> this
>>>>> >>>>>>> way?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>>>> >>>>>>>>>> disk_offering,
>>>>> >>>>>>>>>>>>>>> and
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> global
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps
>>>>> and
>>>>> >> iops)
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>> >> https://issues.apache.org/****
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
>>>>> >
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>> >>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>> >>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://cwiki.apache.org/confluence/**>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>> >>>>>>>>>>>>>>>>>> https://cwiki.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> **
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>
>>>>> >>>>
>>>>> >>
>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
>>>>> execution
>>>>> >>>>>>>>>> success?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>>>>> introduced?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>>>>> >>>>>>> integration)
>>>>> >>>>>>>>>> is
>>>>> >>>>>>>>>>>>>>>>>> included
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> in
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to
>>>>> check for
>>>>> >>>>>>>>>> potential
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS
>>>>> rate on
>>>>> >>>>>>>>>> CloudStack
>>>>> >>>>>>>>>>>>>>> UI.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot,
>>>>> destroy,
>>>>> >>>> expunge.
>>>>> >>>>>>>>>>>>>>> migrate,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>> >>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>> >>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://cwiki.apache.org/confluence/**>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>> >>>>>>>>>>>>>>>>>> https://cwiki.
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> **
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>
>>>>> >>>>
>>>>> >>
>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>> >>>>>>>>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
>>>>> >
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>>>>> >> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>> >>>>>>>>>>>>>>>>>> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>>>>> >
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>>>>> >> <
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>>> SolidFire
>>>>> >> Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>> Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>> Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>>> Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play
>>>>> >>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> >> http://solidfire.com/solution/overview/?video=play
>>>>> >>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play
>>>>> >>>
>>>>> >>>>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> --
>>>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> --
>>>>> >>>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> --
>>>>> >>>>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>>>> *â„¢*
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> --
>>>>> >>>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>>> o: 303.746.7302
>>>>> >>>>>>>>> Advancing the way the world uses the cloud<
>>>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>>> *â„¢*
>>>>> >>>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>> --
>>>>> >>>>>>>> *Mike Tutkowski*
>>>>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>>>> o: 303.746.7302
>>>>> >>>>>>>> Advancing the way the world uses the
>>>>> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>>>> *â„¢*
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> --
>>>>> >>>>>> *Mike Tutkowski*
>>>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>>> o: 303.746.7302
>>>>> >>>>>> Advancing the way the world uses the cloud<
>>>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>>> *â„¢*
>>>>> >>>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> --
>>>>> >>>>> *Mike Tutkowski*
>>>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>>> e: mike.tutkowski@solidfire.com
>>>>> >>>>> o: 303.746.7302
>>>>> >>>>> Advancing the way the world uses the
>>>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> >>>>> *â„¢*
>>>>> >>>>
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>> --
>>>>> >>> *Mike Tutkowski*
>>>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>> e: mike.tutkowski@solidfire.com
>>>>> >>> o: 303.746.7302
>>>>> >>> Advancing the way the world uses the
>>>>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> >>> *â„¢*
>>>>> >>
>>>>> >>
>>>>> >
>>>>> >
>>>>> > --
>>>>> > *Mike Tutkowski*
>>>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>>>> > e: mike.tutkowski@solidfire.com
>>>>> > o: 303.746.7302
>>>>> > Advancing the way the world uses the
>>>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> > *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
See my responses in-line below.
Thanks,
-John
On Jun 6, 2013, at 11:09 AM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi John,
>
> Thanks for the response.
>
> "I am fine with the VolumeManager determining whether or not a Volume is managed (i.e. not based on the StoragePoolType, but an actual isManaged method), and asking the device driver to allocate resources for the volume if it is managed."
>
> Are you thinking you'd like to see an isManaged() method added to the PrimaryDataStoreDriver interface? If it returns true, then the storage framework could call the manage() (or whatever name) method (which would be new to the PrimaryDataStoreDriver interface, as well) and this would call into a new method in the hypervisor code to create, say on XenServer, an SR?
I would like to see canBeManaged() : boolean on DataStoreDriver. Since the notion of Volumes only pertains to primary storage, I would add a allocateStorage and deallocateStorage (Storage is a straw man term -- something other than volume) methods to allocate/create/deallocate/delete underlying storage. To my mind, managed is a mutable property of DataStore which can be enabled if/when the underlying DataStoreDriver can be managed. This approach allows operators to override manageability of devices.
In terms of orchestration/process flow for SR, the Xen plugin would be responsible for composing DataStore/Volume methods to create any directories or files necessary for the SR. There should be no dependencies from the Storage to the Hypervisor layer. As I said earlier, such circular dependencies will lead to a tangled, unmaintainable mess.
>
> Just want to make sure I'm on the same page with you.
>
> Thanks again, John
>
>
> On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Fundamentally, we can't end up with a Storage layer that supports n devices types with each specific behaviors of m hypervisors. Such a scenario will create an unmaintainable and untestable beast. Therefore, my thoughts and recommendations are driven to evolve the Storage layer towards this separation.
>
> I am fine with the VolumeManager determining whether or not a Volume is managed (i.e. not based on the StoragePoolType, but an actual isManaged method), and asking the device driver to allocate resources for the volume if it is managed. Furthermore, the device driver needs to indicate whether or not it supports management operations. Finally, I think we need to provide the ability for an administrator to elect to have something that is manageable be unmanaged (i.e. the driver is capable managing the device, but the administrator has elected to leave it unmanaged).
>
> Creation of a structure on the volume should be done in the Xen hypervisor module using methods exposed by the Storage layer to perform low-level operations (e.g. make directories, create a file, etc). This structure is specific to the operation of the Xen hypervisor, as such, should be confined to its implementation. From my perspective, nothing in the Storage layer should be concerned with content. From its perspective, structure and data are opaque. It provides the means to query the data to support the interpretation of the content by higher-level layers (e.g. Hypervisors). To my mind, attach should be a composition of operations from the Storage layer that varies based on the Volume storage protocol (iSCSI, local file system, NFS, RBD, etc).
>
> Thanks,
> -John
>
> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>
>> Hi John,
>>
>> Alternatively to the way the attach logic is implemented in my patch, we could do the following:
>>
>> Leave the attach logic in the agent code alone. In VolumeManagerImpl we create an AttachVolumeCommand and send it to the hypervisor. Before this command is sent, we could check to see if we're dealing with Dynamic (or whatever we want to call it) storage and - if so - send a "Create SR" command to the hypervisor. If this returns OK, we would then proceed to the AttachVolumeCommand, as usual.
>>
>> This way the attach logic remains the same and we just add another command to the agent code that is called for this particular type of storage.
>>
>> What do you think?
>>
>>
>> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>> Hey John,
>>
>> I created a document for a customer today that outlines how the plug-in works from a user standpoint. This will probably be of use to you, as well, as you perform the code review.
>>
>> I have shared this document with you (you should have received that information in a separate e-mail).
>>
>> Talk to you later!
>>
>>
>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>> Oh, OK, that sounds really good, John.
>>
>> Thanks and talk to you tomorrow! :)
>>
>>
>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
>> Mike,
>>
>> I am never at a loss for an opinion. I some thoughts, but want to
>> confirm assumptions and ideas against the solidfire, disk_io_throttle,
>> and
>> object_store branches. I hope to collect them in a coherent form
>> tomorrow (5 June 2013).
>>
>> Thanks,
>> -John
>>
>>
>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>>
>> > "So, in essence, the SolidFire plugin introduces the notion of a managed
>> > iSCSI device and provisioned IOPS."
>> >
>> > Technically, the SolidFire plug-in just introduces the notion of
>> > provisioned storage IOPS.
>> >
>> > The storage framework that leverages the plug-in was incomplete, so I had
>> > to try to add in the notion of a managed iSCSI device.
>> >
>> > I appreciate all the time you've been spending on this. :) Do you have a
>> > recommendation as to how we should accomplish what you're looking for?
>> >
>> > Thanks!
>> >
>> >
>> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com> wrote:
>> >
>> >> Mike,
>> >>
>> >> So, in essence, the SolidFire plugin introduces the notion of a managed
>> >> iSCSI device and provisioned IOPS.
>> >>
>> >> I want to see a separation of the management capabilities (i.e. can a
>> >> device be managed/does an operator want it managed by CloudStack) from the
>> >> storage protocol. Ideally, we should end up with a semantic that will
>> >> allow any type of storage device to be managed. I also want to make
>> >> progress on decoupling the storage types from the hypervisor definitions.
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
>> >> wrote:
>> >>
>> >>> Hi John,
>> >>>
>> >>> No problem. Answers are below in red.
>> >>>
>> >>> Thanks!
>> >>>
>> >>>
>> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com> wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> Could you please answer the following questions for me with regards to
>> >> the
>> >>>> operation of the SolidFire plugin:
>> >>>>
>> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
>> >>> 1 SAN volume : 1 LUN
>> >>>
>> >>>> What is the cardinality between SAN Volumes and CloudStack Volumes?
>> >>> 1 SAN volume : 1 CloudStack volume
>> >>>
>> >>>> Are the LUN(s) created by the management server or externally by the
>> >>>> operator?
>> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) is
>> >>> created by the management server (via the plug-in) the first time the
>> >>> CloudStack volume is attached to a hypervisor.
>> >>>
>> >>> If you don't want to use the SolidFire plug-in, but still want to use a
>> >>> SolidFire volume (LUN), you can do this already today (prior to 4.2). The
>> >>> admin manually creates the SAN volume and - in this case - multiple VMs
>> >> and
>> >>> data disks can share this SAN volume. While you can do this today, it is
>> >>> not useful if you want to enforce storage QoS.
>> >>>
>> >>>> Are the SAN volumes by the management server or externally by the
>> >> operator?
>> >>> When the SolidFire plug-in is used, the SAN volumes are completely
>> >> managed
>> >>> by the management server (via the plug-in). There is no admin
>> >> interaction.
>> >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
>> >> volume,
>> >>> which is necessary for any storage vendor that supports true QoS.
>> >>>
>> >>>>
>> >>>> I would like to clarify how these pieces are related and expected to
>> >>>> operate.
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com>
>> >>>> wrote:
>> >>>>
>> >>>>> "In particular, how do we ensure that multiple VMs with provisioned
>> >> IOPS
>> >>>>> won't be cut off by the underlying storage."
>> >>>>>
>> >>>>> In the storage QoS world, we need to map a single SAN volume (LUN) to a
>> >>>>> single CloudStack volume. We cannot have multiple CloudStack volumes
>> >>>>> sharing a single SAN volume and still guarantee QoS.
>> >>>>>
>> >>>>> If the user wants to have a single SAN volume house more than one
>> >>>>> CloudStack volume, then can do that today without any of my plug-in
>> >> code.
>> >>>>>
>> >>>>>
>> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com
>> >>>>>> wrote:
>> >>>>>
>> >>>>>> "The administrator will allocate a SAN volume for CloudStack's use
>> >> onto
>> >>>>>> which CloudStack volumes will be created."
>> >>>>>>
>> >>>>>> I think we crossed e-mails. :)
>> >>>>>>
>> >>>>>> Check out my recent e-mail on this.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> Mike,
>> >>>>>>>
>> >>>>>>> You are coming to the part which concerns me -- concepts from the
>> >>>>>>> hypervisor are leaking into storage layer.
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> -John
>> >>>>>>>
>> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> The weird part is that the iSCSI type is today only used (as far as
>> >> I
>> >>>>>>> know)
>> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>> >>>>>>>>
>> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
>> >>>> type.
>> >>>>>>>>
>> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>> >>>>>>> SharedMountPoint
>> >>>>>>>> type.
>> >>>>>>>>
>> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a new type
>> >>>> of
>> >>>>>>>> storage to describe this dynamic ability Edison added into the
>> >> storage
>> >>>>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
>> >>>>>>> versus,
>> >>>>>>>> say, Dynamic_FC.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> "The storage device itself shouldn't know or care that it is being
>> >>>> used
>> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it is
>> >>>>>>> storing."
>> >>>>>>>>>
>> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in needs to
>> >>>>>>> call
>> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
>> >> treat
>> >>>>>>> it as
>> >>>>>>>>> such.
>> >>>>>>>>>
>> >>>>>>>>> I'm totally open to removing that constraint and just calling it
>> >>>> iSCSI
>> >>>>>>> or
>> >>>>>>>>> whatever. We would just need a way for the hypervisor attach logic
>> >> to
>> >>>>>>>>> detect this new requirement and perform the necessary activities.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
>> >>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Mike,
>> >>>>>>>>>>
>> >>>>>>>>>> See my responses in-line.
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> -John
>> >>>>>>>>>>
>> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> I'm trying to picture this:
>> >>>>>>>>>>>
>> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
>> >>>>>>> operator
>> >>>>>>>>>> may
>> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
>> >>>> shared
>> >>>>>>> by
>> >>>>>>>>>>> many services, and the operator has chosen to dedicate only a
>> >>>> portion
>> >>>>>>>>>> of it
>> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>> >>>>>>>>>>>
>> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
>> >>>>>>> (before
>> >>>>>>>>>> an
>> >>>>>>>>>>> attach request to the hypervisor) would work unless the
>> >> hypervisor
>> >>>>>>>>>> consumes
>> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
>> >>>>>>>>>>
>> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator
>> >>>> allocates
>> >>>>>>> a
>> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
>> >>>>>>> simply
>> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but not allow
>> >>>>>>>>>> CloudStack to allocate anymore LUNs.
>> >>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be
>> >> passing
>> >>>>>>> in a
>> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor support
>> >>>> (like
>> >>>>>>> an
>> >>>>>>>>>> SR)
>> >>>>>>>>>>> and the logic will fail.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Are you thinking we should maybe have the storage framework
>> >> itself
>> >>>>>>>>>> detect
>> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor side and
>> >>>>>>> have
>> >>>>>>>>>> it
>> >>>>>>>>>>> call into the agent code specifically to create the SR before the
>> >>>>>>> attach
>> >>>>>>>>>>> logic runs in the agent code?
>> >>>>>>>>>>
>> >>>>>>>>>> I think the hypervisor management plugin should have a rich enough
>> >>>>>>>>>> interface to storage to determine available for volume storage.
>> >> For
>> >>>>>>> Xen,
>> >>>>>>>>>> this interface would allow the interrogation of the device to
>> >>>>>>> determine the
>> >>>>>>>>>> SR is present. The storage device itself shouldn't know or care
>> >>>>>>> that it
>> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer questions
>> >>>>>>> about it
>> >>>>>>>>>> is storing.
>> >>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> So, the flow is as follows:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
>> >>>>>>>>>> so-called
>> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows
>> >>>> up
>> >>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>> applicable zone.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The admin creates a Disk Offering that references the storage
>> >>>> tag
>> >>>>>>> of
>> >>>>>>>>>> the
>> >>>>>>>>>>>> newly created Primary Storage.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a new
>> >>>> row
>> >>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>> cloud.volumes table.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
>> >>>> disk).
>> >>>>>>>>>> This
>> >>>>>>>>>>>> leads to the storage framework calling the plug-in to create a
>> >> new
>> >>>>>>>>>> volume
>> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
>> >>>> updates
>> >>>>>>> the
>> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
>> >>>>>>>>>> volume).
>> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is in
>> >>>> the
>> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
>> >>>>>>> 'Ready'
>> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code will not
>> >> be
>> >>>>>>>>>> called
>> >>>>>>>>>>>> again to create this SAN volume.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
>> >>>>>>> volume
>> >>>>>>>>>> to
>> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>> >>>> structure
>> >>>>>>>>>> (ex.
>> >>>>>>>>>>>> an SR on XenServer).
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
>> >>>>>>>>>>>>> definitions, refers to the physical allocation on the device,
>> >>>>>>>>>> correct?"
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
>> >>>>>>> try to
>> >>>>>>>>>>>>> use 'CloudStack volume'.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm having a
>> >>>> hard
>> >>>>>>>>>> time
>> >>>>>>>>>>>>>> envisioning it.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
>> >> was
>> >>>>>>> the
>> >>>>>>>>>>>>>> initial person to suggest the approach I took.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> What do you think?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>> >>>> jburwell@basho.com
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
>> >> storage
>> >>>>>>>>>> device
>> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we are
>> >>>>>>>>>> communicating with
>> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
>> >>>>>>> (Dynamic)
>> >>>>>>>>>> on the
>> >>>>>>>>>>>>>>> device while the other requires the volume to be volume to be
>> >>>>>>>>>> managed
>> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
>> >>>>>>> sync on
>> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
>> >>>> physical
>> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
>> >> able
>> >>>> to
>> >>>>>>>>>>>>>>> communicate with a storage device to move bits from one place
>> >>>> to
>> >>>>>>>>>> another,
>> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device
>> >> may
>> >>>> be
>> >>>>>>>>>> able to
>> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
>> >>>> iSCSI
>> >>>>>>>>>> device
>> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>> >>>> SolidFire
>> >>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
>> >>>>>>> storing
>> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
>> >>>>>>> device,
>> >>>>>>>>>> an
>> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
>> >>>> the
>> >>>>>>>>>> device is
>> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
>> >>>> dedicate
>> >>>>>>>>>> only a
>> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>> >> appropriate
>> >>>>>>> to
>> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
>> >>>>>>> notion
>> >>>>>>>>>> of a
>> >>>>>>>>>>>>>>> storage device with an attached driver that indicates whether
>> >>>> or
>> >>>>>>>>>> not is
>> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>> >>>> representing a
>> >>>>>>>>>> physical
>> >>>>>>>>>>>>>>> allocation on a device separate that is associated with a
>> >>>> volume.
>> >>>>>>>>>> With
>> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
>> >>>>>>>>>> protocols they
>> >>>>>>>>>>>>>>> support and when they encounter a device managed by
>> >> CloudStack,
>> >>>>>>>>>> utilize the
>> >>>>>>>>>>>>>>> management operations exposed by the driver to automate
>> >>>>>>> allocation.
>> >>>>>>>>>> If
>> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
>> >>>>>>> drill
>> >>>>>>>>>> down to
>> >>>>>>>>>>>>>>> a more detailed design.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
>> >> the
>> >>>>>>>>>> Dynamic
>> >>>>>>>>>>>>>>> type:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
>> >>>> Storage
>> >>>>>>>>>> based
>> >>>>>>>>>>>>>>> on
>> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
>> >>>>>>> exist
>> >>>>>>>>>> on
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
>> >>>> volumes
>> >>>>>>> are
>> >>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
>> >>>> Storage
>> >>>>>>>>>> based
>> >>>>>>>>>>>>>>> on a
>> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
>> >>>>>>> system
>> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework). When
>> >> a
>> >>>>>>> user
>> >>>>>>>>>>>>>>> wants to
>> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
>> >>>> tells
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
>> >> attach
>> >>>>>>>>>> logic
>> >>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
>> >>>>>>> structure
>> >>>>>>>>>>>>>>> exists
>> >>>>>>>>>>>>>>>> at this point because the volume was just created. The
>> >>>>>>> hypervisor
>> >>>>>>>>>> data
>> >>>>>>>>>>>>>>>> structure must be created.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have defined them
>> >>>> up
>> >>>>>>>>>> front
>> >>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>> you. :)
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>> >>>>>>>>>> creates a
>> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
>> >> created
>> >>>>>>> to
>> >>>>>>>>>>>>>>> consume
>> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>> >>>>>>>>>> structure
>> >>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
>> >> this
>> >>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
>> >>>>>>> assumes
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
>> >>>>>>> attaches
>> >>>>>>>>>>>>>>> the VDI
>> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead
>> >>>> of
>> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on
>> >> a
>> >>>>>>> SAN,
>> >>>>>>>>>>>>>>> but the
>> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
>> >>>>>>> when an
>> >>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes have not
>> >>>>>>> been
>> >>>>>>>>>>>>>>> pre-hooked
>> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
>> >> the
>> >>>>>>>>>> attach
>> >>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
>> >> attaches
>> >>>>>>> the
>> >>>>>>>>>>>>>>> one and
>> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>> >>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
>> >>>> behavior
>> >>>>>>>>>>>>>>> works in
>> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
>> >>>>>>> Another
>> >>>>>>>>>>>>>>> way to
>> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
>> >>>>>>> storage
>> >>>>>>>>>>>>>>> pool type?
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>> >>>>>>> XenServer
>> >>>>>>>>>>>>>>> when you
>> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on
>> >> an
>> >>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>>>>> target.
>> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>> >> hypervisor
>> >>>>>>> and
>> >>>>>>>>>>>>>>>>>> creating a
>> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>> >> CloudStack
>> >>>>>>> does
>> >>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>> part
>> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
>> >> For
>> >>>>>>> all
>> >>>>>>>>>>>>>>> other
>> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
>> >>>> perform
>> >>>>>>>>>> this
>> >>>>>>>>>>>>>>> step
>> >>>>>>>>>>>>>>>>>>> manually.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>> >>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference between
>> >>>> the
>> >>>>>>>>>>>>>>> ISCSI and
>> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>> >>>>>>>>>>>>>>> Storage.StoragePoolType
>> >>>>>>>>>>>>>>>>>> class
>> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>> >> overlay
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> RBD(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
>> >>>> up
>> >>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>> customers.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>> >>>>>>>>>> SolidFire)
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> boolean shared;
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> return shared;
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants
>> >> to
>> >>>>>>>>>>>>>>> implement a
>> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
>> >>>>>>> would
>> >>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>> dynamic,
>> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>> >>>>>>> implement a
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>> >>>>>>>>>>>>>>> hypervisor-attach
>> >>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>> >>>>>>> specific.
>> >>>>>>>>>>>>>>> It is
>> >>>>>>>>>>>>>>>>>> not.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>> >>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>> >>>>>>>>>> storage).
>> >>>>>>>>>>>>>>> This
>> >>>>>>>>>>>>>>>>>>>> kind of
>> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>> >>>>>>>>>> changes.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
>> >>>>>>> creation
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>> deletion
>> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
>> >>>> him
>> >>>>>>> in
>> >>>>>>>>>>>>>>> Portland
>> >>>>>>>>>>>>>>>>>>>> back
>> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
>> >>>>>>> realized
>> >>>>>>>>>>>>>>> there
>> >>>>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless
>> >>>> the
>> >>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
>> >>>> create
>> >>>>>>> the
>> >>>>>>>>>>>>>>>>>>>> appropriate
>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>> >>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
>> >>>>>>> Storage
>> >>>>>>>>>>>>>>> layer
>> >>>>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see
>> >> the
>> >>>>>>>>>> Storage
>> >>>>>>>>>>>>>>>>>>>> services
>> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>> >> composed/driven
>> >>>>>>> by
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>> Hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
>> >>>> per
>> >>>>>>>>>> their
>> >>>>>>>>>>>>>>>>>> needs.
>> >>>>>>>>>>>>>>>>>>>> If
>> >>>>>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
>> >>>> with a
>> >>>>>>>>>>>>>>> massive
>> >>>>>>>>>>>>>>>>>>>> n-to-n
>> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>> >>>>>>> difficult to
>> >>>>>>>>>>>>>>>>>> maintain
>> >>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>> >>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>> code
>> >>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>> >>>>>>>>>> XenServer,
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> admin
>> >>>>>>>>>>>>>>>>>>>>>>>> had
>> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
>> >> then
>> >>>> go
>> >>>>>>>>>> into
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
>> >> volume
>> >>>>>>> (ex.
>> >>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
>> >>>> VMs
>> >>>>>>> and
>> >>>>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>>>>> disks
>> >>>>>>>>>>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need
>> >> no
>> >>>>>>>>>> longer
>> >>>>>>>>>>>>>>> be so
>> >>>>>>>>>>>>>>>>>>>>>>>> static
>> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
>> >> between
>> >>>> a
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>> system's
>> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
>> >> storage
>> >>>>>>>>>> Quality
>> >>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>> Service).
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>> >>>>>>> dynamically
>> >>>>>>>>>>>>>>> create
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> delete
>> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
>> >>>> address
>> >>>>>>> is
>> >>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>> creating
>> >>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
>> >> when
>> >>>>>>>>>>>>>>> performing an
>> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do
>> >> so.
>> >>>>>>> I've
>> >>>>>>>>>>>>>>> got
>> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
>> >>>> sandbox
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>> submit
>> >>>>>>>>>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>> >>>>>>> agnostic.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
>> >>>>>>> agent
>> >>>>>>>>>>>>>>> code).
>> >>>>>>>>>>>>>>>>>> The
>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
>> >> create a
>> >>>>>>>>>> volume
>> >>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>>>> needed,
>> >>>>>>>>>>>>>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>> >>>>>>> hypervisor,
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>> >>>>>>> invoked on
>> >>>>>>>>>>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
>> >> create,
>> >>>>>>>>>> say, a
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>> >>>>>>> VMware) to
>> >>>>>>>>>>>>>>> make
>> >>>>>>>>>>>>>>>>>> use
>> >>>>>>>>>>>>>>>>>>>>>>>> of the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making
>> >>>> the
>> >>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>> >>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>> specific?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>> >>>>>>> hypervisors
>> >>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2
>> >> to
>> >>>>>>> get
>> >>>>>>>>>> KVM
>> >>>>>>>>>>>>>>> in.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>> >>>>>>> Depending on
>> >>>>>>>>>>>>>>> what
>> >>>>>>>>>>>>>>>>>> we do
>> >>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working
>> >>>> for
>> >>>>>>>>>>>>>>> VMware in
>> >>>>>>>>>>>>>>>>>> my
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards
>> >> to
>> >>>>>>> Disk
>> >>>>>>>>>>>>>>>>>> Offerings.
>> >>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
>> >>>>>>> Damage
>> >>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
>> >> patch
>> >>>>>>> cycle
>> >>>>>>>>>>>>>>> post
>> >>>>>>>>>>>>>>>>>> 4.2?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>> >>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
>> >> MASTER
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
>> >>>> mine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
>> >> non
>> >>>>>>> issue
>> >>>>>>>>>>>>>>> as I
>> >>>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
>> >> now,
>> >>>>>>>>>> users
>> >>>>>>>>>>>>>>> need
>> >>>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>> >>>>>>>>>>>>>>> service_offering
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>> >>>>>>> calculation.
>> >>>>>>>>>> Of
>> >>>>>>>>>>>>>>> course
>> >>>>>>>>>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>> >>>>>>> individual
>> >>>>>>>>>>>>>>> volume,
>> >>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work
>> >> on
>> >>>> it
>> >>>>>>>>>>>>>>> later. Do
>> >>>>>>>>>>>>>>>>>> you
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>> >> features.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying
>> >> to
>> >>>>>>>>>> adding
>> >>>>>>>>>>>>>>> more.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>> >>>>>>> statistics
>> >>>>>>>>>>>>>>> rather
>> >>>>>>>>>>>>>>>>>> than
>> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
>> >> you
>> >>>>>>> have
>> >>>>>>>>>>>>>>> time.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>> >>>>>>>>>> read/write
>> >>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
>> >>>> more
>> >>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>> volumes
>> >>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>> >>>>>>> operations (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
>> >> imagine
>> >>>>>>> our
>> >>>>>>>>>>>>>>> users
>> >>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> like
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could
>> >> an
>> >>>>>>>>>>>>>>> operator
>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>> >>>>>>> supports
>> >>>>>>>>>>>>>>> setting
>> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
>> >>>> through
>> >>>>>>>>>>>>>>> command
>> >>>>>>>>>>>>>>>>>> line.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>> >>>>>>> parameters
>> >>>>>>>>>>>>>>> of a
>> >>>>>>>>>>>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>> >>>> apologize
>> >>>>>>>>>> for
>> >>>>>>>>>>>>>>> being
>> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
>> >> with a
>> >>>>>>> set
>> >>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>> >>>>>>>>>> resource?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
>> >> better
>> >>>> to
>> >>>>>>>>>>>>>>> commit
>> >>>>>>>>>>>>>>>>>> Mike's
>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider
>> >>>> the
>> >>>>>>>>>>>>>>>>>> limitation in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>> >>>> response
>> >>>>>>> to
>> >>>>>>>>>>>>>>> Mike
>> >>>>>>>>>>>>>>>>>>>>>>>> regarding
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
>> >>>> think
>> >>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but
>> >> I
>> >>>>>>> think
>> >>>>>>>>>>>>>>> we need
>> >>>>>>>>>>>>>>>>>>>>>>>> closer
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
>> >>>> have
>> >>>>>>>>>> have
>> >>>>>>>>>>>>>>>>>> solidfire
>> >>>>>>>>>>>>>>>>>>>>>>>>> track
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
>> >>>> issue.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>> >>>>>>> following:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>> >> policies
>> >>>> be
>> >>>>>>>>>>>>>>> captured
>> >>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>> >> purposes,
>> >>>>>>>>>>>>>>> expressing
>> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model
>> >>>> for
>> >>>>>>>>>>>>>>> reasoning
>> >>>>>>>>>>>>>>>>>>>>>>>> about
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>> >>>>>>>>>>>>>>> configurations
>> >>>>>>>>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled?
>> >> In
>> >>>>>>>>>>>>>>> particular,
>> >>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
>> >> IOPS
>> >>>>>>>>>>>>>>> configured
>> >>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>> a VM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>> >>>> deliver.
>> >>>>>>>>>>>>>>> Another
>> >>>>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>> >>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>> device at
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
>> >>>>>>> device
>> >>>>>>>>>>>>>>>>>> introduces
>> >>>>>>>>>>>>>>>>>>>>>>>>> logical
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski
>> >> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>> >>>> controlling
>> >>>>>>> the
>> >>>>>>>>>>>>>>> max
>> >>>>>>>>>>>>>>>>>>>>>>>> number of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
>> >>>> from
>> >>>>>>> the
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>> system
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
>> >> Burwell
>> >>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
>> >>>>>>> merged
>> >>>>>>>>>>>>>>> until all
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready
>> >> to
>> >>>>>>> turn
>> >>>>>>>>>>>>>>> over to
>> >>>>>>>>>>>>>>>>>>>>>>>> test.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue
>> >>>> is
>> >>>>>>>>>>>>>>> addressed
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>> >>>>>>>>>>>>>>> satisfied.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>> >>>> intersect/compliment
>> >>>>>>> the
>> >>>>>>>>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
>> >> As I
>> >>>>>>>>>>>>>>> understand
>> >>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
>> >>>> like
>> >>>>>>> to
>> >>>>>>>>>>>>>>> ensure
>> >>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>> >>>>>>>>>>>>>>> SolidFire are
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>> >> wido@widodh.nl
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
>> >>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
>> >> reason
>> >>>>>>> till
>> >>>>>>>>>>>>>>> Monday,
>> >>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
>> >>>> mention
>> >>>>>>>>>> again
>> >>>>>>>>>>>>>>>>>> that we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
>> >>>>>>> never
>> >>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>> version
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>> >>>> throttling
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>> that's
>> >>>>>>>>>>>>>>>>>>>>>>>> great!
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from
>> >>>> day
>> >>>>>>>>>> #1.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >>>> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
>> >> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
>> >>>>>>> support
>> >>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
>> >>>> because
>> >>>>>>> I
>> >>>>>>>>>>>>>>> don't see
>> >>>>>>>>>>>>>>>>>>>>>>>> how we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>> >>>>>>>>>> Therefore, we
>> >>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>> >>>>>>> backwards
>> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
>> >>>> now
>> >>>>>>> so
>> >>>>>>>>>>>>>>> that it
>> >>>>>>>>>>>>>>>>>>>>>>>> will go
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
>> >> before
>> >>>>>>> the
>> >>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
>> >> instead
>> >>>>>>> of
>> >>>>>>>>>>>>>>> Total
>> >>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
>> >> freeze
>> >>>>>>> date.
>> >>>>>>>>>>>>>>> But if
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>> >>>> necessary
>> >>>>>>>>>> to do
>> >>>>>>>>>>>>>>>>>> that.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>> >>>>>>>>>> association
>> >>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>> >>>>>>> Hollander <
>> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>> >> Finally I
>> >>>>>>>>>>>>>>> decided to
>> >>>>>>>>>>>>>>>>>>>>>>>> ignore
>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
>> >> because
>> >>>>>>> the
>> >>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
>> >>>>>>> bytes as
>> >>>>>>>>>>>>>>> well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
>> >>>> disk
>> >>>>>>> I/O
>> >>>>>>>>>>>>>>>>>> completely
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>> >>>>>>>>>> situations
>> >>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
>> >>>> work.
>> >>>>>>> It
>> >>>>>>>>>>>>>>> could be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
>> >>>> freeze,
>> >>>>>>> I
>> >>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But
>> >>>> if
>> >>>>>>> it
>> >>>>>>>>>>>>>>> goes
>> >>>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
>> >> release
>> >>>>>>> with
>> >>>>>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >>>> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
>> >>>>>>> branch
>> >>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master
>> >>>> in
>> >>>>>>> 48
>> >>>>>>>>>>>>>>> hours.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
>> >>>>>>> storage
>> >>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>> (local
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>> >>>> limitation
>> >>>>>>> of
>> >>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>> (such as
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
>> >>>> affect
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
>> >>>> device.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
>> >> and
>> >>>>>>> limit
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>>>> I/O
>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>> >>>>>>> difference
>> >>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>> >>>>>>> different
>> >>>>>>>>>>>>>>> rate
>> >>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>> >>>>>>>>>> collect
>> >>>>>>>>>>>>>>> both
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only
>> >> go
>> >>>>>>> for
>> >>>>>>>>>> a
>> >>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>> value.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
>> >>>>>>> then
>> >>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>> IOps,
>> >>>>>>>>>>>>>>>>>>>>>>>> so it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
>> >>>>>>> would
>> >>>>>>>>>> set
>> >>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>> lower
>> >>>>>>>>>>>>>>>>>>>>>>>>> value
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
>> >> point I
>> >>>>>>>>>> think
>> >>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
>> >>>> place
>> >>>>>>> to
>> >>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>> both,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>> >>>>>>> different
>> >>>>>>>>>>>>>>> values
>> >>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
>> >>>> that
>> >>>>>>> as
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
>> >> this
>> >>>>>>> way?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>> >>>>>>>>>> disk_offering,
>> >>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
>> >> iops)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>> >> https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>
>> >>>>
>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>> >>>>>>>>>> success?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>> >>>>>>> integration)
>> >>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>> included
>> >>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>> >>>>>>>>>> potential
>> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>> >>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>> UI.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
>> >>>> expunge.
>> >>>>>>>>>>>>>>> migrate,
>> >>>>>>>>>>>>>>>>>>>>>>>> restore
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>
>> >>>>
>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>> >> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>> >> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> >> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>> cloud<
>> >> http://solidfire.com/solution/overview/?video=play
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>> >>>
>> >>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the
>> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloudâ„¢
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloudâ„¢
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloudâ„¢
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
Thanks for the response.
"I am fine with the VolumeManager determining whether or not a Volume is
managed (i.e. not based on the StoragePoolType, but an actual isManaged
method), and asking the device driver to allocate resources for the volume
if it is managed."
Are you thinking you'd like to see an isManaged() method added to the
PrimaryDataStoreDriver interface? If it returns true, then the storage
framework could call the manage() (or whatever name) method (which would be
new to the PrimaryDataStoreDriver interface, as well) and this would call
into a new method in the hypervisor code to create, say on XenServer, an SR?
Just want to make sure I'm on the same page with you.
Thanks again, John
On Thu, Jun 6, 2013 at 7:44 AM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Fundamentally, we can't end up with a Storage layer that supports n
> devices types with each specific behaviors of m hypervisors. Such a
> scenario will create an unmaintainable and untestable beast. Therefore, my
> thoughts and recommendations are driven to evolve the Storage layer towards
> this separation.
>
> I am fine with the VolumeManager determining whether or not a Volume is
> managed (i.e. not based on the StoragePoolType, but an actual isManaged
> method), and asking the device driver to allocate resources for the volume
> if it is managed. Furthermore, the device driver needs to indicate whether
> or not it supports management operations. Finally, I think we need to
> provide the ability for an administrator to elect to have something that is
> manageable be unmanaged (i.e. the driver is capable managing the device,
> but the administrator has elected to leave it unmanaged).
>
> Creation of a structure on the volume should be done in the Xen hypervisor
> module using methods exposed by the Storage layer to perform low-level
> operations (e.g. make directories, create a file, etc). This structure is
> specific to the operation of the Xen hypervisor, as such, should be
> confined to its implementation. From my perspective, nothing in the
> Storage layer should be concerned with content. From its perspective,
> structure and data are opaque. It provides the means to query the data to
> support the interpretation of the content by higher-level layers (e.g.
> Hypervisors). To my mind, attach should be a composition of operations
> from the Storage layer that varies based on the Volume storage protocol
> (iSCSI, local file system, NFS, RBD, etc).
>
> Thanks,
> -John
>
> On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> Hi John,
>
> Alternatively to the way the attach logic is implemented in my patch, we
> could do the following:
>
> Leave the attach logic in the agent code alone. In VolumeManagerImpl we
> create an AttachVolumeCommand and send it to the hypervisor. Before this
> command is sent, we could check to see if we're dealing with Dynamic (or
> whatever we want to call it) storage and - if so - send a "Create SR"
> command to the hypervisor. If this returns OK, we would then proceed to the
> AttachVolumeCommand, as usual.
>
> This way the attach logic remains the same and we just add another command
> to the agent code that is called for this particular type of storage.
>
> What do you think?
>
>
> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Hey John,
>>
>> I created a document for a customer today that outlines how the plug-in
>> works from a user standpoint. This will probably be of use to you, as well,
>> as you perform the code review.
>>
>> I have shared this document with you (you should have received that
>> information in a separate e-mail).
>>
>> Talk to you later!
>>
>>
>> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Oh, OK, that sounds really good, John.
>>>
>>> Thanks and talk to you tomorrow! :)
>>>
>>>
>>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
>>>
>>>> Mike,
>>>>
>>>> I am never at a loss for an opinion. I some thoughts, but want to
>>>> confirm assumptions and ideas against the solidfire, disk_io_throttle,
>>>> and
>>>> object_store branches. I hope to collect them in a coherent form
>>>> tomorrow (5 June 2013).
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>>
>>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>> > "So, in essence, the SolidFire plugin introduces the notion of a
>>>> managed
>>>> > iSCSI device and provisioned IOPS."
>>>> >
>>>> > Technically, the SolidFire plug-in just introduces the notion of
>>>> > provisioned storage IOPS.
>>>> >
>>>> > The storage framework that leverages the plug-in was incomplete, so I
>>>> had
>>>> > to try to add in the notion of a managed iSCSI device.
>>>> >
>>>> > I appreciate all the time you've been spending on this. :) Do you
>>>> have a
>>>> > recommendation as to how we should accomplish what you're looking for?
>>>> >
>>>> > Thanks!
>>>> >
>>>> >
>>>> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>> >
>>>> >> Mike,
>>>> >>
>>>> >> So, in essence, the SolidFire plugin introduces the notion of a
>>>> managed
>>>> >> iSCSI device and provisioned IOPS.
>>>> >>
>>>> >> I want to see a separation of the management capabilities (i.e. can a
>>>> >> device be managed/does an operator want it managed by CloudStack)
>>>> from the
>>>> >> storage protocol. Ideally, we should end up with a semantic that
>>>> will
>>>> >> allow any type of storage device to be managed. I also want to make
>>>> >> progress on decoupling the storage types from the hypervisor
>>>> definitions.
>>>> >>
>>>> >> Thanks,
>>>> >> -John
>>>> >>
>>>> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>> >> wrote:
>>>> >>
>>>> >>> Hi John,
>>>> >>>
>>>> >>> No problem. Answers are below in red.
>>>> >>>
>>>> >>> Thanks!
>>>> >>>
>>>> >>>
>>>> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>> >>>
>>>> >>>> Mike,
>>>> >>>>
>>>> >>>> Could you please answer the following questions for me with
>>>> regards to
>>>> >> the
>>>> >>>> operation of the SolidFire plugin:
>>>> >>>>
>>>> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>>> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
>>>> >>> 1 SAN volume : 1 LUN
>>>> >>>
>>>> >>>> What is the cardinality between SAN Volumes and CloudStack Volumes?
>>>> >>> 1 SAN volume : 1 CloudStack volume
>>>> >>>
>>>> >>>> Are the LUN(s) created by the management server or externally by
>>>> the
>>>> >>>> operator?
>>>> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN
>>>> LUN) is
>>>> >>> created by the management server (via the plug-in) the first time
>>>> the
>>>> >>> CloudStack volume is attached to a hypervisor.
>>>> >>>
>>>> >>> If you don't want to use the SolidFire plug-in, but still want to
>>>> use a
>>>> >>> SolidFire volume (LUN), you can do this already today (prior to
>>>> 4.2). The
>>>> >>> admin manually creates the SAN volume and - in this case - multiple
>>>> VMs
>>>> >> and
>>>> >>> data disks can share this SAN volume. While you can do this today,
>>>> it is
>>>> >>> not useful if you want to enforce storage QoS.
>>>> >>>
>>>> >>>> Are the SAN volumes by the management server or externally by the
>>>> >> operator?
>>>> >>> When the SolidFire plug-in is used, the SAN volumes are completely
>>>> >> managed
>>>> >>> by the management server (via the plug-in). There is no admin
>>>> >> interaction.
>>>> >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
>>>> >> volume,
>>>> >>> which is necessary for any storage vendor that supports true QoS.
>>>> >>>
>>>> >>>>
>>>> >>>> I would like to clarify how these pieces are related and expected
>>>> to
>>>> >>>> operate.
>>>> >>>>
>>>> >>>> Thanks,
>>>> >>>> -John
>>>> >>>>
>>>> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>>> >> mike.tutkowski@solidfire.com>
>>>> >>>> wrote:
>>>> >>>>
>>>> >>>>> "In particular, how do we ensure that multiple VMs with
>>>> provisioned
>>>> >> IOPS
>>>> >>>>> won't be cut off by the underlying storage."
>>>> >>>>>
>>>> >>>>> In the storage QoS world, we need to map a single SAN volume
>>>> (LUN) to a
>>>> >>>>> single CloudStack volume. We cannot have multiple CloudStack
>>>> volumes
>>>> >>>>> sharing a single SAN volume and still guarantee QoS.
>>>> >>>>>
>>>> >>>>> If the user wants to have a single SAN volume house more than one
>>>> >>>>> CloudStack volume, then can do that today without any of my
>>>> plug-in
>>>> >> code.
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>> >>>> mike.tutkowski@solidfire.com
>>>> >>>>>> wrote:
>>>> >>>>>
>>>> >>>>>> "The administrator will allocate a SAN volume for CloudStack's
>>>> use
>>>> >> onto
>>>> >>>>>> which CloudStack volumes will be created."
>>>> >>>>>>
>>>> >>>>>> I think we crossed e-mails. :)
>>>> >>>>>>
>>>> >>>>>> Check out my recent e-mail on this.
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jburwell@basho.com
>>>> >
>>>> >>>> wrote:
>>>> >>>>>>
>>>> >>>>>>> Mike,
>>>> >>>>>>>
>>>> >>>>>>> You are coming to the part which concerns me -- concepts from
>>>> the
>>>> >>>>>>> hypervisor are leaking into storage layer.
>>>> >>>>>>>
>>>> >>>>>>> Thanks,
>>>> >>>>>>> -John
>>>> >>>>>>>
>>>> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>> >>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>> wrote:
>>>> >>>>>>>
>>>> >>>>>>>> The weird part is that the iSCSI type is today only used (as
>>>> far as
>>>> >> I
>>>> >>>>>>> know)
>>>> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>>>> >>>>>>>>
>>>> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the
>>>> vmfs
>>>> >>>> type.
>>>> >>>>>>>>
>>>> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>>>> >>>>>>> SharedMountPoint
>>>> >>>>>>>> type.
>>>> >>>>>>>>
>>>> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a
>>>> new type
>>>> >>>> of
>>>> >>>>>>>> storage to describe this dynamic ability Edison added into the
>>>> >> storage
>>>> >>>>>>>> framework. Maybe it should be more specificy, though:
>>>> Dynamic_iSCSI
>>>> >>>>>>> versus,
>>>> >>>>>>>> say, Dynamic_FC.
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>>> >>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>> wrote:
>>>> >>>>>>>>
>>>> >>>>>>>>> "The storage device itself shouldn't know or care that it is
>>>> being
>>>> >>>> used
>>>> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it is
>>>> >>>>>>> storing."
>>>> >>>>>>>>>
>>>> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in
>>>> needs to
>>>> >>>>>>> call
>>>> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
>>>> >> treat
>>>> >>>>>>> it as
>>>> >>>>>>>>> such.
>>>> >>>>>>>>>
>>>> >>>>>>>>> I'm totally open to removing that constraint and just calling
>>>> it
>>>> >>>> iSCSI
>>>> >>>>>>> or
>>>> >>>>>>>>> whatever. We would just need a way for the hypervisor attach
>>>> logic
>>>> >> to
>>>> >>>>>>>>> detect this new requirement and perform the necessary
>>>> activities.
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>>>> jburwell@basho.com>
>>>> >>>>>>> wrote:
>>>> >>>>>>>>>
>>>> >>>>>>>>>> Mike,
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> See my responses in-line.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Thanks,
>>>> >>>>>>>>>> -John
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>>> >>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>> I'm trying to picture this:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device,
>>>> an
>>>> >>>>>>> operator
>>>> >>>>>>>>>> may
>>>> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device
>>>> is
>>>> >>>> shared
>>>> >>>>>>> by
>>>> >>>>>>>>>>> many services, and the operator has chosen to dedicate only
>>>> a
>>>> >>>> portion
>>>> >>>>>>>>>> of it
>>>> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
>>>> plug-in
>>>> >>>>>>> (before
>>>> >>>>>>>>>> an
>>>> >>>>>>>>>>> attach request to the hypervisor) would work unless the
>>>> >> hypervisor
>>>> >>>>>>>>>> consumes
>>>> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator
>>>> >>>> allocates
>>>> >>>>>>> a
>>>> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN.
>>>> They
>>>> >>>>>>> simply
>>>> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but not
>>>> allow
>>>> >>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be
>>>> >> passing
>>>> >>>>>>> in a
>>>> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor
>>>> support
>>>> >>>> (like
>>>> >>>>>>> an
>>>> >>>>>>>>>> SR)
>>>> >>>>>>>>>>> and the logic will fail.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Are you thinking we should maybe have the storage framework
>>>> >> itself
>>>> >>>>>>>>>> detect
>>>> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor
>>>> side and
>>>> >>>>>>> have
>>>> >>>>>>>>>> it
>>>> >>>>>>>>>>> call into the agent code specifically to create the SR
>>>> before the
>>>> >>>>>>> attach
>>>> >>>>>>>>>>> logic runs in the agent code?
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> I think the hypervisor management plugin should have a rich
>>>> enough
>>>> >>>>>>>>>> interface to storage to determine available for volume
>>>> storage.
>>>> >> For
>>>> >>>>>>> Xen,
>>>> >>>>>>>>>> this interface would allow the interrogation of the device to
>>>> >>>>>>> determine the
>>>> >>>>>>>>>> SR is present. The storage device itself shouldn't know or
>>>> care
>>>> >>>>>>> that it
>>>> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>>>> questions
>>>> >>>>>>> about it
>>>> >>>>>>>>>> is storing.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>> >>>>>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>> So, the flow is as follows:
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a
>>>> type of
>>>> >>>>>>>>>> so-called
>>>> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage
>>>> shows
>>>> >>>> up
>>>> >>>>>>> in
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>> applicable zone.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> * The admin creates a Disk Offering that references the
>>>> storage
>>>> >>>> tag
>>>> >>>>>>> of
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>> newly created Primary Storage.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to
>>>> a new
>>>> >>>> row
>>>> >>>>>>> in
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>> cloud.volumes table.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
>>>> (attach
>>>> >>>> disk).
>>>> >>>>>>>>>> This
>>>> >>>>>>>>>>>> leads to the storage framework calling the plug-in to
>>>> create a
>>>> >> new
>>>> >>>>>>>>>> volume
>>>> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
>>>> >>>> updates
>>>> >>>>>>> the
>>>> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of
>>>> the SAN
>>>> >>>>>>>>>> volume).
>>>> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume
>>>> is in
>>>> >>>> the
>>>> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in
>>>> the
>>>> >>>>>>> 'Ready'
>>>> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code will
>>>> not
>>>> >> be
>>>> >>>>>>>>>> called
>>>> >>>>>>>>>>>> again to create this SAN volume.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>>>> CloudStack
>>>> >>>>>>> volume
>>>> >>>>>>>>>> to
>>>> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>>>> >>>> structure
>>>> >>>>>>>>>> (ex.
>>>> >>>>>>>>>>>> an SR on XenServer).
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in
>>>> these
>>>> >>>>>>>>>>>>> definitions, refers to the physical allocation on the
>>>> device,
>>>> >>>>>>>>>> correct?"
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>>>> CloudStack, I
>>>> >>>>>>> try to
>>>> >>>>>>>>>>>>> use 'CloudStack volume'.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> Hi John,
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm
>>>> having a
>>>> >>>> hard
>>>> >>>>>>>>>> time
>>>> >>>>>>>>>>>>>> envisioning it.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as
>>>> he
>>>> >> was
>>>> >>>>>>> the
>>>> >>>>>>>>>>>>>> initial person to suggest the approach I took.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> What do you think?
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> Thanks!
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>>>> >>>> jburwell@basho.com
>>>> >>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
>>>> >> storage
>>>> >>>>>>>>>> device
>>>> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we are
>>>> >>>>>>>>>> communicating with
>>>> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete
>>>> volumes
>>>> >>>>>>> (Dynamic)
>>>> >>>>>>>>>> on the
>>>> >>>>>>>>>>>>>>> device while the other requires the volume to be volume
>>>> to be
>>>> >>>>>>>>>> managed
>>>> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we
>>>> are in
>>>> >>>>>>> sync on
>>>> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
>>>> >>>> physical
>>>> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must
>>>> be
>>>> >> able
>>>> >>>> to
>>>> >>>>>>>>>>>>>>> communicate with a storage device to move bits from one
>>>> place
>>>> >>>> to
>>>> >>>>>>>>>> another,
>>>> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage
>>>> device
>>>> >> may
>>>> >>>> be
>>>> >>>>>>>>>> able to
>>>> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
>>>> unmanaged
>>>> >>>> iSCSI
>>>> >>>>>>>>>> device
>>>> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>>>> >>>> SolidFire
>>>> >>>>>>>>>> iSCSI
>>>> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating
>>>> LUNs and
>>>> >>>>>>> storing
>>>> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
>>>> manage a
>>>> >>>>>>> device,
>>>> >>>>>>>>>> an
>>>> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack
>>>> (e.g.
>>>> >>>> the
>>>> >>>>>>>>>> device is
>>>> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
>>>> >>>> dedicate
>>>> >>>>>>>>>> only a
>>>> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make
>>>> sense?
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>>>> >> appropriate
>>>> >>>>>>> to
>>>> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add
>>>> the
>>>> >>>>>>> notion
>>>> >>>>>>>>>> of a
>>>> >>>>>>>>>>>>>>> storage device with an attached driver that indicates
>>>> whether
>>>> >>>> or
>>>> >>>>>>>>>> not is
>>>> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>>>> >>>> representing a
>>>> >>>>>>>>>> physical
>>>> >>>>>>>>>>>>>>> allocation on a device separate that is associated with
>>>> a
>>>> >>>> volume.
>>>> >>>>>>>>>> With
>>>> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare
>>>> which
>>>> >>>>>>>>>> protocols they
>>>> >>>>>>>>>>>>>>> support and when they encounter a device managed by
>>>> >> CloudStack,
>>>> >>>>>>>>>> utilize the
>>>> >>>>>>>>>>>>>>> management operations exposed by the driver to automate
>>>> >>>>>>> allocation.
>>>> >>>>>>>>>> If
>>>> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit
>>>> down and
>>>> >>>>>>> drill
>>>> >>>>>>>>>> down to
>>>> >>>>>>>>>>>>>>> a more detailed design.
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type
>>>> and
>>>> >> the
>>>> >>>>>>>>>> Dynamic
>>>> >>>>>>>>>>>>>>> type:
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
>>>> >>>> Storage
>>>> >>>>>>>>>> based
>>>> >>>>>>>>>>>>>>> on
>>>> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI
>>>> volume must
>>>> >>>>>>> exist
>>>> >>>>>>>>>> on
>>>> >>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
>>>> >>>> volumes
>>>> >>>>>>> are
>>>> >>>>>>>>>>>>>>> created
>>>> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
>>>> >>>> Storage
>>>> >>>>>>>>>> based
>>>> >>>>>>>>>>>>>>> on a
>>>> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
>>>> storage
>>>> >>>>>>> system
>>>> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework).
>>>> When
>>>> >> a
>>>> >>>>>>> user
>>>> >>>>>>>>>>>>>>> wants to
>>>> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>>>> framework
>>>> >>>> tells
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
>>>> >> attach
>>>> >>>>>>>>>> logic
>>>> >>>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor
>>>> data
>>>> >>>>>>> structure
>>>> >>>>>>>>>>>>>>> exists
>>>> >>>>>>>>>>>>>>>> at this point because the volume was just created. The
>>>> >>>>>>> hypervisor
>>>> >>>>>>>>>> data
>>>> >>>>>>>>>>>>>>>> structure must be created.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have
>>>> defined them
>>>> >>>> up
>>>> >>>>>>>>>> front
>>>> >>>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>> you. :)
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an
>>>> admin
>>>> >>>>>>>>>> creates a
>>>> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
>>>> >> created
>>>> >>>>>>> to
>>>> >>>>>>>>>>>>>>> consume
>>>> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor
>>>> data
>>>> >>>>>>>>>> structure
>>>> >>>>>>>>>>>>>>> is
>>>> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed
>>>> on
>>>> >> this
>>>> >>>>>>>>>>>>>>> hypervisor
>>>> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach
>>>> logic
>>>> >>>>>>> assumes
>>>> >>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and
>>>> simply
>>>> >>>>>>> attaches
>>>> >>>>>>>>>>>>>>> the VDI
>>>> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in
>>>> question.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
>>>> Instead
>>>> >>>> of
>>>> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
>>>> volume on
>>>> >> a
>>>> >>>>>>> SAN,
>>>> >>>>>>>>>>>>>>> but the
>>>> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
>>>> created
>>>> >>>>>>> when an
>>>> >>>>>>>>>>>>>>> attach
>>>> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes
>>>> have not
>>>> >>>>>>> been
>>>> >>>>>>>>>>>>>>> pre-hooked
>>>> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin.
>>>> Once
>>>> >> the
>>>> >>>>>>>>>> attach
>>>> >>>>>>>>>>>>>>> logic
>>>> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
>>>> >> attaches
>>>> >>>>>>> the
>>>> >>>>>>>>>>>>>>> one and
>>>> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>>> >>>>>>> jburwell@basho.com>
>>>> >>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
>>>> >>>> behavior
>>>> >>>>>>>>>>>>>>> works in
>>>> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>>>> difference.
>>>> >>>>>>> Another
>>>> >>>>>>>>>>>>>>> way to
>>>> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
>>>> Dynamic
>>>> >>>>>>> storage
>>>> >>>>>>>>>>>>>>> pool type?
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>>>> >>>>>>> XenServer
>>>> >>>>>>>>>>>>>>> when you
>>>> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly
>>>> based on
>>>> >> an
>>>> >>>>>>>>>> iSCSI
>>>> >>>>>>>>>>>>>>>>>> target.
>>>> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>>>> >> hypervisor
>>>> >>>>>>> and
>>>> >>>>>>>>>>>>>>>>>> creating a
>>>> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>>>> >> CloudStack
>>>> >>>>>>> does
>>>> >>>>>>>>>>>>>>> that
>>>> >>>>>>>>>>>>>>>>>> part
>>>> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
>>>> XenServer.
>>>> >> For
>>>> >>>>>>> all
>>>> >>>>>>>>>>>>>>> other
>>>> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
>>>> >>>> perform
>>>> >>>>>>>>>> this
>>>> >>>>>>>>>>>>>>> step
>>>> >>>>>>>>>>>>>>>>>>> manually.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>>> >>>>>>>>>> jburwell@basho.com>
>>>> >>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
>>>> between
>>>> >>>> the
>>>> >>>>>>>>>>>>>>> ISCSI and
>>>> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>> >>>>>>>>>>>>>>> Storage.StoragePoolType
>>>> >>>>>>>>>>>>>>>>>> class
>>>> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>>>> >> overlay
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is
>>>> set
>>>> >>>> up
>>>> >>>>>>>>>> by
>>>> >>>>>>>>>>>>>>>>>>>>> customers.
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>>> >>>>>>>>>> SolidFire)
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> }
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> return shared;
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> }
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> }
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company
>>>> wants
>>>> >> to
>>>> >>>>>>>>>>>>>>> implement a
>>>> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>>>> feature. It
>>>> >>>>>>> would
>>>> >>>>>>>>>> be
>>>> >>>>>>>>>>>>>>>>>> dynamic,
>>>> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>>>> >>>>>>> implement a
>>>> >>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>> >>>>>>>>>>>>>>> hypervisor-attach
>>>> >>>>>>>>>>>>>>>>>>>> logic
>>>> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
>>>> SolidFire
>>>> >>>>>>> specific.
>>>> >>>>>>>>>>>>>>> It is
>>>> >>>>>>>>>>>>>>>>>> not.
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
>>>> dynamic,
>>>> >>>>>>>>>> zone-wide
>>>> >>>>>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this
>>>> kind of
>>>> >>>>>>>>>> storage).
>>>> >>>>>>>>>>>>>>> This
>>>> >>>>>>>>>>>>>>>>>>>> kind of
>>>> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
>>>> framework
>>>> >>>>>>>>>> changes.
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported
>>>> the
>>>> >>>>>>> creation
>>>> >>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>> deletion
>>>> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited
>>>> with
>>>> >>>> him
>>>> >>>>>>> in
>>>> >>>>>>>>>>>>>>> Portland
>>>> >>>>>>>>>>>>>>>>>>>> back
>>>> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete.
>>>> We
>>>> >>>>>>> realized
>>>> >>>>>>>>>>>>>>> there
>>>> >>>>>>>>>>>>>>>>>> was
>>>> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes
>>>> unless
>>>> >>>> the
>>>> >>>>>>>>>>>>>>> attach
>>>> >>>>>>>>>>>>>>>>>> logic
>>>> >>>>>>>>>>>>>>>>>>>> was
>>>> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage
>>>> and
>>>> >>>> create
>>>> >>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>> appropriate
>>>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>> >>>>>>>>>>>>>>> jburwell@basho.com>
>>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation
>>>> in the
>>>> >>>>>>> Storage
>>>> >>>>>>>>>>>>>>> layer
>>>> >>>>>>>>>>>>>>>>>>>> would
>>>> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to
>>>> see
>>>> >> the
>>>> >>>>>>>>>> Storage
>>>> >>>>>>>>>>>>>>>>>>>> services
>>>> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>>>> >> composed/driven
>>>> >>>>>>> by
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>> Hypervisor
>>>> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>>>> structures
>>>> >>>> per
>>>> >>>>>>>>>> their
>>>> >>>>>>>>>>>>>>>>>> needs.
>>>> >>>>>>>>>>>>>>>>>>>> If
>>>> >>>>>>>>>>>>>>>>>>>>>>>> we
>>>> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to
>>>> end
>>>> >>>> with a
>>>> >>>>>>>>>>>>>>> massive
>>>> >>>>>>>>>>>>>>>>>>>> n-to-n
>>>> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>>>> >>>>>>> difficult to
>>>> >>>>>>>>>>>>>>>>>> maintain
>>>> >>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
>>>> specific
>>>> >>>>>>>>>> SolidFire
>>>> >>>>>>>>>>>>>>> code
>>>> >>>>>>>>>>>>>>>>>> is
>>>> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski
>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method
>>>> for
>>>> >>>>>>>>>> XenServer,
>>>> >>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>> admin
>>>> >>>>>>>>>>>>>>>>>>>>>>>> had
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage
>>>> system,
>>>> >> then
>>>> >>>> go
>>>> >>>>>>>>>> into
>>>> >>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
>>>> >> volume
>>>> >>>>>>> (ex.
>>>> >>>>>>>>>> a
>>>> >>>>>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
>>>> ESX(i)).
>>>> >>>> VMs
>>>> >>>>>>> and
>>>> >>>>>>>>>>>>>>> data
>>>> >>>>>>>>>>>>>>>>>> disks
>>>> >>>>>>>>>>>>>>>>>>>>>>>> then
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage
>>>> need
>>>> >> no
>>>> >>>>>>>>>> longer
>>>> >>>>>>>>>>>>>>> be so
>>>> >>>>>>>>>>>>>>>>>>>>>>>> static
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
>>>> >> between
>>>> >>>> a
>>>> >>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>> system's
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
>>>> >> storage
>>>> >>>>>>>>>> Quality
>>>> >>>>>>>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>>>> >>>>>>> dynamically
>>>> >>>>>>>>>>>>>>> create
>>>> >>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>> delete
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
>>>> >>>> address
>>>> >>>>>>> is
>>>> >>>>>>>>>> in
>>>> >>>>>>>>>>>>>>>>>> creating
>>>> >>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data
>>>> structure
>>>> >> when
>>>> >>>>>>>>>>>>>>> performing an
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it
>>>> to do
>>>> >> so.
>>>> >>>>>>> I've
>>>> >>>>>>>>>>>>>>> got
>>>> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in
>>>> my
>>>> >>>> sandbox
>>>> >>>>>>>>>> and
>>>> >>>>>>>>>>>>>>> can
>>>> >>>>>>>>>>>>>>>>>>>> submit
>>>> >>>>>>>>>>>>>>>>>>>>>>>> this
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike
>>>> Tutkowski <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
>>>> hypervisor
>>>> >>>>>>> agnostic.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic
>>>> (in the
>>>> >>>>>>> agent
>>>> >>>>>>>>>>>>>>> code).
>>>> >>>>>>>>>>>>>>>>>> The
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
>>>> >> create a
>>>> >>>>>>>>>> volume
>>>> >>>>>>>>>>>>>>> as
>>>> >>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> but
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>>>> >>>>>>> hypervisor,
>>>> >>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>> attach
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's
>>>> being
>>>> >>>>>>> invoked on
>>>> >>>>>>>>>>>>>>>>>> zone-wide
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
>>>> >> create,
>>>> >>>>>>>>>> say, a
>>>> >>>>>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore
>>>> (for
>>>> >>>>>>> VMware) to
>>>> >>>>>>>>>>>>>>> make
>>>> >>>>>>>>>>>>>>>>>> use
>>>> >>>>>>>>>>>>>>>>>>>>>>>> of the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently
>>>> making
>>>> >>>> the
>>>> >>>>>>>>>>>>>>> attach
>>>> >>>>>>>>>>>>>>>>>> logic
>>>> >>>>>>>>>>>>>>>>>>>>>>>> work
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John
>>>> Burwell <
>>>> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver
>>>> is
>>>> >>>>>>> hypervisor
>>>> >>>>>>>>>>>>>>>>>> specific?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>>>> >>>>>>> hypervisors
>>>> >>>>>>>>>>>>>>> that
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for
>>>> 4.2
>>>> >> to
>>>> >>>>>>> get
>>>> >>>>>>>>>> KVM
>>>> >>>>>>>>>>>>>>> in.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>>>> >>>>>>> Depending on
>>>> >>>>>>>>>>>>>>> what
>>>> >>>>>>>>>>>>>>>>>> we do
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> with
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
>>>> working
>>>> >>>> for
>>>> >>>>>>>>>>>>>>> VMware in
>>>> >>>>>>>>>>>>>>>>>> my
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
>>>> regards
>>>> >> to
>>>> >>>>>>> Disk
>>>> >>>>>>>>>>>>>>>>>> Offerings.
>>>> >>>>>>>>>>>>>>>>>>>>>>>> I
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
>>>> Jamison
>>>> >>>>>>> Damage
>>>> >>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
>>>> >> patch
>>>> >>>>>>> cycle
>>>> >>>>>>>>>>>>>>> post
>>>> >>>>>>>>>>>>>>>>>> 4.2?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>>> >>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
>>>> >> MASTER
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first,
>>>> then
>>>> >>>> mine.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it
>>>> is a
>>>> >> non
>>>> >>>>>>> issue
>>>> >>>>>>>>>>>>>>> as I
>>>> >>>>>>>>>>>>>>>>>> don't
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
>>>> billing
>>>> >> now,
>>>> >>>>>>>>>> users
>>>> >>>>>>>>>>>>>>> need
>>>> >>>>>>>>>>>>>>>>>> to
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get
>>>> the
>>>> >>>>>>>>>>>>>>> service_offering
>>>> >>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>>>> >>>>>>> calculation.
>>>> >>>>>>>>>> Of
>>>> >>>>>>>>>>>>>>> course
>>>> >>>>>>>>>>>>>>>>>>>>>>>> it is
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value
>>>> of
>>>> >>>>>>> individual
>>>> >>>>>>>>>>>>>>> volume,
>>>> >>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can
>>>> work
>>>> >> on
>>>> >>>> it
>>>> >>>>>>>>>>>>>>> later. Do
>>>> >>>>>>>>>>>>>>>>>> you
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>>>> question.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>>>> >> features.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>> jburwell@basho.com>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
>>>> trying
>>>> >> to
>>>> >>>>>>>>>> adding
>>>> >>>>>>>>>>>>>>> more.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>>>> >>>>>>> statistics
>>>> >>>>>>>>>>>>>>> rather
>>>> >>>>>>>>>>>>>>>>>> than
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat
>>>> if
>>>> >> you
>>>> >>>>>>> have
>>>> >>>>>>>>>>>>>>> time.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>>>> bytes/iops
>>>> >>>>>>>>>> read/write
>>>> >>>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>>> an
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are
>>>> billed
>>>> >>>> more
>>>> >>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>>> volumes
>>>> >>>>>>>>>>>>>>>>>>>>>>>> with
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>>>> >>>>>>> operations (
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
>>>> >> imagine
>>>> >>>>>>> our
>>>> >>>>>>>>>>>>>>> users
>>>> >>>>>>>>>>>>>>>>>> would
>>>> >>>>>>>>>>>>>>>>>>>>>>>> like
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
>>>> Could
>>>> >> an
>>>> >>>>>>>>>>>>>>> operator
>>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change?
>>>> KVM
>>>> >>>>>>> supports
>>>> >>>>>>>>>>>>>>> setting
>>>> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual
>>>> machine
>>>> >>>> through
>>>> >>>>>>>>>>>>>>> command
>>>> >>>>>>>>>>>>>>>>>> line.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing
>>>> the
>>>> >>>>>>> parameters
>>>> >>>>>>>>>>>>>>> of a
>>>> >>>>>>>>>>>>>>>>>>>>>>>> created
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>>>> >>>> apologize
>>>> >>>>>>>>>> for
>>>> >>>>>>>>>>>>>>> being
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
>>>> >> with a
>>>> >>>>>>> set
>>>> >>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees
>>>> provided by a
>>>> >>>>>>>>>> resource?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
>>>> >> better
>>>> >>>> to
>>>> >>>>>>>>>>>>>>> commit
>>>> >>>>>>>>>>>>>>>>>> Mike's
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
>>>> consider
>>>> >>>> the
>>>> >>>>>>>>>>>>>>>>>> limitation in
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>>>> >>>> response
>>>> >>>>>>> to
>>>> >>>>>>>>>>>>>>> Mike
>>>> >>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two
>>>> features. I
>>>> >>>> think
>>>> >>>>>>>>>> that
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before
>>>> SolidFire, but
>>>> >> I
>>>> >>>>>>> think
>>>> >>>>>>>>>>>>>>> we need
>>>> >>>>>>>>>>>>>>>>>>>>>>>> closer
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>>>> (possibly
>>>> >>>> have
>>>> >>>>>>>>>> have
>>>> >>>>>>>>>>>>>>>>>> solidfire
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> track
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on
>>>> this
>>>> >>>> issue.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>>> jburwell@basho.com>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are
>>>> the
>>>> >>>>>>> following:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>>>> >> policies
>>>> >>>> be
>>>> >>>>>>>>>>>>>>> captured
>>>> >>>>>>>>>>>>>>>>>> in
>>>> >>>>>>>>>>>>>>>>>>>>>>>> a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>>>> >> purposes,
>>>> >>>>>>>>>>>>>>> expressing
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface
>>>> model
>>>> >>>> for
>>>> >>>>>>>>>>>>>>> reasoning
>>>> >>>>>>>>>>>>>>>>>>>>>>>> about
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned
>>>> IOPS
>>>> >>>>>>>>>>>>>>> configurations
>>>> >>>>>>>>>>>>>>>>>>>>>>>> between
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>>>> reconciled?
>>>> >> In
>>>> >>>>>>>>>>>>>>> particular,
>>>> >>>>>>>>>>>>>>>>>> a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for
>>>> more
>>>> >> IOPS
>>>> >>>>>>>>>>>>>>> configured
>>>> >>>>>>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>>>> >>>> deliver.
>>>> >>>>>>>>>>>>>>> Another
>>>> >>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM
>>>> and a
>>>> >>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>> device at
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
>>>> storage
>>>> >>>>>>> device
>>>> >>>>>>>>>>>>>>>>>> introduces
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
>>>> Tutkowski
>>>> >> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>>>> >>>> controlling
>>>> >>>>>>> the
>>>> >>>>>>>>>>>>>>> max
>>>> >>>>>>>>>>>>>>>>>>>>>>>> number of
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling
>>>> IOPS
>>>> >>>> from
>>>> >>>>>>> the
>>>> >>>>>>>>>>>>>>> storage
>>>> >>>>>>>>>>>>>>>>>>>>>>>> system
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
>>>> >> Burwell
>>>> >>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features
>>>> should be
>>>> >>>>>>> merged
>>>> >>>>>>>>>>>>>>> until all
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is
>>>> ready
>>>> >> to
>>>> >>>>>>> turn
>>>> >>>>>>>>>>>>>>> over to
>>>> >>>>>>>>>>>>>>>>>>>>>>>> test.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops
>>>> issue
>>>> >>>> is
>>>> >>>>>>>>>>>>>>> addressed
>>>> >>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria
>>>> has been
>>>> >>>>>>>>>>>>>>> satisfied.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>>>> >>>> intersect/compliment
>>>> >>>>>>> the
>>>> >>>>>>>>>>>>>>>>>> SolidFire
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/
>>>> )?
>>>> >> As I
>>>> >>>>>>>>>>>>>>> understand
>>>> >>>>>>>>>>>>>>>>>> it
>>>> >>>>>>>>>>>>>>>>>>>>>>>> that
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I
>>>> would
>>>> >>>> like
>>>> >>>>>>> to
>>>> >>>>>>>>>>>>>>> ensure
>>>> >>>>>>>>>>>>>>>>>> we
>>>> >>>>>>>>>>>>>>>>>>>>>>>> don't
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in
>>>> KVM and
>>>> >>>>>>>>>>>>>>> SolidFire are
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU
>>>> <
>>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>>>> >> wido@widodh.nl
>>>> >>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU
>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features
>>>> into
>>>> >>>>>>> master.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
>>>> >> reason
>>>> >>>>>>> till
>>>> >>>>>>>>>>>>>>> Monday,
>>>> >>>>>>>>>>>>>>>>>> I
>>>> >>>>>>>>>>>>>>>>>>>>>>>> will
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want
>>>> to
>>>> >>>> mention
>>>> >>>>>>>>>> again
>>>> >>>>>>>>>>>>>>>>>> that we
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so
>>>> that we
>>>> >>>>>>> never
>>>> >>>>>>>>>>>>>>> release
>>>> >>>>>>>>>>>>>>>>>> a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> version
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>>>> >>>> throttling
>>>> >>>>>>>>>> and
>>>> >>>>>>>>>>>>>>> that's
>>>> >>>>>>>>>>>>>>>>>>>>>>>> great!
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create
>>>> legacy from
>>>> >>>> day
>>>> >>>>>>>>>> #1.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>> >>>> wido@widodh.nl>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
>>>> >> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>>>> discretely
>>>> >>>>>>> support
>>>> >>>>>>>>>>>>>>> read
>>>> >>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>> write
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
>>>> >>>> because
>>>> >>>>>>> I
>>>> >>>>>>>>>>>>>>> don't see
>>>> >>>>>>>>>>>>>>>>>>>>>>>> how we
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>>>> >>>>>>>>>> Therefore, we
>>>> >>>>>>>>>>>>>>>>>> would
>>>> >>>>>>>>>>>>>>>>>>>>>>>> be
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to
>>>> break
>>>> >>>>>>> backwards
>>>> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into
>>>> master
>>>> >>>> now
>>>> >>>>>>> so
>>>> >>>>>>>>>>>>>>> that it
>>>> >>>>>>>>>>>>>>>>>>>>>>>> will go
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
>>>> >> before
>>>> >>>>>>> the
>>>> >>>>>>>>>> 4.2
>>>> >>>>>>>>>>>>>>>>>> release
>>>> >>>>>>>>>>>>>>>>>>>>>>>> so
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
>>>> >> instead
>>>> >>>>>>> of
>>>> >>>>>>>>>>>>>>> Total
>>>> >>>>>>>>>>>>>>>>>> IOps.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
>>>> >> freeze
>>>> >>>>>>> date.
>>>> >>>>>>>>>>>>>>> But if
>>>> >>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>>>> >>>> necessary
>>>> >>>>>>>>>> to do
>>>> >>>>>>>>>>>>>>>>>> that.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there
>>>> is no
>>>> >>>>>>>>>> association
>>>> >>>>>>>>>>>>>>>>>> between
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>>>> >>>>>>> Hollander <
>>>> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU
>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>>>> >> Finally I
>>>> >>>>>>>>>>>>>>> decided to
>>>> >>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
>>>> >> because
>>>> >>>>>>> the
>>>> >>>>>>>>>>>>>>> network
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
>>>> received
>>>> >>>>>>> bytes as
>>>> >>>>>>>>>>>>>>> well.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking
>>>> and
>>>> >>>> disk
>>>> >>>>>>> I/O
>>>> >>>>>>>>>>>>>>>>>> completely
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in
>>>> most
>>>> >>>>>>>>>> situations
>>>> >>>>>>>>>>>>>>> then
>>>> >>>>>>>>>>>>>>>>>>>>>>>> network
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
>>>> copy-paste
>>>> >>>> work.
>>>> >>>>>>> It
>>>> >>>>>>>>>>>>>>> could be
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
>>>> >>>> freeze,
>>>> >>>>>>> I
>>>> >>>>>>>>>> will
>>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> it
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't
>>>> miss. But
>>>> >>>> if
>>>> >>>>>>> it
>>>> >>>>>>>>>>>>>>> goes
>>>> >>>>>>>>>>>>>>>>>> into
>>>> >>>>>>>>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
>>>> >> release
>>>> >>>>>>> with
>>>> >>>>>>>>>>>>>>> only
>>>> >>>>>>>>>>>>>>>>>> total
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>> >>>> wido@widodh.nl>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU
>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>>>> disk_io_throttling
>>>> >>>>>>> branch
>>>> >>>>>>>>>> into
>>>> >>>>>>>>>>>>>>>>>> master.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into
>>>> master
>>>> >>>> in
>>>> >>>>>>> 48
>>>> >>>>>>>>>>>>>>> hours.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the
>>>> same
>>>> >>>>>>> storage
>>>> >>>>>>>>>>>>>>> device
>>>> >>>>>>>>>>>>>>>>>>>>>>>> (local
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>>>> >>>> limitation
>>>> >>>>>>> of
>>>> >>>>>>>>>>>>>>> device
>>>> >>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it
>>>> may
>>>> >>>> affect
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>>>>> disk
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
>>>> >>>> device.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum
>>>> rate
>>>> >> and
>>>> >>>>>>> limit
>>>> >>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>> disk
>>>> >>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>>>> >>>>>>> difference
>>>> >>>>>>>>>>>>>>> between
>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting
>>>> both a
>>>> >>>>>>> different
>>>> >>>>>>>>>>>>>>> rate
>>>> >>>>>>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
>>>> users.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling
>>>> side you
>>>> >>>>>>>>>> collect
>>>> >>>>>>>>>>>>>>> both
>>>> >>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you
>>>> only
>>>> >> go
>>>> >>>>>>> for
>>>> >>>>>>>>>> a
>>>> >>>>>>>>>>>>>>> global
>>>> >>>>>>>>>>>>>>>>>>>>>>>> value.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
>>>> expensive
>>>> >>>>>>> then
>>>> >>>>>>>>>> Read
>>>> >>>>>>>>>>>>>>>>>> IOps,
>>>> >>>>>>>>>>>>>>>>>>>>>>>> so it
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an
>>>> admin
>>>> >>>>>>> would
>>>> >>>>>>>>>> set
>>>> >>>>>>>>>>>>>>> a
>>>> >>>>>>>>>>>>>>>>>> lower
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> value
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
>>>> >> point I
>>>> >>>>>>>>>> think
>>>> >>>>>>>>>>>>>>> it
>>>> >>>>>>>>>>>>>>>>>> would
>>>> >>>>>>>>>>>>>>>>>>>>>>>> be
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism
>>>> in
>>>> >>>> place
>>>> >>>>>>> to
>>>> >>>>>>>>>>>>>>> support
>>>> >>>>>>>>>>>>>>>>>>>>>>>> both,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
>>>> setting
>>>> >>>>>>> different
>>>> >>>>>>>>>>>>>>> values
>>>> >>>>>>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> read
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and
>>>> set
>>>> >>>> that
>>>> >>>>>>> as
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>> total
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented
>>>> it
>>>> >> this
>>>> >>>>>>> way?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>>> >>>>>>>>>> disk_offering,
>>>> >>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>>>>> global
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps
>>>> and
>>>> >> iops)
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>>> >> https://issues.apache.org/****
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192
>>>> >
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192
>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>> >>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>> >>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://cwiki.apache.org/confluence/**>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>> >>>>>>>>>>>>>>>>>> https://cwiki.
>>>> >>>>>>>>>>>>>>>>>>>>>>>> **
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>
>>>> >>>>
>>>> >>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
>>>> execution
>>>> >>>>>>>>>> success?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>>>> introduced?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>>>> >>>>>>> integration)
>>>> >>>>>>>>>> is
>>>> >>>>>>>>>>>>>>>>>> included
>>>> >>>>>>>>>>>>>>>>>>>>>>>> in
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to
>>>> check for
>>>> >>>>>>>>>> potential
>>>> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate
>>>> on
>>>> >>>>>>>>>> CloudStack
>>>> >>>>>>>>>>>>>>> UI.
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
>>>> >>>> expunge.
>>>> >>>>>>>>>>>>>>> migrate,
>>>> >>>>>>>>>>>>>>>>>>>>>>>> restore
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>> >>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>> >>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://cwiki.apache.org/confluence/**>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>> >>>>>>>>>>>>>>>>>> https://cwiki.
>>>> >>>>>>>>>>>>>>>>>>>>>>>> **
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>
>>>> >>>>
>>>> >>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>> >>>>>>>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301
>>>> >
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301
>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>>>> >> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>> >>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>>>> >> <
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071
>>>> >
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer,
>>>> SolidFire
>>>> >> Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>>> Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://solidfire.com/solution/overview/?video=play
>>>> >>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>> >
>>>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>> >
>>>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>>>>>>> cloud<
>>>> >> http://solidfire.com/solution/overview/?video=play
>>>> >>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play
>>>> >>>
>>>> >>>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> --
>>>> >>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> --
>>>> >>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> --
>>>> >>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>> *â„¢*
>>>> >>>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> --
>>>> >>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>> o: 303.746.7302
>>>> >>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>> *â„¢*
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> *Mike Tutkowski*
>>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>> o: 303.746.7302
>>>> >>>>>> Advancing the way the world uses the cloud<
>>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>> *â„¢*
>>>> >>>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> --
>>>> >>>>> *Mike Tutkowski*
>>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>> o: 303.746.7302
>>>> >>>>> Advancing the way the world uses the
>>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>>>> *â„¢*
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>> --
>>>> >>> *Mike Tutkowski*
>>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>> e: mike.tutkowski@solidfire.com
>>>> >>> o: 303.746.7302
>>>> >>> Advancing the way the world uses the
>>>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>> *â„¢*
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> > --
>>>> > *Mike Tutkowski*
>>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>>> > e: mike.tutkowski@solidfire.com
>>>> > o: 303.746.7302
>>>> > Advancing the way the world uses the
>>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Fundamentally, we can't end up with a Storage layer that supports n devices types with each specific behaviors of m hypervisors. Such a scenario will create an unmaintainable and untestable beast. Therefore, my thoughts and recommendations are driven to evolve the Storage layer towards this separation.
I am fine with the VolumeManager determining whether or not a Volume is managed (i.e. not based on the StoragePoolType, but an actual isManaged method), and asking the device driver to allocate resources for the volume if it is managed. Furthermore, the device driver needs to indicate whether or not it supports management operations. Finally, I think we need to provide the ability for an administrator to elect to have something that is manageable be unmanaged (i.e. the driver is capable managing the device, but the administrator has elected to leave it unmanaged).
Creation of a structure on the volume should be done in the Xen hypervisor module using methods exposed by the Storage layer to perform low-level operations (e.g. make directories, create a file, etc). This structure is specific to the operation of the Xen hypervisor, as such, should be confined to its implementation. From my perspective, nothing in the Storage layer should be concerned with content. From its perspective, structure and data are opaque. It provides the means to query the data to support the interpretation of the content by higher-level layers (e.g. Hypervisors). To my mind, attach should be a composition of operations from the Storage layer that varies based on the Volume storage protocol (iSCSI, local file system, NFS, RBD, etc).
Thanks,
-John
On Jun 5, 2013, at 12:25 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi John,
>
> Alternatively to the way the attach logic is implemented in my patch, we could do the following:
>
> Leave the attach logic in the agent code alone. In VolumeManagerImpl we create an AttachVolumeCommand and send it to the hypervisor. Before this command is sent, we could check to see if we're dealing with Dynamic (or whatever we want to call it) storage and - if so - send a "Create SR" command to the hypervisor. If this returns OK, we would then proceed to the AttachVolumeCommand, as usual.
>
> This way the attach logic remains the same and we just add another command to the agent code that is called for this particular type of storage.
>
> What do you think?
>
>
> On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hey John,
>
> I created a document for a customer today that outlines how the plug-in works from a user standpoint. This will probably be of use to you, as well, as you perform the code review.
>
> I have shared this document with you (you should have received that information in a separate e-mail).
>
> Talk to you later!
>
>
> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Oh, OK, that sounds really good, John.
>
> Thanks and talk to you tomorrow! :)
>
>
> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I am never at a loss for an opinion. I some thoughts, but want to
> confirm assumptions and ideas against the solidfire, disk_io_throttle,
> and
> object_store branches. I hope to collect them in a coherent form
> tomorrow (5 June 2013).
>
> Thanks,
> -John
>
>
> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
>
> > "So, in essence, the SolidFire plugin introduces the notion of a managed
> > iSCSI device and provisioned IOPS."
> >
> > Technically, the SolidFire plug-in just introduces the notion of
> > provisioned storage IOPS.
> >
> > The storage framework that leverages the plug-in was incomplete, so I had
> > to try to add in the notion of a managed iSCSI device.
> >
> > I appreciate all the time you've been spending on this. :) Do you have a
> > recommendation as to how we should accomplish what you're looking for?
> >
> > Thanks!
> >
> >
> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com> wrote:
> >
> >> Mike,
> >>
> >> So, in essence, the SolidFire plugin introduces the notion of a managed
> >> iSCSI device and provisioned IOPS.
> >>
> >> I want to see a separation of the management capabilities (i.e. can a
> >> device be managed/does an operator want it managed by CloudStack) from the
> >> storage protocol. Ideally, we should end up with a semantic that will
> >> allow any type of storage device to be managed. I also want to make
> >> progress on decoupling the storage types from the hypervisor definitions.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
> >> wrote:
> >>
> >>> Hi John,
> >>>
> >>> No problem. Answers are below in red.
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Could you please answer the following questions for me with regards to
> >> the
> >>>> operation of the SolidFire plugin:
> >>>>
> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
> >>> 1 SAN volume : 1 LUN
> >>>
> >>>> What is the cardinality between SAN Volumes and CloudStack Volumes?
> >>> 1 SAN volume : 1 CloudStack volume
> >>>
> >>>> Are the LUN(s) created by the management server or externally by the
> >>>> operator?
> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) is
> >>> created by the management server (via the plug-in) the first time the
> >>> CloudStack volume is attached to a hypervisor.
> >>>
> >>> If you don't want to use the SolidFire plug-in, but still want to use a
> >>> SolidFire volume (LUN), you can do this already today (prior to 4.2). The
> >>> admin manually creates the SAN volume and - in this case - multiple VMs
> >> and
> >>> data disks can share this SAN volume. While you can do this today, it is
> >>> not useful if you want to enforce storage QoS.
> >>>
> >>>> Are the SAN volumes by the management server or externally by the
> >> operator?
> >>> When the SolidFire plug-in is used, the SAN volumes are completely
> >> managed
> >>> by the management server (via the plug-in). There is no admin
> >> interaction.
> >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
> >> volume,
> >>> which is necessary for any storage vendor that supports true QoS.
> >>>
> >>>>
> >>>> I would like to clarify how these pieces are related and expected to
> >>>> operate.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> "In particular, how do we ensure that multiple VMs with provisioned
> >> IOPS
> >>>>> won't be cut off by the underlying storage."
> >>>>>
> >>>>> In the storage QoS world, we need to map a single SAN volume (LUN) to a
> >>>>> single CloudStack volume. We cannot have multiple CloudStack volumes
> >>>>> sharing a single SAN volume and still guarantee QoS.
> >>>>>
> >>>>> If the user wants to have a single SAN volume house more than one
> >>>>> CloudStack volume, then can do that today without any of my plug-in
> >> code.
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com
> >>>>>> wrote:
> >>>>>
> >>>>>> "The administrator will allocate a SAN volume for CloudStack's use
> >> onto
> >>>>>> which CloudStack volumes will be created."
> >>>>>>
> >>>>>> I think we crossed e-mails. :)
> >>>>>>
> >>>>>> Check out my recent e-mail on this.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> You are coming to the part which concerns me -- concepts from the
> >>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> The weird part is that the iSCSI type is today only used (as far as
> >> I
> >>>>>>> know)
> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
> >>>>>>>>
> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
> >>>> type.
> >>>>>>>>
> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
> >>>>>>> SharedMountPoint
> >>>>>>>> type.
> >>>>>>>>
> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a new type
> >>>> of
> >>>>>>>> storage to describe this dynamic ability Edison added into the
> >> storage
> >>>>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
> >>>>>>> versus,
> >>>>>>>> say, Dynamic_FC.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> "The storage device itself shouldn't know or care that it is being
> >>>> used
> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it is
> >>>>>>> storing."
> >>>>>>>>>
> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in needs to
> >>>>>>> call
> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
> >> treat
> >>>>>>> it as
> >>>>>>>>> such.
> >>>>>>>>>
> >>>>>>>>> I'm totally open to removing that constraint and just calling it
> >>>> iSCSI
> >>>>>>> or
> >>>>>>>>> whatever. We would just need a way for the hypervisor attach logic
> >> to
> >>>>>>>>> detect this new requirement and perform the necessary activities.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> See my responses in-line.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>
> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
> >>>>>>> operator
> >>>>>>>>>> may
> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
> >>>> shared
> >>>>>>> by
> >>>>>>>>>>> many services, and the operator has chosen to dedicate only a
> >>>> portion
> >>>>>>>>>> of it
> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>>>>>>>>
> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
> >>>>>>> (before
> >>>>>>>>>> an
> >>>>>>>>>>> attach request to the hypervisor) would work unless the
> >> hypervisor
> >>>>>>>>>> consumes
> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>>>>>
> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator
> >>>> allocates
> >>>>>>> a
> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
> >>>>>>> simply
> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but not allow
> >>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be
> >> passing
> >>>>>>> in a
> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor support
> >>>> (like
> >>>>>>> an
> >>>>>>>>>> SR)
> >>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>
> >>>>>>>>>>> Are you thinking we should maybe have the storage framework
> >> itself
> >>>>>>>>>> detect
> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor side and
> >>>>>>> have
> >>>>>>>>>> it
> >>>>>>>>>>> call into the agent code specifically to create the SR before the
> >>>>>>> attach
> >>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>
> >>>>>>>>>> I think the hypervisor management plugin should have a rich enough
> >>>>>>>>>> interface to storage to determine available for volume storage.
> >> For
> >>>>>>> Xen,
> >>>>>>>>>> this interface would allow the interrogation of the device to
> >>>>>>> determine the
> >>>>>>>>>> SR is present. The storage device itself shouldn't know or care
> >>>>>>> that it
> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer questions
> >>>>>>> about it
> >>>>>>>>>> is storing.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
> >>>>>>>>>> so-called
> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows
> >>>> up
> >>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The admin creates a Disk Offering that references the storage
> >>>> tag
> >>>>>>> of
> >>>>>>>>>> the
> >>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a new
> >>>> row
> >>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
> >>>> disk).
> >>>>>>>>>> This
> >>>>>>>>>>>> leads to the storage framework calling the plug-in to create a
> >> new
> >>>>>>>>>> volume
> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
> >>>> updates
> >>>>>>> the
> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
> >>>>>>>>>> volume).
> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is in
> >>>> the
> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
> >>>>>>> 'Ready'
> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code will not
> >> be
> >>>>>>>>>> called
> >>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
> >>>>>>> volume
> >>>>>>>>>> to
> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
> >>>> structure
> >>>>>>>>>> (ex.
> >>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
> >>>>>>>>>>>>> definitions, refers to the physical allocation on the device,
> >>>>>>>>>> correct?"
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
> >>>>>>> try to
> >>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm having a
> >>>> hard
> >>>>>>>>>> time
> >>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
> >> was
> >>>>>>> the
> >>>>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >>>> jburwell@basho.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
> >> storage
> >>>>>>>>>> device
> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we are
> >>>>>>>>>> communicating with
> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
> >>>>>>> (Dynamic)
> >>>>>>>>>> on the
> >>>>>>>>>>>>>>> device while the other requires the volume to be volume to be
> >>>>>>>>>> managed
> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
> >>>>>>> sync on
> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
> >>>> physical
> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
> >> able
> >>>> to
> >>>>>>>>>>>>>>> communicate with a storage device to move bits from one place
> >>>> to
> >>>>>>>>>> another,
> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device
> >> may
> >>>> be
> >>>>>>>>>> able to
> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
> >>>> iSCSI
> >>>>>>>>>> device
> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
> >>>> SolidFire
> >>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
> >>>>>>> storing
> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
> >>>>>>> device,
> >>>>>>>>>> an
> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
> >>>> the
> >>>>>>>>>> device is
> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
> >>>> dedicate
> >>>>>>>>>> only a
> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
> >> appropriate
> >>>>>>> to
> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
> >>>>>>> notion
> >>>>>>>>>> of a
> >>>>>>>>>>>>>>> storage device with an attached driver that indicates whether
> >>>> or
> >>>>>>>>>> not is
> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
> >>>> representing a
> >>>>>>>>>> physical
> >>>>>>>>>>>>>>> allocation on a device separate that is associated with a
> >>>> volume.
> >>>>>>>>>> With
> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
> >>>>>>>>>> protocols they
> >>>>>>>>>>>>>>> support and when they encounter a device managed by
> >> CloudStack,
> >>>>>>>>>> utilize the
> >>>>>>>>>>>>>>> management operations exposed by the driver to automate
> >>>>>>> allocation.
> >>>>>>>>>> If
> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
> >>>>>>> drill
> >>>>>>>>>> down to
> >>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
> >> the
> >>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
> >>>> Storage
> >>>>>>>>>> based
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
> >>>>>>> exist
> >>>>>>>>>> on
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
> >>>> volumes
> >>>>>>> are
> >>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
> >>>> Storage
> >>>>>>>>>> based
> >>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
> >>>>>>> system
> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework). When
> >> a
> >>>>>>> user
> >>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
> >>>> tells
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
> >> attach
> >>>>>>>>>> logic
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
> >>>>>>> structure
> >>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>> at this point because the volume was just created. The
> >>>>>>> hypervisor
> >>>>>>>>>> data
> >>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have defined them
> >>>> up
> >>>>>>>>>> front
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
> >>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
> >> created
> >>>>>>> to
> >>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
> >>>>>>>>>> structure
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
> >> this
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
> >>>>>>> assumes
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
> >>>>>>> attaches
> >>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead
> >>>> of
> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on
> >> a
> >>>>>>> SAN,
> >>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
> >>>>>>> when an
> >>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes have not
> >>>>>>> been
> >>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
> >> the
> >>>>>>>>>> attach
> >>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
> >> attaches
> >>>>>>> the
> >>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
> >>>> behavior
> >>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
> >>>>>>> Another
> >>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
> >>>>>>> storage
> >>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
> >>>>>>> XenServer
> >>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on
> >> an
> >>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
> >> hypervisor
> >>>>>>> and
> >>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
> >> CloudStack
> >>>>>>> does
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
> >> For
> >>>>>>> all
> >>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
> >>>> perform
> >>>>>>>>>> this
> >>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference between
> >>>> the
> >>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
> >> overlay
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
> >>>> up
> >>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
> >>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants
> >> to
> >>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
> >>>>>>> would
> >>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
> >>>>>>> implement a
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
> >>>>>>> specific.
> >>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
> >>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
> >>>>>>>>>> storage).
> >>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
> >>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
> >>>>>>> creation
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
> >>>> him
> >>>>>>> in
> >>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
> >>>>>>> realized
> >>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless
> >>>> the
> >>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
> >>>> create
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
> >>>>>>> Storage
> >>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see
> >> the
> >>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> >> composed/driven
> >>>>>>> by
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
> >>>> per
> >>>>>>>>>> their
> >>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
> >>>> with a
> >>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
> >>>>>>> difficult to
> >>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
> >>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
> >> then
> >>>> go
> >>>>>>>>>> into
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
> >> volume
> >>>>>>> (ex.
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
> >>>> VMs
> >>>>>>> and
> >>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need
> >> no
> >>>>>>>>>> longer
> >>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
> >> between
> >>>> a
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
> >> storage
> >>>>>>>>>> Quality
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
> >>>>>>> dynamically
> >>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
> >>>> address
> >>>>>>> is
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
> >> when
> >>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do
> >> so.
> >>>>>>> I've
> >>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
> >>>> sandbox
> >>>>>>>>>> and
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
> >>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
> >>>>>>> agent
> >>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
> >> create a
> >>>>>>>>>> volume
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
> >>>>>>> hypervisor,
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
> >>>>>>> invoked on
> >>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
> >> create,
> >>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
> >>>>>>> VMware) to
> >>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making
> >>>> the
> >>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
> >>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
> >>>>>>> hypervisors
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2
> >> to
> >>>>>>> get
> >>>>>>>>>> KVM
> >>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
> >>>>>>> Depending on
> >>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working
> >>>> for
> >>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards
> >> to
> >>>>>>> Disk
> >>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
> >>>>>>> Damage
> >>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
> >> patch
> >>>>>>> cycle
> >>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
> >> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
> >>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
> >> non
> >>>>>>> issue
> >>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
> >> now,
> >>>>>>>>>> users
> >>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
> >>>>>>> calculation.
> >>>>>>>>>> Of
> >>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
> >>>>>>> individual
> >>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work
> >> on
> >>>> it
> >>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
> >> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying
> >> to
> >>>>>>>>>> adding
> >>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
> >>>>>>> statistics
> >>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
> >> you
> >>>>>>> have
> >>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
> >>>>>>>>>> read/write
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
> >>>> more
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
> >>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
> >> imagine
> >>>>>>> our
> >>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could
> >> an
> >>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
> >>>>>>> supports
> >>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
> >>>> through
> >>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
> >>>>>>> parameters
> >>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
> >>>> apologize
> >>>>>>>>>> for
> >>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
> >> with a
> >>>>>>> set
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
> >>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
> >> better
> >>>> to
> >>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider
> >>>> the
> >>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
> >>>> response
> >>>>>>> to
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
> >>>> think
> >>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but
> >> I
> >>>>>>> think
> >>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
> >>>> have
> >>>>>>>>>> have
> >>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
> >>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
> >>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
> >> policies
> >>>> be
> >>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
> >> purposes,
> >>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model
> >>>> for
> >>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> >>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled?
> >> In
> >>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
> >> IOPS
> >>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
> >>>> deliver.
> >>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
> >>>>>>> device
> >>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
> >>>> controlling
> >>>>>>> the
> >>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
> >>>> from
> >>>>>>> the
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
> >> Burwell
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
> >>>>>>> merged
> >>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready
> >> to
> >>>>>>> turn
> >>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue
> >>>> is
> >>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
> >>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >>>> intersect/compliment
> >>>>>>> the
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
> >> As I
> >>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
> >>>> like
> >>>>>>> to
> >>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
> >>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
> >> wido@widodh.nl
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
> >>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
> >> reason
> >>>>>>> till
> >>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
> >>>> mention
> >>>>>>>>>> again
> >>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
> >>>>>>> never
> >>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
> >>>> throttling
> >>>>>>>>>> and
> >>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from
> >>>> day
> >>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
> >>>>>>> support
> >>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
> >>>> because
> >>>>>>> I
> >>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
> >>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
> >>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
> >>>> now
> >>>>>>> so
> >>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
> >> before
> >>>>>>> the
> >>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
> >> instead
> >>>>>>> of
> >>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
> >> freeze
> >>>>>>> date.
> >>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
> >>>> necessary
> >>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> >>>>>>>>>> association
> >>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
> >>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
> >> Finally I
> >>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
> >> because
> >>>>>>> the
> >>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
> >>>>>>> bytes as
> >>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
> >>>> disk
> >>>>>>> I/O
> >>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
> >>>>>>>>>> situations
> >>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
> >>>> work.
> >>>>>>> It
> >>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
> >>>> freeze,
> >>>>>>> I
> >>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But
> >>>> if
> >>>>>>> it
> >>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
> >> release
> >>>>>>> with
> >>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
> >>>>>>> branch
> >>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master
> >>>> in
> >>>>>>> 48
> >>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
> >>>>>>> storage
> >>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
> >>>> limitation
> >>>>>>> of
> >>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
> >>>> affect
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
> >>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
> >> and
> >>>>>>> limit
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
> >>>>>>> difference
> >>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
> >>>>>>> different
> >>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
> >>>>>>>>>> collect
> >>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only
> >> go
> >>>>>>> for
> >>>>>>>>>> a
> >>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
> >>>>>>> then
> >>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
> >>>>>>> would
> >>>>>>>>>> set
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
> >> point I
> >>>>>>>>>> think
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
> >>>> place
> >>>>>>> to
> >>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
> >>>>>>> different
> >>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
> >>>> that
> >>>>>>> as
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
> >> this
> >>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
> >>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
> >> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> >> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
> >>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
> >>>>>>> integration)
> >>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
> >>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
> >>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
> >>>> expunge.
> >>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloudâ„¢
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
Alternatively to the way the attach logic is implemented in my patch, we
could do the following:
Leave the attach logic in the agent code alone. In VolumeManagerImpl we
create an AttachVolumeCommand and send it to the hypervisor. Before this
command is sent, we could check to see if we're dealing with Dynamic (or
whatever we want to call it) storage and - if so - send a "Create SR"
command to the hypervisor. If this returns OK, we would then proceed to the
AttachVolumeCommand, as usual.
This way the attach logic remains the same and we just add another command
to the agent code that is called for this particular type of storage.
What do you think?
On Tue, Jun 4, 2013 at 5:42 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> Hey John,
>
> I created a document for a customer today that outlines how the plug-in
> works from a user standpoint. This will probably be of use to you, as well,
> as you perform the code review.
>
> I have shared this document with you (you should have received that
> information in a separate e-mail).
>
> Talk to you later!
>
>
> On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Oh, OK, that sounds really good, John.
>>
>> Thanks and talk to you tomorrow! :)
>>
>>
>> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> I am never at a loss for an opinion. I some thoughts, but want to
>>> confirm assumptions and ideas against the solidfire, disk_io_throttle,
>>> and
>>> object_store branches. I hope to collect them in a coherent form
>>> tomorrow (5 June 2013).
>>>
>>> Thanks,
>>> -John
>>>
>>>
>>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>> > "So, in essence, the SolidFire plugin introduces the notion of a
>>> managed
>>> > iSCSI device and provisioned IOPS."
>>> >
>>> > Technically, the SolidFire plug-in just introduces the notion of
>>> > provisioned storage IOPS.
>>> >
>>> > The storage framework that leverages the plug-in was incomplete, so I
>>> had
>>> > to try to add in the notion of a managed iSCSI device.
>>> >
>>> > I appreciate all the time you've been spending on this. :) Do you have
>>> a
>>> > recommendation as to how we should accomplish what you're looking for?
>>> >
>>> > Thanks!
>>> >
>>> >
>>> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>> >
>>> >> Mike,
>>> >>
>>> >> So, in essence, the SolidFire plugin introduces the notion of a
>>> managed
>>> >> iSCSI device and provisioned IOPS.
>>> >>
>>> >> I want to see a separation of the management capabilities (i.e. can a
>>> >> device be managed/does an operator want it managed by CloudStack)
>>> from the
>>> >> storage protocol. Ideally, we should end up with a semantic that will
>>> >> allow any type of storage device to be managed. I also want to make
>>> >> progress on decoupling the storage types from the hypervisor
>>> definitions.
>>> >>
>>> >> Thanks,
>>> >> -John
>>> >>
>>> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>> >> wrote:
>>> >>
>>> >>> Hi John,
>>> >>>
>>> >>> No problem. Answers are below in red.
>>> >>>
>>> >>> Thanks!
>>> >>>
>>> >>>
>>> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>> >>>
>>> >>>> Mike,
>>> >>>>
>>> >>>> Could you please answer the following questions for me with regards
>>> to
>>> >> the
>>> >>>> operation of the SolidFire plugin:
>>> >>>>
>>> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
>>> >>> 1 SAN volume : 1 LUN
>>> >>>
>>> >>>> What is the cardinality between SAN Volumes and CloudStack Volumes?
>>> >>> 1 SAN volume : 1 CloudStack volume
>>> >>>
>>> >>>> Are the LUN(s) created by the management server or externally by the
>>> >>>> operator?
>>> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN
>>> LUN) is
>>> >>> created by the management server (via the plug-in) the first time the
>>> >>> CloudStack volume is attached to a hypervisor.
>>> >>>
>>> >>> If you don't want to use the SolidFire plug-in, but still want to
>>> use a
>>> >>> SolidFire volume (LUN), you can do this already today (prior to
>>> 4.2). The
>>> >>> admin manually creates the SAN volume and - in this case - multiple
>>> VMs
>>> >> and
>>> >>> data disks can share this SAN volume. While you can do this today,
>>> it is
>>> >>> not useful if you want to enforce storage QoS.
>>> >>>
>>> >>>> Are the SAN volumes by the management server or externally by the
>>> >> operator?
>>> >>> When the SolidFire plug-in is used, the SAN volumes are completely
>>> >> managed
>>> >>> by the management server (via the plug-in). There is no admin
>>> >> interaction.
>>> >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
>>> >> volume,
>>> >>> which is necessary for any storage vendor that supports true QoS.
>>> >>>
>>> >>>>
>>> >>>> I would like to clarify how these pieces are related and expected to
>>> >>>> operate.
>>> >>>>
>>> >>>> Thanks,
>>> >>>> -John
>>> >>>>
>>> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>>> >> mike.tutkowski@solidfire.com>
>>> >>>> wrote:
>>> >>>>
>>> >>>>> "In particular, how do we ensure that multiple VMs with provisioned
>>> >> IOPS
>>> >>>>> won't be cut off by the underlying storage."
>>> >>>>>
>>> >>>>> In the storage QoS world, we need to map a single SAN volume (LUN)
>>> to a
>>> >>>>> single CloudStack volume. We cannot have multiple CloudStack
>>> volumes
>>> >>>>> sharing a single SAN volume and still guarantee QoS.
>>> >>>>>
>>> >>>>> If the user wants to have a single SAN volume house more than one
>>> >>>>> CloudStack volume, then can do that today without any of my plug-in
>>> >> code.
>>> >>>>>
>>> >>>>>
>>> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>> >>>> mike.tutkowski@solidfire.com
>>> >>>>>> wrote:
>>> >>>>>
>>> >>>>>> "The administrator will allocate a SAN volume for CloudStack's use
>>> >> onto
>>> >>>>>> which CloudStack volumes will be created."
>>> >>>>>>
>>> >>>>>> I think we crossed e-mails. :)
>>> >>>>>>
>>> >>>>>> Check out my recent e-mail on this.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
>>> >>>> wrote:
>>> >>>>>>
>>> >>>>>>> Mike,
>>> >>>>>>>
>>> >>>>>>> You are coming to the part which concerns me -- concepts from the
>>> >>>>>>> hypervisor are leaking into storage layer.
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>> -John
>>> >>>>>>>
>>> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>> >>>> mike.tutkowski@solidfire.com>
>>> >>>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>>> The weird part is that the iSCSI type is today only used (as
>>> far as
>>> >> I
>>> >>>>>>> know)
>>> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>>> >>>>>>>>
>>> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the
>>> vmfs
>>> >>>> type.
>>> >>>>>>>>
>>> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>>> >>>>>>> SharedMountPoint
>>> >>>>>>>> type.
>>> >>>>>>>>
>>> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a new
>>> type
>>> >>>> of
>>> >>>>>>>> storage to describe this dynamic ability Edison added into the
>>> >> storage
>>> >>>>>>>> framework. Maybe it should be more specificy, though:
>>> Dynamic_iSCSI
>>> >>>>>>> versus,
>>> >>>>>>>> say, Dynamic_FC.
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>> >>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> "The storage device itself shouldn't know or care that it is
>>> being
>>> >>>> used
>>> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it is
>>> >>>>>>> storing."
>>> >>>>>>>>>
>>> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in
>>> needs to
>>> >>>>>>> call
>>> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
>>> >> treat
>>> >>>>>>> it as
>>> >>>>>>>>> such.
>>> >>>>>>>>>
>>> >>>>>>>>> I'm totally open to removing that constraint and just calling
>>> it
>>> >>>> iSCSI
>>> >>>>>>> or
>>> >>>>>>>>> whatever. We would just need a way for the hypervisor attach
>>> logic
>>> >> to
>>> >>>>>>>>> detect this new requirement and perform the necessary
>>> activities.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>>> jburwell@basho.com>
>>> >>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>> Mike,
>>> >>>>>>>>>>
>>> >>>>>>>>>> See my responses in-line.
>>> >>>>>>>>>>
>>> >>>>>>>>>> Thanks,
>>> >>>>>>>>>> -John
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>> >>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> I'm trying to picture this:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
>>> >>>>>>> operator
>>> >>>>>>>>>> may
>>> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
>>> >>>> shared
>>> >>>>>>> by
>>> >>>>>>>>>>> many services, and the operator has chosen to dedicate only a
>>> >>>> portion
>>> >>>>>>>>>> of it
>>> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the
>>> plug-in
>>> >>>>>>> (before
>>> >>>>>>>>>> an
>>> >>>>>>>>>>> attach request to the hypervisor) would work unless the
>>> >> hypervisor
>>> >>>>>>>>>> consumes
>>> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
>>> >>>>>>>>>>
>>> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator
>>> >>>> allocates
>>> >>>>>>> a
>>> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN.
>>> They
>>> >>>>>>> simply
>>> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but not
>>> allow
>>> >>>>>>>>>> CloudStack to allocate anymore LUNs.
>>> >>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be
>>> >> passing
>>> >>>>>>> in a
>>> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor
>>> support
>>> >>>> (like
>>> >>>>>>> an
>>> >>>>>>>>>> SR)
>>> >>>>>>>>>>> and the logic will fail.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Are you thinking we should maybe have the storage framework
>>> >> itself
>>> >>>>>>>>>> detect
>>> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor
>>> side and
>>> >>>>>>> have
>>> >>>>>>>>>> it
>>> >>>>>>>>>>> call into the agent code specifically to create the SR
>>> before the
>>> >>>>>>> attach
>>> >>>>>>>>>>> logic runs in the agent code?
>>> >>>>>>>>>>
>>> >>>>>>>>>> I think the hypervisor management plugin should have a rich
>>> enough
>>> >>>>>>>>>> interface to storage to determine available for volume
>>> storage.
>>> >> For
>>> >>>>>>> Xen,
>>> >>>>>>>>>> this interface would allow the interrogation of the device to
>>> >>>>>>> determine the
>>> >>>>>>>>>> SR is present. The storage device itself shouldn't know or
>>> care
>>> >>>>>>> that it
>>> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>>> questions
>>> >>>>>>> about it
>>> >>>>>>>>>> is storing.
>>> >>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>> >>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> So, the flow is as follows:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a type
>>> of
>>> >>>>>>>>>> so-called
>>> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage
>>> shows
>>> >>>> up
>>> >>>>>>> in
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>> applicable zone.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> * The admin creates a Disk Offering that references the
>>> storage
>>> >>>> tag
>>> >>>>>>> of
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>> newly created Primary Storage.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a
>>> new
>>> >>>> row
>>> >>>>>>> in
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>> cloud.volumes table.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM
>>> (attach
>>> >>>> disk).
>>> >>>>>>>>>> This
>>> >>>>>>>>>>>> leads to the storage framework calling the plug-in to
>>> create a
>>> >> new
>>> >>>>>>>>>> volume
>>> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
>>> >>>> updates
>>> >>>>>>> the
>>> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the
>>> SAN
>>> >>>>>>>>>> volume).
>>> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume
>>> is in
>>> >>>> the
>>> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in
>>> the
>>> >>>>>>> 'Ready'
>>> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code will
>>> not
>>> >> be
>>> >>>>>>>>>> called
>>> >>>>>>>>>>>> again to create this SAN volume.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>>> CloudStack
>>> >>>>>>> volume
>>> >>>>>>>>>> to
>>> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>>> >>>> structure
>>> >>>>>>>>>> (ex.
>>> >>>>>>>>>>>> an SR on XenServer).
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in
>>> these
>>> >>>>>>>>>>>>> definitions, refers to the physical allocation on the
>>> device,
>>> >>>>>>>>>> correct?"
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>>> CloudStack, I
>>> >>>>>>> try to
>>> >>>>>>>>>>>>> use 'CloudStack volume'.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Hi John,
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm
>>> having a
>>> >>>> hard
>>> >>>>>>>>>> time
>>> >>>>>>>>>>>>>> envisioning it.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as
>>> he
>>> >> was
>>> >>>>>>> the
>>> >>>>>>>>>>>>>> initial person to suggest the approach I took.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> What do you think?
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>>> >>>> jburwell@basho.com
>>> >>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
>>> >> storage
>>> >>>>>>>>>> device
>>> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we are
>>> >>>>>>>>>> communicating with
>>> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
>>> >>>>>>> (Dynamic)
>>> >>>>>>>>>> on the
>>> >>>>>>>>>>>>>>> device while the other requires the volume to be volume
>>> to be
>>> >>>>>>>>>> managed
>>> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we
>>> are in
>>> >>>>>>> sync on
>>> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
>>> >>>> physical
>>> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
>>> >> able
>>> >>>> to
>>> >>>>>>>>>>>>>>> communicate with a storage device to move bits from one
>>> place
>>> >>>> to
>>> >>>>>>>>>> another,
>>> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage
>>> device
>>> >> may
>>> >>>> be
>>> >>>>>>>>>> able to
>>> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a
>>> unmanaged
>>> >>>> iSCSI
>>> >>>>>>>>>> device
>>> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>>> >>>> SolidFire
>>> >>>>>>>>>> iSCSI
>>> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs
>>> and
>>> >>>>>>> storing
>>> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to
>>> manage a
>>> >>>>>>> device,
>>> >>>>>>>>>> an
>>> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack
>>> (e.g.
>>> >>>> the
>>> >>>>>>>>>> device is
>>> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
>>> >>>> dedicate
>>> >>>>>>>>>> only a
>>> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make
>>> sense?
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>>> >> appropriate
>>> >>>>>>> to
>>> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add
>>> the
>>> >>>>>>> notion
>>> >>>>>>>>>> of a
>>> >>>>>>>>>>>>>>> storage device with an attached driver that indicates
>>> whether
>>> >>>> or
>>> >>>>>>>>>> not is
>>> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>>> >>>> representing a
>>> >>>>>>>>>> physical
>>> >>>>>>>>>>>>>>> allocation on a device separate that is associated with a
>>> >>>> volume.
>>> >>>>>>>>>> With
>>> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare
>>> which
>>> >>>>>>>>>> protocols they
>>> >>>>>>>>>>>>>>> support and when they encounter a device managed by
>>> >> CloudStack,
>>> >>>>>>>>>> utilize the
>>> >>>>>>>>>>>>>>> management operations exposed by the driver to automate
>>> >>>>>>> allocation.
>>> >>>>>>>>>> If
>>> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down
>>> and
>>> >>>>>>> drill
>>> >>>>>>>>>> down to
>>> >>>>>>>>>>>>>>> a more detailed design.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type
>>> and
>>> >> the
>>> >>>>>>>>>> Dynamic
>>> >>>>>>>>>>>>>>> type:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
>>> >>>> Storage
>>> >>>>>>>>>> based
>>> >>>>>>>>>>>>>>> on
>>> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume
>>> must
>>> >>>>>>> exist
>>> >>>>>>>>>> on
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
>>> >>>> volumes
>>> >>>>>>> are
>>> >>>>>>>>>>>>>>> created
>>> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
>>> >>>> Storage
>>> >>>>>>>>>> based
>>> >>>>>>>>>>>>>>> on a
>>> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
>>> storage
>>> >>>>>>> system
>>> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework).
>>> When
>>> >> a
>>> >>>>>>> user
>>> >>>>>>>>>>>>>>> wants to
>>> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>>> framework
>>> >>>> tells
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
>>> >> attach
>>> >>>>>>>>>> logic
>>> >>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
>>> >>>>>>> structure
>>> >>>>>>>>>>>>>>> exists
>>> >>>>>>>>>>>>>>>> at this point because the volume was just created. The
>>> >>>>>>> hypervisor
>>> >>>>>>>>>> data
>>> >>>>>>>>>>>>>>>> structure must be created.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have defined
>>> them
>>> >>>> up
>>> >>>>>>>>>> front
>>> >>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>> you. :)
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an
>>> admin
>>> >>>>>>>>>> creates a
>>> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
>>> >> created
>>> >>>>>>> to
>>> >>>>>>>>>>>>>>> consume
>>> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor
>>> data
>>> >>>>>>>>>> structure
>>> >>>>>>>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
>>> >> this
>>> >>>>>>>>>>>>>>> hypervisor
>>> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach
>>> logic
>>> >>>>>>> assumes
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and
>>> simply
>>> >>>>>>> attaches
>>> >>>>>>>>>>>>>>> the VDI
>>> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
>>> Instead
>>> >>>> of
>>> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a
>>> volume on
>>> >> a
>>> >>>>>>> SAN,
>>> >>>>>>>>>>>>>>> but the
>>> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
>>> created
>>> >>>>>>> when an
>>> >>>>>>>>>>>>>>> attach
>>> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes
>>> have not
>>> >>>>>>> been
>>> >>>>>>>>>>>>>>> pre-hooked
>>> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin.
>>> Once
>>> >> the
>>> >>>>>>>>>> attach
>>> >>>>>>>>>>>>>>> logic
>>> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
>>> >> attaches
>>> >>>>>>> the
>>> >>>>>>>>>>>>>>> one and
>>> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>> >>>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
>>> >>>> behavior
>>> >>>>>>>>>>>>>>> works in
>>> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>>> difference.
>>> >>>>>>> Another
>>> >>>>>>>>>>>>>>> way to
>>> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a
>>> Dynamic
>>> >>>>>>> storage
>>> >>>>>>>>>>>>>>> pool type?
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>>> >>>>>>> XenServer
>>> >>>>>>>>>>>>>>> when you
>>> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly
>>> based on
>>> >> an
>>> >>>>>>>>>> iSCSI
>>> >>>>>>>>>>>>>>>>>> target.
>>> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>>> >> hypervisor
>>> >>>>>>> and
>>> >>>>>>>>>>>>>>>>>> creating a
>>> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>>> >> CloudStack
>>> >>>>>>> does
>>> >>>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>> part
>>> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for
>>> XenServer.
>>> >> For
>>> >>>>>>> all
>>> >>>>>>>>>>>>>>> other
>>> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
>>> >>>> perform
>>> >>>>>>>>>> this
>>> >>>>>>>>>>>>>>> step
>>> >>>>>>>>>>>>>>>>>>> manually.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>> >>>>>>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
>>> between
>>> >>>> the
>>> >>>>>>>>>>>>>>> ISCSI and
>>> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>> >>>>>>>>>>>>>>> Storage.StoragePoolType
>>> >>>>>>>>>>>>>>>>>> class
>>> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>>> >> overlay
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> RBD(true),
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is
>>> set
>>> >>>> up
>>> >>>>>>>>>> by
>>> >>>>>>>>>>>>>>>>>>>>> customers.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>> >>>>>>>>>> SolidFire)
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> boolean shared;
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> }
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> return shared;
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> }
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> }
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company
>>> wants
>>> >> to
>>> >>>>>>>>>>>>>>> implement a
>>> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>>> feature. It
>>> >>>>>>> would
>>> >>>>>>>>>> be
>>> >>>>>>>>>>>>>>>>>> dynamic,
>>> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>>> >>>>>>> implement a
>>> >>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>> >>>>>>>>>>>>>>> hypervisor-attach
>>> >>>>>>>>>>>>>>>>>>>> logic
>>> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is
>>> SolidFire
>>> >>>>>>> specific.
>>> >>>>>>>>>>>>>>> It is
>>> >>>>>>>>>>>>>>>>>> not.
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
>>> dynamic,
>>> >>>>>>>>>> zone-wide
>>> >>>>>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind
>>> of
>>> >>>>>>>>>> storage).
>>> >>>>>>>>>>>>>>> This
>>> >>>>>>>>>>>>>>>>>>>> kind of
>>> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
>>> framework
>>> >>>>>>>>>> changes.
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
>>> >>>>>>> creation
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>> deletion
>>> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited
>>> with
>>> >>>> him
>>> >>>>>>> in
>>> >>>>>>>>>>>>>>> Portland
>>> >>>>>>>>>>>>>>>>>>>> back
>>> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete.
>>> We
>>> >>>>>>> realized
>>> >>>>>>>>>>>>>>> there
>>> >>>>>>>>>>>>>>>>>> was
>>> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes
>>> unless
>>> >>>> the
>>> >>>>>>>>>>>>>>> attach
>>> >>>>>>>>>>>>>>>>>> logic
>>> >>>>>>>>>>>>>>>>>>>> was
>>> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
>>> >>>> create
>>> >>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>> appropriate
>>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>> >>>>>>>>>>>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in
>>> the
>>> >>>>>>> Storage
>>> >>>>>>>>>>>>>>> layer
>>> >>>>>>>>>>>>>>>>>>>> would
>>> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to
>>> see
>>> >> the
>>> >>>>>>>>>> Storage
>>> >>>>>>>>>>>>>>>>>>>> services
>>> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>>> >> composed/driven
>>> >>>>>>> by
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>> Hypervisor
>>> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>>> structures
>>> >>>> per
>>> >>>>>>>>>> their
>>> >>>>>>>>>>>>>>>>>> needs.
>>> >>>>>>>>>>>>>>>>>>>> If
>>> >>>>>>>>>>>>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to
>>> end
>>> >>>> with a
>>> >>>>>>>>>>>>>>> massive
>>> >>>>>>>>>>>>>>>>>>>> n-to-n
>>> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>>> >>>>>>> difficult to
>>> >>>>>>>>>>>>>>>>>> maintain
>>> >>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
>>> specific
>>> >>>>>>>>>> SolidFire
>>> >>>>>>>>>>>>>>> code
>>> >>>>>>>>>>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method
>>> for
>>> >>>>>>>>>> XenServer,
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> admin
>>> >>>>>>>>>>>>>>>>>>>>>>>> had
>>> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
>>> >> then
>>> >>>> go
>>> >>>>>>>>>> into
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
>>> >> volume
>>> >>>>>>> (ex.
>>> >>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
>>> ESX(i)).
>>> >>>> VMs
>>> >>>>>>> and
>>> >>>>>>>>>>>>>>> data
>>> >>>>>>>>>>>>>>>>>> disks
>>> >>>>>>>>>>>>>>>>>>>>>>>> then
>>> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage
>>> need
>>> >> no
>>> >>>>>>>>>> longer
>>> >>>>>>>>>>>>>>> be so
>>> >>>>>>>>>>>>>>>>>>>>>>>> static
>>> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
>>> >> between
>>> >>>> a
>>> >>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>> system's
>>> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
>>> >> storage
>>> >>>>>>>>>> Quality
>>> >>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>> Service).
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>>> >>>>>>> dynamically
>>> >>>>>>>>>>>>>>> create
>>> >>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>> delete
>>> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
>>> >>>> address
>>> >>>>>>> is
>>> >>>>>>>>>> in
>>> >>>>>>>>>>>>>>>>>> creating
>>> >>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
>>> >> when
>>> >>>>>>>>>>>>>>> performing an
>>> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to
>>> do
>>> >> so.
>>> >>>>>>> I've
>>> >>>>>>>>>>>>>>> got
>>> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
>>> >>>> sandbox
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>>>>> submit
>>> >>>>>>>>>>>>>>>>>>>>>>>> this
>>> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski
>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is
>>> hypervisor
>>> >>>>>>> agnostic.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in
>>> the
>>> >>>>>>> agent
>>> >>>>>>>>>>>>>>> code).
>>> >>>>>>>>>>>>>>>>>> The
>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
>>> >> create a
>>> >>>>>>>>>> volume
>>> >>>>>>>>>>>>>>> as
>>> >>>>>>>>>>>>>>>>>>>>>>>> needed,
>>> >>>>>>>>>>>>>>>>>>>>>>>>> but
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>>> >>>>>>> hypervisor,
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> attach
>>> >>>>>>>>>>>>>>>>>>>>>>>>> logic
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>>> >>>>>>> invoked on
>>> >>>>>>>>>>>>>>>>>> zone-wide
>>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
>>> >> create,
>>> >>>>>>>>>> say, a
>>> >>>>>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>>> >>>>>>> VMware) to
>>> >>>>>>>>>>>>>>> make
>>> >>>>>>>>>>>>>>>>>> use
>>> >>>>>>>>>>>>>>>>>>>>>>>> of the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently
>>> making
>>> >>>> the
>>> >>>>>>>>>>>>>>> attach
>>> >>>>>>>>>>>>>>>>>> logic
>>> >>>>>>>>>>>>>>>>>>>>>>>> work
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell
>>> <
>>> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>>> >>>>>>> hypervisor
>>> >>>>>>>>>>>>>>>>>> specific?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>>> >>>>>>> hypervisors
>>> >>>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for
>>> 4.2
>>> >> to
>>> >>>>>>> get
>>> >>>>>>>>>> KVM
>>> >>>>>>>>>>>>>>> in.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>>> >>>>>>> Depending on
>>> >>>>>>>>>>>>>>> what
>>> >>>>>>>>>>>>>>>>>> we do
>>> >>>>>>>>>>>>>>>>>>>>>>>>> with
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
>>> working
>>> >>>> for
>>> >>>>>>>>>>>>>>> VMware in
>>> >>>>>>>>>>>>>>>>>> my
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
>>> regards
>>> >> to
>>> >>>>>>> Disk
>>> >>>>>>>>>>>>>>>>>> Offerings.
>>> >>>>>>>>>>>>>>>>>>>>>>>> I
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
>>> Jamison
>>> >>>>>>> Damage
>>> >>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
>>> >> patch
>>> >>>>>>> cycle
>>> >>>>>>>>>>>>>>> post
>>> >>>>>>>>>>>>>>>>>> 4.2?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>> >>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
>>> >> MASTER
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first,
>>> then
>>> >>>> mine.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is
>>> a
>>> >> non
>>> >>>>>>> issue
>>> >>>>>>>>>>>>>>> as I
>>> >>>>>>>>>>>>>>>>>> don't
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on
>>> billing
>>> >> now,
>>> >>>>>>>>>> users
>>> >>>>>>>>>>>>>>> need
>>> >>>>>>>>>>>>>>>>>> to
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get
>>> the
>>> >>>>>>>>>>>>>>> service_offering
>>> >>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>>> >>>>>>> calculation.
>>> >>>>>>>>>> Of
>>> >>>>>>>>>>>>>>> course
>>> >>>>>>>>>>>>>>>>>>>>>>>> it is
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>>> >>>>>>> individual
>>> >>>>>>>>>>>>>>> volume,
>>> >>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can
>>> work
>>> >> on
>>> >>>> it
>>> >>>>>>>>>>>>>>> later. Do
>>> >>>>>>>>>>>>>>>>>> you
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>>> question.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>>> >> features.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jburwell@basho.com
>>> >
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
>>> trying
>>> >> to
>>> >>>>>>>>>> adding
>>> >>>>>>>>>>>>>>> more.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>>> >>>>>>> statistics
>>> >>>>>>>>>>>>>>> rather
>>> >>>>>>>>>>>>>>>>>> than
>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat
>>> if
>>> >> you
>>> >>>>>>> have
>>> >>>>>>>>>>>>>>> time.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>>> bytes/iops
>>> >>>>>>>>>> read/write
>>> >>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>> an
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are
>>> billed
>>> >>>> more
>>> >>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>> volumes
>>> >>>>>>>>>>>>>>>>>>>>>>>> with
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>>> >>>>>>> operations (
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
>>> >> imagine
>>> >>>>>>> our
>>> >>>>>>>>>>>>>>> users
>>> >>>>>>>>>>>>>>>>>> would
>>> >>>>>>>>>>>>>>>>>>>>>>>> like
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
>>> Could
>>> >> an
>>> >>>>>>>>>>>>>>> operator
>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>>> >>>>>>> supports
>>> >>>>>>>>>>>>>>> setting
>>> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
>>> >>>> through
>>> >>>>>>>>>>>>>>> command
>>> >>>>>>>>>>>>>>>>>> line.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>>> >>>>>>> parameters
>>> >>>>>>>>>>>>>>> of a
>>> >>>>>>>>>>>>>>>>>>>>>>>> created
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>>> >>>> apologize
>>> >>>>>>>>>> for
>>> >>>>>>>>>>>>>>> being
>>> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
>>> >> with a
>>> >>>>>>> set
>>> >>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided
>>> by a
>>> >>>>>>>>>> resource?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
>>> >> better
>>> >>>> to
>>> >>>>>>>>>>>>>>> commit
>>> >>>>>>>>>>>>>>>>>> Mike's
>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
>>> consider
>>> >>>> the
>>> >>>>>>>>>>>>>>>>>> limitation in
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>>> >>>> response
>>> >>>>>>> to
>>> >>>>>>>>>>>>>>> Mike
>>> >>>>>>>>>>>>>>>>>>>>>>>> regarding
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features.
>>> I
>>> >>>> think
>>> >>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire,
>>> but
>>> >> I
>>> >>>>>>> think
>>> >>>>>>>>>>>>>>> we need
>>> >>>>>>>>>>>>>>>>>>>>>>>> closer
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>>> (possibly
>>> >>>> have
>>> >>>>>>>>>> have
>>> >>>>>>>>>>>>>>>>>> solidfire
>>> >>>>>>>>>>>>>>>>>>>>>>>>> track
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
>>> >>>> issue.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>>> jburwell@basho.com>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>>> >>>>>>> following:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>>> >> policies
>>> >>>> be
>>> >>>>>>>>>>>>>>> captured
>>> >>>>>>>>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>>>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>>> >> purposes,
>>> >>>>>>>>>>>>>>> expressing
>>> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface
>>> model
>>> >>>> for
>>> >>>>>>>>>>>>>>> reasoning
>>> >>>>>>>>>>>>>>>>>>>>>>>> about
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>> >>>>>>>>>>>>>>> configurations
>>> >>>>>>>>>>>>>>>>>>>>>>>> between
>>> >>>>>>>>>>>>>>>>>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>>> reconciled?
>>> >> In
>>> >>>>>>>>>>>>>>> particular,
>>> >>>>>>>>>>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for
>>> more
>>> >> IOPS
>>> >>>>>>>>>>>>>>> configured
>>> >>>>>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>>>>>>>> a VM
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>>> >>>> deliver.
>>> >>>>>>>>>>>>>>> Another
>>> >>>>>>>>>>>>>>>>>>>>>>>> scenario
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM
>>> and a
>>> >>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>> device at
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
>>> storage
>>> >>>>>>> device
>>> >>>>>>>>>>>>>>>>>> introduces
>>> >>>>>>>>>>>>>>>>>>>>>>>>> logical
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
>>> Tutkowski
>>> >> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>>> >>>> controlling
>>> >>>>>>> the
>>> >>>>>>>>>>>>>>> max
>>> >>>>>>>>>>>>>>>>>>>>>>>> number of
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling
>>> IOPS
>>> >>>> from
>>> >>>>>>> the
>>> >>>>>>>>>>>>>>> storage
>>> >>>>>>>>>>>>>>>>>>>>>>>> system
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
>>> >> Burwell
>>> >>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should
>>> be
>>> >>>>>>> merged
>>> >>>>>>>>>>>>>>> until all
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is
>>> ready
>>> >> to
>>> >>>>>>> turn
>>> >>>>>>>>>>>>>>> over to
>>> >>>>>>>>>>>>>>>>>>>>>>>> test.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops
>>> issue
>>> >>>> is
>>> >>>>>>>>>>>>>>> addressed
>>> >>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has
>>> been
>>> >>>>>>>>>>>>>>> satisfied.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>>> >>>> intersect/compliment
>>> >>>>>>> the
>>> >>>>>>>>>>>>>>>>>> SolidFire
>>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
>>> >> As I
>>> >>>>>>>>>>>>>>> understand
>>> >>>>>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I
>>> would
>>> >>>> like
>>> >>>>>>> to
>>> >>>>>>>>>>>>>>> ensure
>>> >>>>>>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>>>>>>>>> don't
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in
>>> KVM and
>>> >>>>>>>>>>>>>>> SolidFire are
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>>> >> wido@widodh.nl
>>> >>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU
>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features
>>> into
>>> >>>>>>> master.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
>>> >> reason
>>> >>>>>>> till
>>> >>>>>>>>>>>>>>> Monday,
>>> >>>>>>>>>>>>>>>>>> I
>>> >>>>>>>>>>>>>>>>>>>>>>>> will
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
>>> >>>> mention
>>> >>>>>>>>>> again
>>> >>>>>>>>>>>>>>>>>> that we
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so
>>> that we
>>> >>>>>>> never
>>> >>>>>>>>>>>>>>> release
>>> >>>>>>>>>>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>>>>>>>>> version
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>>> >>>> throttling
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>>>> that's
>>> >>>>>>>>>>>>>>>>>>>>>>>> great!
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy
>>> from
>>> >>>> day
>>> >>>>>>>>>> #1.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>> >>>> wido@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
>>> >> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>>> discretely
>>> >>>>>>> support
>>> >>>>>>>>>>>>>>> read
>>> >>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>> write
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
>>> >>>> because
>>> >>>>>>> I
>>> >>>>>>>>>>>>>>> don't see
>>> >>>>>>>>>>>>>>>>>>>>>>>> how we
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>>> >>>>>>>>>> Therefore, we
>>> >>>>>>>>>>>>>>>>>> would
>>> >>>>>>>>>>>>>>>>>>>>>>>> be
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>>> >>>>>>> backwards
>>> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into
>>> master
>>> >>>> now
>>> >>>>>>> so
>>> >>>>>>>>>>>>>>> that it
>>> >>>>>>>>>>>>>>>>>>>>>>>> will go
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
>>> >> before
>>> >>>>>>> the
>>> >>>>>>>>>> 4.2
>>> >>>>>>>>>>>>>>>>>> release
>>> >>>>>>>>>>>>>>>>>>>>>>>> so
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
>>> >> instead
>>> >>>>>>> of
>>> >>>>>>>>>>>>>>> Total
>>> >>>>>>>>>>>>>>>>>> IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
>>> >> freeze
>>> >>>>>>> date.
>>> >>>>>>>>>>>>>>> But if
>>> >>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>>> >>>> necessary
>>> >>>>>>>>>> to do
>>> >>>>>>>>>>>>>>>>>> that.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is
>>> no
>>> >>>>>>>>>> association
>>> >>>>>>>>>>>>>>>>>> between
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>>> >>>>>>> Hollander <
>>> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU
>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>>> >> Finally I
>>> >>>>>>>>>>>>>>> decided to
>>> >>>>>>>>>>>>>>>>>>>>>>>> ignore
>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
>>> >> because
>>> >>>>>>> the
>>> >>>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
>>> received
>>> >>>>>>> bytes as
>>> >>>>>>>>>>>>>>> well.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking
>>> and
>>> >>>> disk
>>> >>>>>>> I/O
>>> >>>>>>>>>>>>>>>>>> completely
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in
>>> most
>>> >>>>>>>>>> situations
>>> >>>>>>>>>>>>>>> then
>>> >>>>>>>>>>>>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some
>>> copy-paste
>>> >>>> work.
>>> >>>>>>> It
>>> >>>>>>>>>>>>>>> could be
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
>>> >>>> freeze,
>>> >>>>>>> I
>>> >>>>>>>>>> will
>>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>>> >>>>>>>>>>>>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't
>>> miss. But
>>> >>>> if
>>> >>>>>>> it
>>> >>>>>>>>>>>>>>> goes
>>> >>>>>>>>>>>>>>>>>> into
>>> >>>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
>>> >> release
>>> >>>>>>> with
>>> >>>>>>>>>>>>>>> only
>>> >>>>>>>>>>>>>>>>>> total
>>> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>> >>>> wido@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU
>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>>> disk_io_throttling
>>> >>>>>>> branch
>>> >>>>>>>>>> into
>>> >>>>>>>>>>>>>>>>>> master.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into
>>> master
>>> >>>> in
>>> >>>>>>> 48
>>> >>>>>>>>>>>>>>> hours.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the
>>> same
>>> >>>>>>> storage
>>> >>>>>>>>>>>>>>> device
>>> >>>>>>>>>>>>>>>>>>>>>>>> (local
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>>> >>>> limitation
>>> >>>>>>> of
>>> >>>>>>>>>>>>>>> device
>>> >>>>>>>>>>>>>>>>>>>>>>>> (such as
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it
>>> may
>>> >>>> affect
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>> disk
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
>>> >>>> device.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum
>>> rate
>>> >> and
>>> >>>>>>> limit
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> disk
>>> >>>>>>>>>>>>>>>>>>>>>>>> I/O
>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>>> >>>>>>> difference
>>> >>>>>>>>>>>>>>> between
>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>> >>>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both
>>> a
>>> >>>>>>> different
>>> >>>>>>>>>>>>>>> rate
>>> >>>>>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
>>> users.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling
>>> side you
>>> >>>>>>>>>> collect
>>> >>>>>>>>>>>>>>> both
>>> >>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you
>>> only
>>> >> go
>>> >>>>>>> for
>>> >>>>>>>>>> a
>>> >>>>>>>>>>>>>>> global
>>> >>>>>>>>>>>>>>>>>>>>>>>> value.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
>>> expensive
>>> >>>>>>> then
>>> >>>>>>>>>> Read
>>> >>>>>>>>>>>>>>>>>> IOps,
>>> >>>>>>>>>>>>>>>>>>>>>>>> so it
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an
>>> admin
>>> >>>>>>> would
>>> >>>>>>>>>> set
>>> >>>>>>>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>> lower
>>> >>>>>>>>>>>>>>>>>>>>>>>>> value
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
>>> >> point I
>>> >>>>>>>>>> think
>>> >>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>> would
>>> >>>>>>>>>>>>>>>>>>>>>>>> be
>>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism
>>> in
>>> >>>> place
>>> >>>>>>> to
>>> >>>>>>>>>>>>>>> support
>>> >>>>>>>>>>>>>>>>>>>>>>>> both,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support
>>> setting
>>> >>>>>>> different
>>> >>>>>>>>>>>>>>> values
>>> >>>>>>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>>>>>>>>> read
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and
>>> set
>>> >>>> that
>>> >>>>>>> as
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> total
>>> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
>>> >> this
>>> >>>>>>> way?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>> >>>>>>>>>> disk_offering,
>>> >>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>>>>> global
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
>>> >> iops)
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>>> >> https://issues.apache.org/****
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>> >>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>> >>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**
>>> >
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> >>>>>>>>>>>>>>>>>> https://cwiki.
>>> >>>>>>>>>>>>>>>>>>>>>>>> **
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>
>>> >>>>
>>> >>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
>>> execution
>>> >>>>>>>>>> success?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>>> introduced?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>>> >>>>>>> integration)
>>> >>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>>> included
>>> >>>>>>>>>>>>>>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check
>>> for
>>> >>>>>>>>>> potential
>>> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate
>>> on
>>> >>>>>>>>>> CloudStack
>>> >>>>>>>>>>>>>>> UI.
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
>>> >>>> expunge.
>>> >>>>>>>>>>>>>>> migrate,
>>> >>>>>>>>>>>>>>>>>>>>>>>> restore
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>> >>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>> >>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**
>>> >
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> >>>>>>>>>>>>>>>>>> https://cwiki.
>>> >>>>>>>>>>>>>>>>>>>>>>>> **
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>
>>> >>>>
>>> >>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>> >>>>>>>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>>> >> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>> >>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>>> >> <
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>> >> Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>> Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>>> Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> http://solidfire.com/solution/overview/?video=play
>>> >>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>>>> cloud<
>>> >> http://solidfire.com/solution/overview/?video=play
>>> >>>>>
>>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play
>>> >>>
>>> >>>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> --
>>> >>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> --
>>> >>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>> o: 303.746.7302
>>> >>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>> *â„¢*
>>> >>>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>> *Mike Tutkowski*
>>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>> o: 303.746.7302
>>> >>>>>>>> Advancing the way the world uses the
>>> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>> *â„¢*
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> *Mike Tutkowski*
>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>> o: 303.746.7302
>>> >>>>>> Advancing the way the world uses the cloud<
>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>> *â„¢*
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> *Mike Tutkowski*
>>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>> e: mike.tutkowski@solidfire.com
>>> >>>>> o: 303.746.7302
>>> >>>>> Advancing the way the world uses the
>>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>> *â„¢*
>>> >>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Mike Tutkowski*
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>> e: mike.tutkowski@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the
>>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>> *â„¢*
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hey John,
I created a document for a customer today that outlines how the plug-in
works from a user standpoint. This will probably be of use to you, as well,
as you perform the code review.
I have shared this document with you (you should have received that
information in a separate e-mail).
Talk to you later!
On Tue, Jun 4, 2013 at 3:48 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> Oh, OK, that sounds really good, John.
>
> Thanks and talk to you tomorrow! :)
>
>
> On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> I am never at a loss for an opinion. I some thoughts, but want to
>> confirm assumptions and ideas against the solidfire, disk_io_throttle,
>> and
>> object_store branches. I hope to collect them in a coherent form
>> tomorrow (5 June 2013).
>>
>> Thanks,
>> -John
>>
>>
>> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > "So, in essence, the SolidFire plugin introduces the notion of a managed
>> > iSCSI device and provisioned IOPS."
>> >
>> > Technically, the SolidFire plug-in just introduces the notion of
>> > provisioned storage IOPS.
>> >
>> > The storage framework that leverages the plug-in was incomplete, so I
>> had
>> > to try to add in the notion of a managed iSCSI device.
>> >
>> > I appreciate all the time you've been spending on this. :) Do you have a
>> > recommendation as to how we should accomplish what you're looking for?
>> >
>> > Thanks!
>> >
>> >
>> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Mike,
>> >>
>> >> So, in essence, the SolidFire plugin introduces the notion of a managed
>> >> iSCSI device and provisioned IOPS.
>> >>
>> >> I want to see a separation of the management capabilities (i.e. can a
>> >> device be managed/does an operator want it managed by CloudStack) from
>> the
>> >> storage protocol. Ideally, we should end up with a semantic that will
>> >> allow any type of storage device to be managed. I also want to make
>> >> progress on decoupling the storage types from the hypervisor
>> definitions.
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >> wrote:
>> >>
>> >>> Hi John,
>> >>>
>> >>> No problem. Answers are below in red.
>> >>>
>> >>> Thanks!
>> >>>
>> >>>
>> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> Could you please answer the following questions for me with regards
>> to
>> >> the
>> >>>> operation of the SolidFire plugin:
>> >>>>
>> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
>> >>> 1 SAN volume : 1 LUN
>> >>>
>> >>>> What is the cardinality between SAN Volumes and CloudStack Volumes?
>> >>> 1 SAN volume : 1 CloudStack volume
>> >>>
>> >>>> Are the LUN(s) created by the management server or externally by the
>> >>>> operator?
>> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN
>> LUN) is
>> >>> created by the management server (via the plug-in) the first time the
>> >>> CloudStack volume is attached to a hypervisor.
>> >>>
>> >>> If you don't want to use the SolidFire plug-in, but still want to use
>> a
>> >>> SolidFire volume (LUN), you can do this already today (prior to 4.2).
>> The
>> >>> admin manually creates the SAN volume and - in this case - multiple
>> VMs
>> >> and
>> >>> data disks can share this SAN volume. While you can do this today, it
>> is
>> >>> not useful if you want to enforce storage QoS.
>> >>>
>> >>>> Are the SAN volumes by the management server or externally by the
>> >> operator?
>> >>> When the SolidFire plug-in is used, the SAN volumes are completely
>> >> managed
>> >>> by the management server (via the plug-in). There is no admin
>> >> interaction.
>> >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
>> >> volume,
>> >>> which is necessary for any storage vendor that supports true QoS.
>> >>>
>> >>>>
>> >>>> I would like to clarify how these pieces are related and expected to
>> >>>> operate.
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com>
>> >>>> wrote:
>> >>>>
>> >>>>> "In particular, how do we ensure that multiple VMs with provisioned
>> >> IOPS
>> >>>>> won't be cut off by the underlying storage."
>> >>>>>
>> >>>>> In the storage QoS world, we need to map a single SAN volume (LUN)
>> to a
>> >>>>> single CloudStack volume. We cannot have multiple CloudStack volumes
>> >>>>> sharing a single SAN volume and still guarantee QoS.
>> >>>>>
>> >>>>> If the user wants to have a single SAN volume house more than one
>> >>>>> CloudStack volume, then can do that today without any of my plug-in
>> >> code.
>> >>>>>
>> >>>>>
>> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com
>> >>>>>> wrote:
>> >>>>>
>> >>>>>> "The administrator will allocate a SAN volume for CloudStack's use
>> >> onto
>> >>>>>> which CloudStack volumes will be created."
>> >>>>>>
>> >>>>>> I think we crossed e-mails. :)
>> >>>>>>
>> >>>>>> Check out my recent e-mail on this.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> Mike,
>> >>>>>>>
>> >>>>>>> You are coming to the part which concerns me -- concepts from the
>> >>>>>>> hypervisor are leaking into storage layer.
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> -John
>> >>>>>>>
>> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> The weird part is that the iSCSI type is today only used (as far
>> as
>> >> I
>> >>>>>>> know)
>> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>> >>>>>>>>
>> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the
>> vmfs
>> >>>> type.
>> >>>>>>>>
>> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>> >>>>>>> SharedMountPoint
>> >>>>>>>> type.
>> >>>>>>>>
>> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a new
>> type
>> >>>> of
>> >>>>>>>> storage to describe this dynamic ability Edison added into the
>> >> storage
>> >>>>>>>> framework. Maybe it should be more specificy, though:
>> Dynamic_iSCSI
>> >>>>>>> versus,
>> >>>>>>>> say, Dynamic_FC.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> "The storage device itself shouldn't know or care that it is
>> being
>> >>>> used
>> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it is
>> >>>>>>> storing."
>> >>>>>>>>>
>> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in
>> needs to
>> >>>>>>> call
>> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
>> >> treat
>> >>>>>>> it as
>> >>>>>>>>> such.
>> >>>>>>>>>
>> >>>>>>>>> I'm totally open to removing that constraint and just calling it
>> >>>> iSCSI
>> >>>>>>> or
>> >>>>>>>>> whatever. We would just need a way for the hypervisor attach
>> logic
>> >> to
>> >>>>>>>>> detect this new requirement and perform the necessary
>> activities.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <
>> jburwell@basho.com>
>> >>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Mike,
>> >>>>>>>>>>
>> >>>>>>>>>> See my responses in-line.
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> -John
>> >>>>>>>>>>
>> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> I'm trying to picture this:
>> >>>>>>>>>>>
>> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
>> >>>>>>> operator
>> >>>>>>>>>> may
>> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
>> >>>> shared
>> >>>>>>> by
>> >>>>>>>>>>> many services, and the operator has chosen to dedicate only a
>> >>>> portion
>> >>>>>>>>>> of it
>> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>> >>>>>>>>>>>
>> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
>> >>>>>>> (before
>> >>>>>>>>>> an
>> >>>>>>>>>>> attach request to the hypervisor) would work unless the
>> >> hypervisor
>> >>>>>>>>>> consumes
>> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
>> >>>>>>>>>>
>> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator
>> >>>> allocates
>> >>>>>>> a
>> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN.
>> They
>> >>>>>>> simply
>> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but not
>> allow
>> >>>>>>>>>> CloudStack to allocate anymore LUNs.
>> >>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be
>> >> passing
>> >>>>>>> in a
>> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor support
>> >>>> (like
>> >>>>>>> an
>> >>>>>>>>>> SR)
>> >>>>>>>>>>> and the logic will fail.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Are you thinking we should maybe have the storage framework
>> >> itself
>> >>>>>>>>>> detect
>> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor side
>> and
>> >>>>>>> have
>> >>>>>>>>>> it
>> >>>>>>>>>>> call into the agent code specifically to create the SR before
>> the
>> >>>>>>> attach
>> >>>>>>>>>>> logic runs in the agent code?
>> >>>>>>>>>>
>> >>>>>>>>>> I think the hypervisor management plugin should have a rich
>> enough
>> >>>>>>>>>> interface to storage to determine available for volume storage.
>> >> For
>> >>>>>>> Xen,
>> >>>>>>>>>> this interface would allow the interrogation of the device to
>> >>>>>>> determine the
>> >>>>>>>>>> SR is present. The storage device itself shouldn't know or
>> care
>> >>>>>>> that it
>> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer
>> questions
>> >>>>>>> about it
>> >>>>>>>>>> is storing.
>> >>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> So, the flow is as follows:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a type
>> of
>> >>>>>>>>>> so-called
>> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage
>> shows
>> >>>> up
>> >>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>> applicable zone.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The admin creates a Disk Offering that references the
>> storage
>> >>>> tag
>> >>>>>>> of
>> >>>>>>>>>> the
>> >>>>>>>>>>>> newly created Primary Storage.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a
>> new
>> >>>> row
>> >>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>> cloud.volumes table.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
>> >>>> disk).
>> >>>>>>>>>> This
>> >>>>>>>>>>>> leads to the storage framework calling the plug-in to create
>> a
>> >> new
>> >>>>>>>>>> volume
>> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
>> >>>> updates
>> >>>>>>> the
>> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the
>> SAN
>> >>>>>>>>>> volume).
>> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume
>> is in
>> >>>> the
>> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in
>> the
>> >>>>>>> 'Ready'
>> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code will
>> not
>> >> be
>> >>>>>>>>>> called
>> >>>>>>>>>>>> again to create this SAN volume.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
>> CloudStack
>> >>>>>>> volume
>> >>>>>>>>>> to
>> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>> >>>> structure
>> >>>>>>>>>> (ex.
>> >>>>>>>>>>>> an SR on XenServer).
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in
>> these
>> >>>>>>>>>>>>> definitions, refers to the physical allocation on the
>> device,
>> >>>>>>>>>> correct?"
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in
>> CloudStack, I
>> >>>>>>> try to
>> >>>>>>>>>>>>> use 'CloudStack volume'.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm having
>> a
>> >>>> hard
>> >>>>>>>>>> time
>> >>>>>>>>>>>>>> envisioning it.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
>> >> was
>> >>>>>>> the
>> >>>>>>>>>>>>>> initial person to suggest the approach I took.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> What do you think?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>> >>>> jburwell@basho.com
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
>> >> storage
>> >>>>>>>>>> device
>> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we are
>> >>>>>>>>>> communicating with
>> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
>> >>>>>>> (Dynamic)
>> >>>>>>>>>> on the
>> >>>>>>>>>>>>>>> device while the other requires the volume to be volume
>> to be
>> >>>>>>>>>> managed
>> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are
>> in
>> >>>>>>> sync on
>> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
>> >>>> physical
>> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
>> >> able
>> >>>> to
>> >>>>>>>>>>>>>>> communicate with a storage device to move bits from one
>> place
>> >>>> to
>> >>>>>>>>>> another,
>> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device
>> >> may
>> >>>> be
>> >>>>>>>>>> able to
>> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
>> >>>> iSCSI
>> >>>>>>>>>> device
>> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>> >>>> SolidFire
>> >>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs
>> and
>> >>>>>>> storing
>> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage
>> a
>> >>>>>>> device,
>> >>>>>>>>>> an
>> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack
>> (e.g.
>> >>>> the
>> >>>>>>>>>> device is
>> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
>> >>>> dedicate
>> >>>>>>>>>> only a
>> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make
>> sense?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>> >> appropriate
>> >>>>>>> to
>> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add
>> the
>> >>>>>>> notion
>> >>>>>>>>>> of a
>> >>>>>>>>>>>>>>> storage device with an attached driver that indicates
>> whether
>> >>>> or
>> >>>>>>>>>> not is
>> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>> >>>> representing a
>> >>>>>>>>>> physical
>> >>>>>>>>>>>>>>> allocation on a device separate that is associated with a
>> >>>> volume.
>> >>>>>>>>>> With
>> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare
>> which
>> >>>>>>>>>> protocols they
>> >>>>>>>>>>>>>>> support and when they encounter a device managed by
>> >> CloudStack,
>> >>>>>>>>>> utilize the
>> >>>>>>>>>>>>>>> management operations exposed by the driver to automate
>> >>>>>>> allocation.
>> >>>>>>>>>> If
>> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down
>> and
>> >>>>>>> drill
>> >>>>>>>>>> down to
>> >>>>>>>>>>>>>>> a more detailed design.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
>> >> the
>> >>>>>>>>>> Dynamic
>> >>>>>>>>>>>>>>> type:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
>> >>>> Storage
>> >>>>>>>>>> based
>> >>>>>>>>>>>>>>> on
>> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume
>> must
>> >>>>>>> exist
>> >>>>>>>>>> on
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
>> >>>> volumes
>> >>>>>>> are
>> >>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
>> >>>> Storage
>> >>>>>>>>>> based
>> >>>>>>>>>>>>>>> on a
>> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its
>> storage
>> >>>>>>> system
>> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework).
>> When
>> >> a
>> >>>>>>> user
>> >>>>>>>>>>>>>>> wants to
>> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the
>> framework
>> >>>> tells
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
>> >> attach
>> >>>>>>>>>> logic
>> >>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
>> >>>>>>> structure
>> >>>>>>>>>>>>>>> exists
>> >>>>>>>>>>>>>>>> at this point because the volume was just created. The
>> >>>>>>> hypervisor
>> >>>>>>>>>> data
>> >>>>>>>>>>>>>>>> structure must be created.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have defined
>> them
>> >>>> up
>> >>>>>>>>>> front
>> >>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>> you. :)
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an
>> admin
>> >>>>>>>>>> creates a
>> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
>> >> created
>> >>>>>>> to
>> >>>>>>>>>>>>>>> consume
>> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor
>> data
>> >>>>>>>>>> structure
>> >>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
>> >> this
>> >>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach
>> logic
>> >>>>>>> assumes
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
>> >>>>>>> attaches
>> >>>>>>>>>>>>>>> the VDI
>> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
>> Instead
>> >>>> of
>> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume
>> on
>> >> a
>> >>>>>>> SAN,
>> >>>>>>>>>>>>>>> but the
>> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
>> created
>> >>>>>>> when an
>> >>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes have
>> not
>> >>>>>>> been
>> >>>>>>>>>>>>>>> pre-hooked
>> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
>> >> the
>> >>>>>>>>>> attach
>> >>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
>> >> attaches
>> >>>>>>> the
>> >>>>>>>>>>>>>>> one and
>> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>> >>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
>> >>>> behavior
>> >>>>>>>>>>>>>>> works in
>> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the
>> difference.
>> >>>>>>> Another
>> >>>>>>>>>>>>>>> way to
>> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
>> >>>>>>> storage
>> >>>>>>>>>>>>>>> pool type?
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>> >>>>>>> XenServer
>> >>>>>>>>>>>>>>> when you
>> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based
>> on
>> >> an
>> >>>>>>>>>> iSCSI
>> >>>>>>>>>>>>>>>>>> target.
>> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>> >> hypervisor
>> >>>>>>> and
>> >>>>>>>>>>>>>>>>>> creating a
>> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>> >> CloudStack
>> >>>>>>> does
>> >>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>> part
>> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
>> >> For
>> >>>>>>> all
>> >>>>>>>>>>>>>>> other
>> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
>> >>>> perform
>> >>>>>>>>>> this
>> >>>>>>>>>>>>>>> step
>> >>>>>>>>>>>>>>>>>>> manually.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>> >>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
>> between
>> >>>> the
>> >>>>>>>>>>>>>>> ISCSI and
>> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>> >>>>>>>>>>>>>>> Storage.StoragePoolType
>> >>>>>>>>>>>>>>>>>> class
>> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>> >> overlay
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> RBD(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is
>> set
>> >>>> up
>> >>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>> customers.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>> >>>>>>>>>> SolidFire)
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> boolean shared;
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> return shared;
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company
>> wants
>> >> to
>> >>>>>>>>>>>>>>> implement a
>> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service
>> feature. It
>> >>>>>>> would
>> >>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>> dynamic,
>> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>> >>>>>>> implement a
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>> >>>>>>>>>>>>>>> hypervisor-attach
>> >>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>> >>>>>>> specific.
>> >>>>>>>>>>>>>>> It is
>> >>>>>>>>>>>>>>>>>> not.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of
>> dynamic,
>> >>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind
>> of
>> >>>>>>>>>> storage).
>> >>>>>>>>>>>>>>> This
>> >>>>>>>>>>>>>>>>>>>> kind of
>> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
>> framework
>> >>>>>>>>>> changes.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
>> >>>>>>> creation
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>> deletion
>> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited
>> with
>> >>>> him
>> >>>>>>> in
>> >>>>>>>>>>>>>>> Portland
>> >>>>>>>>>>>>>>>>>>>> back
>> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
>> >>>>>>> realized
>> >>>>>>>>>>>>>>> there
>> >>>>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes
>> unless
>> >>>> the
>> >>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
>> >>>> create
>> >>>>>>> the
>> >>>>>>>>>>>>>>>>>>>> appropriate
>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>> >>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in
>> the
>> >>>>>>> Storage
>> >>>>>>>>>>>>>>> layer
>> >>>>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to
>> see
>> >> the
>> >>>>>>>>>> Storage
>> >>>>>>>>>>>>>>>>>>>> services
>> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>> >> composed/driven
>> >>>>>>> by
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>> Hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
>> structures
>> >>>> per
>> >>>>>>>>>> their
>> >>>>>>>>>>>>>>>>>> needs.
>> >>>>>>>>>>>>>>>>>>>> If
>> >>>>>>>>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
>> >>>> with a
>> >>>>>>>>>>>>>>> massive
>> >>>>>>>>>>>>>>>>>>>> n-to-n
>> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>> >>>>>>> difficult to
>> >>>>>>>>>>>>>>>>>> maintain
>> >>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen
>> specific
>> >>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>> code
>> >>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>> >>>>>>>>>> XenServer,
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> admin
>> >>>>>>>>>>>>>>>>>>>>>>>> had
>> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
>> >> then
>> >>>> go
>> >>>>>>>>>> into
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
>> >> volume
>> >>>>>>> (ex.
>> >>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
>> ESX(i)).
>> >>>> VMs
>> >>>>>>> and
>> >>>>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>>>>> disks
>> >>>>>>>>>>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage
>> need
>> >> no
>> >>>>>>>>>> longer
>> >>>>>>>>>>>>>>> be so
>> >>>>>>>>>>>>>>>>>>>>>>>> static
>> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
>> >> between
>> >>>> a
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>> system's
>> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
>> >> storage
>> >>>>>>>>>> Quality
>> >>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>> Service).
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>> >>>>>>> dynamically
>> >>>>>>>>>>>>>>> create
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> delete
>> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
>> >>>> address
>> >>>>>>> is
>> >>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>> creating
>> >>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
>> >> when
>> >>>>>>>>>>>>>>> performing an
>> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to
>> do
>> >> so.
>> >>>>>>> I've
>> >>>>>>>>>>>>>>> got
>> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
>> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
>> >>>> sandbox
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>> submit
>> >>>>>>>>>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>> >>>>>>> agnostic.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in
>> the
>> >>>>>>> agent
>> >>>>>>>>>>>>>>> code).
>> >>>>>>>>>>>>>>>>>> The
>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
>> >> create a
>> >>>>>>>>>> volume
>> >>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>>>>>>>> needed,
>> >>>>>>>>>>>>>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>> >>>>>>> hypervisor,
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>> >>>>>>> invoked on
>> >>>>>>>>>>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
>> >> create,
>> >>>>>>>>>> say, a
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>> >>>>>>> VMware) to
>> >>>>>>>>>>>>>>> make
>> >>>>>>>>>>>>>>>>>> use
>> >>>>>>>>>>>>>>>>>>>>>>>> of the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently
>> making
>> >>>> the
>> >>>>>>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>> >>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>> specific?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>> >>>>>>> hypervisors
>> >>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for
>> 4.2
>> >> to
>> >>>>>>> get
>> >>>>>>>>>> KVM
>> >>>>>>>>>>>>>>> in.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>> >>>>>>> Depending on
>> >>>>>>>>>>>>>>> what
>> >>>>>>>>>>>>>>>>>> we do
>> >>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
>> working
>> >>>> for
>> >>>>>>>>>>>>>>> VMware in
>> >>>>>>>>>>>>>>>>>> my
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in
>> regards
>> >> to
>> >>>>>>> Disk
>> >>>>>>>>>>>>>>>>>> Offerings.
>> >>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
>> Jamison
>> >>>>>>> Damage
>> >>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
>> >> patch
>> >>>>>>> cycle
>> >>>>>>>>>>>>>>> post
>> >>>>>>>>>>>>>>>>>> 4.2?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>> >>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
>> >> MASTER
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
>> >>>> mine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
>> >> non
>> >>>>>>> issue
>> >>>>>>>>>>>>>>> as I
>> >>>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
>> >> now,
>> >>>>>>>>>> users
>> >>>>>>>>>>>>>>> need
>> >>>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>> >>>>>>>>>>>>>>> service_offering
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>> >>>>>>> calculation.
>> >>>>>>>>>> Of
>> >>>>>>>>>>>>>>> course
>> >>>>>>>>>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>> >>>>>>> individual
>> >>>>>>>>>>>>>>> volume,
>> >>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can
>> work
>> >> on
>> >>>> it
>> >>>>>>>>>>>>>>> later. Do
>> >>>>>>>>>>>>>>>>>> you
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the
>> question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>> >> features.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am
>> trying
>> >> to
>> >>>>>>>>>> adding
>> >>>>>>>>>>>>>>> more.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>> >>>>>>> statistics
>> >>>>>>>>>>>>>>> rather
>> >>>>>>>>>>>>>>>>>> than
>> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
>> >> you
>> >>>>>>> have
>> >>>>>>>>>>>>>>> time.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including
>> bytes/iops
>> >>>>>>>>>> read/write
>> >>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are
>> billed
>> >>>> more
>> >>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>> volumes
>> >>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>> >>>>>>> operations (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
>> >> imagine
>> >>>>>>> our
>> >>>>>>>>>>>>>>> users
>> >>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> like
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
>> Could
>> >> an
>> >>>>>>>>>>>>>>> operator
>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>> >>>>>>> supports
>> >>>>>>>>>>>>>>> setting
>> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
>> >>>> through
>> >>>>>>>>>>>>>>> command
>> >>>>>>>>>>>>>>>>>> line.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>> >>>>>>> parameters
>> >>>>>>>>>>>>>>> of a
>> >>>>>>>>>>>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>> >>>> apologize
>> >>>>>>>>>> for
>> >>>>>>>>>>>>>>> being
>> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
>> >> with a
>> >>>>>>> set
>> >>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided
>> by a
>> >>>>>>>>>> resource?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
>> >> better
>> >>>> to
>> >>>>>>>>>>>>>>> commit
>> >>>>>>>>>>>>>>>>>> Mike's
>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
>> consider
>> >>>> the
>> >>>>>>>>>>>>>>>>>> limitation in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>> >>>> response
>> >>>>>>> to
>> >>>>>>>>>>>>>>> Mike
>> >>>>>>>>>>>>>>>>>>>>>>>> regarding
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features.
>> I
>> >>>> think
>> >>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire,
>> but
>> >> I
>> >>>>>>> think
>> >>>>>>>>>>>>>>> we need
>> >>>>>>>>>>>>>>>>>>>>>>>> closer
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches
>> (possibly
>> >>>> have
>> >>>>>>>>>> have
>> >>>>>>>>>>>>>>>>>> solidfire
>> >>>>>>>>>>>>>>>>>>>>>>>>> track
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
>> >>>> issue.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <
>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>> >>>>>>> following:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>> >> policies
>> >>>> be
>> >>>>>>>>>>>>>>> captured
>> >>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>> >> purposes,
>> >>>>>>>>>>>>>>> expressing
>> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface
>> model
>> >>>> for
>> >>>>>>>>>>>>>>> reasoning
>> >>>>>>>>>>>>>>>>>>>>>>>> about
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>> >>>>>>>>>>>>>>> configurations
>> >>>>>>>>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device
>> reconciled?
>> >> In
>> >>>>>>>>>>>>>>> particular,
>> >>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
>> >> IOPS
>> >>>>>>>>>>>>>>> configured
>> >>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>> a VM
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>> >>>> deliver.
>> >>>>>>>>>>>>>>> Another
>> >>>>>>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM
>> and a
>> >>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>> device at
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to
>> storage
>> >>>>>>> device
>> >>>>>>>>>>>>>>>>>> introduces
>> >>>>>>>>>>>>>>>>>>>>>>>>> logical
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
>> Tutkowski
>> >> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>> >>>> controlling
>> >>>>>>> the
>> >>>>>>>>>>>>>>> max
>> >>>>>>>>>>>>>>>>>>>>>>>> number of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling
>> IOPS
>> >>>> from
>> >>>>>>> the
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>>>>>>> system
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
>> >> Burwell
>> >>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should
>> be
>> >>>>>>> merged
>> >>>>>>>>>>>>>>> until all
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is
>> ready
>> >> to
>> >>>>>>> turn
>> >>>>>>>>>>>>>>> over to
>> >>>>>>>>>>>>>>>>>>>>>>>> test.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops
>> issue
>> >>>> is
>> >>>>>>>>>>>>>>> addressed
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has
>> been
>> >>>>>>>>>>>>>>> satisfied.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>> >>>> intersect/compliment
>> >>>>>>> the
>> >>>>>>>>>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
>> >> As I
>> >>>>>>>>>>>>>>> understand
>> >>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
>> >>>> like
>> >>>>>>> to
>> >>>>>>>>>>>>>>> ensure
>> >>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM
>> and
>> >>>>>>>>>>>>>>> SolidFire are
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>> >> wido@widodh.nl
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features
>> into
>> >>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
>> >> reason
>> >>>>>>> till
>> >>>>>>>>>>>>>>> Monday,
>> >>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
>> >>>> mention
>> >>>>>>>>>> again
>> >>>>>>>>>>>>>>>>>> that we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so
>> that we
>> >>>>>>> never
>> >>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>> version
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>> >>>> throttling
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>> that's
>> >>>>>>>>>>>>>>>>>>>>>>>> great!
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy
>> from
>> >>>> day
>> >>>>>>>>>> #1.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >>>> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
>> >> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
>> discretely
>> >>>>>>> support
>> >>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
>> >>>> because
>> >>>>>>> I
>> >>>>>>>>>>>>>>> don't see
>> >>>>>>>>>>>>>>>>>>>>>>>> how we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>> >>>>>>>>>> Therefore, we
>> >>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>> >>>>>>> backwards
>> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into
>> master
>> >>>> now
>> >>>>>>> so
>> >>>>>>>>>>>>>>> that it
>> >>>>>>>>>>>>>>>>>>>>>>>> will go
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
>> >> before
>> >>>>>>> the
>> >>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
>> >> instead
>> >>>>>>> of
>> >>>>>>>>>>>>>>> Total
>> >>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
>> >> freeze
>> >>>>>>> date.
>> >>>>>>>>>>>>>>> But if
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>> >>>> necessary
>> >>>>>>>>>> to do
>> >>>>>>>>>>>>>>>>>> that.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is
>> no
>> >>>>>>>>>> association
>> >>>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>> >>>>>>> Hollander <
>> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>> >> Finally I
>> >>>>>>>>>>>>>>> decided to
>> >>>>>>>>>>>>>>>>>>>>>>>> ignore
>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
>> >> because
>> >>>>>>> the
>> >>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and
>> received
>> >>>>>>> bytes as
>> >>>>>>>>>>>>>>> well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking
>> and
>> >>>> disk
>> >>>>>>> I/O
>> >>>>>>>>>>>>>>>>>> completely
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>> >>>>>>>>>> situations
>> >>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
>> >>>> work.
>> >>>>>>> It
>> >>>>>>>>>>>>>>> could be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
>> >>>> freeze,
>> >>>>>>> I
>> >>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss.
>> But
>> >>>> if
>> >>>>>>> it
>> >>>>>>>>>>>>>>> goes
>> >>>>>>>>>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
>> >> release
>> >>>>>>> with
>> >>>>>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> >>>> wido@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge
>> disk_io_throttling
>> >>>>>>> branch
>> >>>>>>>>>> into
>> >>>>>>>>>>>>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into
>> master
>> >>>> in
>> >>>>>>> 48
>> >>>>>>>>>>>>>>> hours.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the
>> same
>> >>>>>>> storage
>> >>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>> (local
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>> >>>> limitation
>> >>>>>>> of
>> >>>>>>>>>>>>>>> device
>> >>>>>>>>>>>>>>>>>>>>>>>> (such as
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
>> >>>> affect
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
>> >>>> device.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
>> >> and
>> >>>>>>> limit
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>>>>>> I/O
>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>> >>>>>>> difference
>> >>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>> >>>>>>> different
>> >>>>>>>>>>>>>>> rate
>> >>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of
>> users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side
>> you
>> >>>>>>>>>> collect
>> >>>>>>>>>>>>>>> both
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you
>> only
>> >> go
>> >>>>>>> for
>> >>>>>>>>>> a
>> >>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>> value.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
>> expensive
>> >>>>>>> then
>> >>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>> IOps,
>> >>>>>>>>>>>>>>>>>>>>>>>> so it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an
>> admin
>> >>>>>>> would
>> >>>>>>>>>> set
>> >>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>> lower
>> >>>>>>>>>>>>>>>>>>>>>>>>> value
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
>> >> point I
>> >>>>>>>>>> think
>> >>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
>> >>>> place
>> >>>>>>> to
>> >>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>>>> both,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>> >>>>>>> different
>> >>>>>>>>>>>>>>> values
>> >>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and
>> set
>> >>>> that
>> >>>>>>> as
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
>> >> this
>> >>>>>>> way?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>> >>>>>>>>>> disk_offering,
>> >>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
>> >> iops)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>> >> https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
>> execution
>> >>>>>>>>>> success?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies
>> introduced?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>> >>>>>>> integration)
>> >>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>> included
>> >>>>>>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check
>> for
>> >>>>>>>>>> potential
>> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>> >>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>> UI.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
>> >>>> expunge.
>> >>>>>>>>>>>>>>> migrate,
>> >>>>>>>>>>>>>>>>>>>>>>>> restore
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>> >> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>> >> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> >> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> http://solidfire.com/solution/overview/?video=play
>> >>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>> cloud<
>> >> http://solidfire.com/solution/overview/?video=play
>> >>>>>
>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>> >>>
>> >>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>> >
>> >>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the
>> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the cloud<
>> >>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Oh, OK, that sounds really good, John.
Thanks and talk to you tomorrow! :)
On Tue, Jun 4, 2013 at 3:42 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> I am never at a loss for an opinion. I some thoughts, but want to
> confirm assumptions and ideas against the solidfire, disk_io_throttle,
> and
> object_store branches. I hope to collect them in a coherent form
> tomorrow (5 June 2013).
>
> Thanks,
> -John
>
>
> On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > "So, in essence, the SolidFire plugin introduces the notion of a managed
> > iSCSI device and provisioned IOPS."
> >
> > Technically, the SolidFire plug-in just introduces the notion of
> > provisioned storage IOPS.
> >
> > The storage framework that leverages the plug-in was incomplete, so I had
> > to try to add in the notion of a managed iSCSI device.
> >
> > I appreciate all the time you've been spending on this. :) Do you have a
> > recommendation as to how we should accomplish what you're looking for?
> >
> > Thanks!
> >
> >
> > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com> wrote:
> >
> >> Mike,
> >>
> >> So, in essence, the SolidFire plugin introduces the notion of a managed
> >> iSCSI device and provisioned IOPS.
> >>
> >> I want to see a separation of the management capabilities (i.e. can a
> >> device be managed/does an operator want it managed by CloudStack) from
> the
> >> storage protocol. Ideally, we should end up with a semantic that will
> >> allow any type of storage device to be managed. I also want to make
> >> progress on decoupling the storage types from the hypervisor
> definitions.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Hi John,
> >>>
> >>> No problem. Answers are below in red.
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com>
> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Could you please answer the following questions for me with regards to
> >> the
> >>>> operation of the SolidFire plugin:
> >>>>
> >>>> What is the cardinality between iSCSI LUNs and SAN volumes?
> >>> Each SAN volume is equivalent to a single LUN (LUN 0).
> >>> 1 SAN volume : 1 LUN
> >>>
> >>>> What is the cardinality between SAN Volumes and CloudStack Volumes?
> >>> 1 SAN volume : 1 CloudStack volume
> >>>
> >>>> Are the LUN(s) created by the management server or externally by the
> >>>> operator?
> >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN)
> is
> >>> created by the management server (via the plug-in) the first time the
> >>> CloudStack volume is attached to a hypervisor.
> >>>
> >>> If you don't want to use the SolidFire plug-in, but still want to use a
> >>> SolidFire volume (LUN), you can do this already today (prior to 4.2).
> The
> >>> admin manually creates the SAN volume and - in this case - multiple VMs
> >> and
> >>> data disks can share this SAN volume. While you can do this today, it
> is
> >>> not useful if you want to enforce storage QoS.
> >>>
> >>>> Are the SAN volumes by the management server or externally by the
> >> operator?
> >>> When the SolidFire plug-in is used, the SAN volumes are completely
> >> managed
> >>> by the management server (via the plug-in). There is no admin
> >> interaction.
> >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
> >> volume,
> >>> which is necessary for any storage vendor that supports true QoS.
> >>>
> >>>>
> >>>> I would like to clarify how these pieces are related and expected to
> >>>> operate.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> "In particular, how do we ensure that multiple VMs with provisioned
> >> IOPS
> >>>>> won't be cut off by the underlying storage."
> >>>>>
> >>>>> In the storage QoS world, we need to map a single SAN volume (LUN)
> to a
> >>>>> single CloudStack volume. We cannot have multiple CloudStack volumes
> >>>>> sharing a single SAN volume and still guarantee QoS.
> >>>>>
> >>>>> If the user wants to have a single SAN volume house more than one
> >>>>> CloudStack volume, then can do that today without any of my plug-in
> >> code.
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com
> >>>>>> wrote:
> >>>>>
> >>>>>> "The administrator will allocate a SAN volume for CloudStack's use
> >> onto
> >>>>>> which CloudStack volumes will be created."
> >>>>>>
> >>>>>> I think we crossed e-mails. :)
> >>>>>>
> >>>>>> Check out my recent e-mail on this.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> You are coming to the part which concerns me -- concepts from the
> >>>>>>> hypervisor are leaking into storage layer.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> The weird part is that the iSCSI type is today only used (as far
> as
> >> I
> >>>>>>> know)
> >>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
> >>>>>>>>
> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
> >>>> type.
> >>>>>>>>
> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
> >>>>>>> SharedMountPoint
> >>>>>>>> type.
> >>>>>>>>
> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a new
> type
> >>>> of
> >>>>>>>> storage to describe this dynamic ability Edison added into the
> >> storage
> >>>>>>>> framework. Maybe it should be more specificy, though:
> Dynamic_iSCSI
> >>>>>>> versus,
> >>>>>>>> say, Dynamic_FC.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> "The storage device itself shouldn't know or care that it is
> being
> >>>> used
> >>>>>>>>> for a Xen SR -- simply be able to answer questions about it is
> >>>>>>> storing."
> >>>>>>>>>
> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in needs
> to
> >>>>>>> call
> >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
> >> treat
> >>>>>>> it as
> >>>>>>>>> such.
> >>>>>>>>>
> >>>>>>>>> I'm totally open to removing that constraint and just calling it
> >>>> iSCSI
> >>>>>>> or
> >>>>>>>>> whatever. We would just need a way for the hypervisor attach
> logic
> >> to
> >>>>>>>>> detect this new requirement and perform the necessary activities.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jburwell@basho.com
> >
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> See my responses in-line.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I'm trying to picture this:
> >>>>>>>>>>>
> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
> >>>>>>> operator
> >>>>>>>>>> may
> >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
> >>>> shared
> >>>>>>> by
> >>>>>>>>>>> many services, and the operator has chosen to dedicate only a
> >>>> portion
> >>>>>>>>>> of it
> >>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>>>>>>>>
> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
> >>>>>>> (before
> >>>>>>>>>> an
> >>>>>>>>>>> attach request to the hypervisor) would work unless the
> >> hypervisor
> >>>>>>>>>> consumes
> >>>>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>>>>>
> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator
> >>>> allocates
> >>>>>>> a
> >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN.
> They
> >>>>>>> simply
> >>>>>>>>>> want to turn control of that LUN over to CloudStack, but not
> allow
> >>>>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be
> >> passing
> >>>>>>> in a
> >>>>>>>>>>> SAN volume that does not have the necessary hypervisor support
> >>>> (like
> >>>>>>> an
> >>>>>>>>>> SR)
> >>>>>>>>>>> and the logic will fail.
> >>>>>>>>>>>
> >>>>>>>>>>> Are you thinking we should maybe have the storage framework
> >> itself
> >>>>>>>>>> detect
> >>>>>>>>>>> that such a SAN volume needs support from the hypervisor side
> and
> >>>>>>> have
> >>>>>>>>>> it
> >>>>>>>>>>> call into the agent code specifically to create the SR before
> the
> >>>>>>> attach
> >>>>>>>>>>> logic runs in the agent code?
> >>>>>>>>>>
> >>>>>>>>>> I think the hypervisor management plugin should have a rich
> enough
> >>>>>>>>>> interface to storage to determine available for volume storage.
> >> For
> >>>>>>> Xen,
> >>>>>>>>>> this interface would allow the interrogation of the device to
> >>>>>>> determine the
> >>>>>>>>>> SR is present. The storage device itself shouldn't know or
> care
> >>>>>>> that it
> >>>>>>>>>> is being used for a Xen SR -- simply be able to answer questions
> >>>>>>> about it
> >>>>>>>>>> is storing.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
> >>>>>>>>>> so-called
> >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage
> shows
> >>>> up
> >>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>> applicable zone.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The admin creates a Disk Offering that references the
> storage
> >>>> tag
> >>>>>>> of
> >>>>>>>>>> the
> >>>>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a
> new
> >>>> row
> >>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
> >>>> disk).
> >>>>>>>>>> This
> >>>>>>>>>>>> leads to the storage framework calling the plug-in to create a
> >> new
> >>>>>>>>>> volume
> >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
> >>>> updates
> >>>>>>> the
> >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the
> SAN
> >>>>>>>>>> volume).
> >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is
> in
> >>>> the
> >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
> >>>>>>> 'Ready'
> >>>>>>>>>>>> state (even after a detach disk) and the plug-in code will not
> >> be
> >>>>>>>>>> called
> >>>>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>>>
> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the
> CloudStack
> >>>>>>> volume
> >>>>>>>>>> to
> >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
> >>>> structure
> >>>>>>>>>> (ex.
> >>>>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in
> these
> >>>>>>>>>>>>> definitions, refers to the physical allocation on the device,
> >>>>>>>>>> correct?"
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in
> CloudStack, I
> >>>>>>> try to
> >>>>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm having a
> >>>> hard
> >>>>>>>>>> time
> >>>>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
> >> was
> >>>>>>> the
> >>>>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >>>> jburwell@basho.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
> >> storage
> >>>>>>>>>> device
> >>>>>>>>>>>>>>> management and storage protocols. In both cases, we are
> >>>>>>>>>> communicating with
> >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
> >>>>>>> (Dynamic)
> >>>>>>>>>> on the
> >>>>>>>>>>>>>>> device while the other requires the volume to be volume to
> be
> >>>>>>>>>> managed
> >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are
> in
> >>>>>>> sync on
> >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
> >>>> physical
> >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
> >> able
> >>>> to
> >>>>>>>>>>>>>>> communicate with a storage device to move bits from one
> place
> >>>> to
> >>>>>>>>>> another,
> >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device
> >> may
> >>>> be
> >>>>>>>>>> able to
> >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
> >>>> iSCSI
> >>>>>>>>>> device
> >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
> >>>> SolidFire
> >>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs
> and
> >>>>>>> storing
> >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
> >>>>>>> device,
> >>>>>>>>>> an
> >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack
> (e.g.
> >>>> the
> >>>>>>>>>> device is
> >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
> >>>> dedicate
> >>>>>>>>>> only a
> >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make
> sense?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
> >> appropriate
> >>>>>>> to
> >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
> >>>>>>> notion
> >>>>>>>>>> of a
> >>>>>>>>>>>>>>> storage device with an attached driver that indicates
> whether
> >>>> or
> >>>>>>>>>> not is
> >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
> >>>> representing a
> >>>>>>>>>> physical
> >>>>>>>>>>>>>>> allocation on a device separate that is associated with a
> >>>> volume.
> >>>>>>>>>> With
> >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare
> which
> >>>>>>>>>> protocols they
> >>>>>>>>>>>>>>> support and when they encounter a device managed by
> >> CloudStack,
> >>>>>>>>>> utilize the
> >>>>>>>>>>>>>>> management operations exposed by the driver to automate
> >>>>>>> allocation.
> >>>>>>>>>> If
> >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down
> and
> >>>>>>> drill
> >>>>>>>>>> down to
> >>>>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
> >> the
> >>>>>>>>>> Dynamic
> >>>>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
> >>>> Storage
> >>>>>>>>>> based
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume
> must
> >>>>>>> exist
> >>>>>>>>>> on
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
> >>>> volumes
> >>>>>>> are
> >>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
> >>>> Storage
> >>>>>>>>>> based
> >>>>>>>>>>>>>>> on a
> >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
> >>>>>>> system
> >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework).
> When
> >> a
> >>>>>>> user
> >>>>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
> >>>> tells
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
> >> attach
> >>>>>>>>>> logic
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
> >>>>>>> structure
> >>>>>>>>>>>>>>> exists
> >>>>>>>>>>>>>>>> at this point because the volume was just created. The
> >>>>>>> hypervisor
> >>>>>>>>>> data
> >>>>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have defined
> them
> >>>> up
> >>>>>>>>>> front
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an
> admin
> >>>>>>>>>> creates a
> >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
> >> created
> >>>>>>> to
> >>>>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
> >>>>>>>>>> structure
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
> >> this
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach
> logic
> >>>>>>> assumes
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
> >>>>>>> attaches
> >>>>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated.
> Instead
> >>>> of
> >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume
> on
> >> a
> >>>>>>> SAN,
> >>>>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be
> created
> >>>>>>> when an
> >>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>> volume is performed because these types of volumes have
> not
> >>>>>>> been
> >>>>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
> >> the
> >>>>>>>>>> attach
> >>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
> >> attaches
> >>>>>>> the
> >>>>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
> >>>> behavior
> >>>>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
> >>>>>>> Another
> >>>>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
> >>>>>>> storage
> >>>>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
> >>>>>>> XenServer
> >>>>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based
> on
> >> an
> >>>>>>>>>> iSCSI
> >>>>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
> >> hypervisor
> >>>>>>> and
> >>>>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
> >> CloudStack
> >>>>>>> does
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
> >> For
> >>>>>>> all
> >>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
> >>>> perform
> >>>>>>>>>> this
> >>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference
> between
> >>>> the
> >>>>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
> >> overlay
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
> >>>> up
> >>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
> >>>>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants
> >> to
> >>>>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature.
> It
> >>>>>>> would
> >>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
> >>>>>>> implement a
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
> >>>>>>> specific.
> >>>>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
> >>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
> >>>>>>>>>> storage).
> >>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage
> framework
> >>>>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
> >>>>>>> creation
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited
> with
> >>>> him
> >>>>>>> in
> >>>>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
> >>>>>>> realized
> >>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes
> unless
> >>>> the
> >>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
> >>>> create
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in
> the
> >>>>>>> Storage
> >>>>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see
> >> the
> >>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> >> composed/driven
> >>>>>>> by
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create
> structures
> >>>> per
> >>>>>>>>>> their
> >>>>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
> >>>> with a
> >>>>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
> >>>>>>> difficult to
> >>>>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
> >>>>>>>>>> XenServer,
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
> >> then
> >>>> go
> >>>>>>>>>> into
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
> >> volume
> >>>>>>> (ex.
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on
> ESX(i)).
> >>>> VMs
> >>>>>>> and
> >>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need
> >> no
> >>>>>>>>>> longer
> >>>>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
> >> between
> >>>> a
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
> >> storage
> >>>>>>>>>> Quality
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
> >>>>>>> dynamically
> >>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
> >>>> address
> >>>>>>> is
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
> >> when
> >>>>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do
> >> so.
> >>>>>>> I've
> >>>>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
> >>>> sandbox
> >>>>>>>>>> and
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
> >>>>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in
> the
> >>>>>>> agent
> >>>>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
> >> create a
> >>>>>>>>>> volume
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
> >>>>>>> hypervisor,
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
> >>>>>>> invoked on
> >>>>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
> >> create,
> >>>>>>>>>> say, a
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
> >>>>>>> VMware) to
> >>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently
> making
> >>>> the
> >>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
> >>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
> >>>>>>> hypervisors
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2
> >> to
> >>>>>>> get
> >>>>>>>>>> KVM
> >>>>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
> >>>>>>> Depending on
> >>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it
> working
> >>>> for
> >>>>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards
> >> to
> >>>>>>> Disk
> >>>>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey
> Jamison
> >>>>>>> Damage
> >>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
> >> patch
> >>>>>>> cycle
> >>>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
> >> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
> >>>> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
> >> non
> >>>>>>> issue
> >>>>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
> >> now,
> >>>>>>>>>> users
> >>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
> >>>>>>> calculation.
> >>>>>>>>>> Of
> >>>>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
> >>>>>>> individual
> >>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work
> >> on
> >>>> it
> >>>>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
> >> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying
> >> to
> >>>>>>>>>> adding
> >>>>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
> >>>>>>> statistics
> >>>>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
> >> you
> >>>>>>> have
> >>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
> >>>>>>>>>> read/write
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are
> billed
> >>>> more
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
> >>>>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
> >> imagine
> >>>>>>> our
> >>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.
> Could
> >> an
> >>>>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
> >>>>>>> supports
> >>>>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
> >>>> through
> >>>>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
> >>>>>>> parameters
> >>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
> >>>> apologize
> >>>>>>>>>> for
> >>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
> >> with a
> >>>>>>> set
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided
> by a
> >>>>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
> >> better
> >>>> to
> >>>>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to
> consider
> >>>> the
> >>>>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
> >>>> response
> >>>>>>> to
> >>>>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
> >>>> think
> >>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire,
> but
> >> I
> >>>>>>> think
> >>>>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
> >>>> have
> >>>>>>>>>> have
> >>>>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
> >>>> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jburwell@basho.com
> >
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
> >>>>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
> >> policies
> >>>> be
> >>>>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
> >> purposes,
> >>>>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface
> model
> >>>> for
> >>>>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> >>>>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled?
> >> In
> >>>>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
> >> IOPS
> >>>>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
> >>>> deliver.
> >>>>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM
> and a
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
> >>>>>>> device
> >>>>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike
> Tutkowski
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
> >>>> controlling
> >>>>>>> the
> >>>>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
> >>>> from
> >>>>>>> the
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
> >> Burwell
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
> >>>>>>> merged
> >>>>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is
> ready
> >> to
> >>>>>>> turn
> >>>>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops
> issue
> >>>> is
> >>>>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has
> been
> >>>>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >>>> intersect/compliment
> >>>>>>> the
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
> >> As I
> >>>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
> >>>> like
> >>>>>>> to
> >>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM
> and
> >>>>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
> >> wido@widodh.nl
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
> >>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
> >> reason
> >>>>>>> till
> >>>>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
> >>>> mention
> >>>>>>>>>> again
> >>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that
> we
> >>>>>>> never
> >>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
> >>>> throttling
> >>>>>>>>>> and
> >>>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy
> from
> >>>> day
> >>>>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
> >> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to
> discretely
> >>>>>>> support
> >>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
> >>>> because
> >>>>>>> I
> >>>>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
> >>>>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
> >>>>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into
> master
> >>>> now
> >>>>>>> so
> >>>>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
> >> before
> >>>>>>> the
> >>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
> >> instead
> >>>>>>> of
> >>>>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
> >> freeze
> >>>>>>> date.
> >>>>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
> >>>> necessary
> >>>>>>>>>> to do
> >>>>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> >>>>>>>>>> association
> >>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
> >>>>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
> >> Finally I
> >>>>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
> >> because
> >>>>>>> the
> >>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
> >>>>>>> bytes as
> >>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
> >>>> disk
> >>>>>>> I/O
> >>>>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
> >>>>>>>>>> situations
> >>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
> >>>> work.
> >>>>>>> It
> >>>>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
> >>>> freeze,
> >>>>>>> I
> >>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss.
> But
> >>>> if
> >>>>>>> it
> >>>>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
> >> release
> >>>>>>> with
> >>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >>>> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
> >>>>>>> branch
> >>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into
> master
> >>>> in
> >>>>>>> 48
> >>>>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
> >>>>>>> storage
> >>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
> >>>> limitation
> >>>>>>> of
> >>>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
> >>>> affect
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
> >>>> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
> >> and
> >>>>>>> limit
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
> >>>>>>> difference
> >>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
> >>>>>>> different
> >>>>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side
> you
> >>>>>>>>>> collect
> >>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you
> only
> >> go
> >>>>>>> for
> >>>>>>>>>> a
> >>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more
> expensive
> >>>>>>> then
> >>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an
> admin
> >>>>>>> would
> >>>>>>>>>> set
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
> >> point I
> >>>>>>>>>> think
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
> >>>> place
> >>>>>>> to
> >>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
> >>>>>>> different
> >>>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
> >>>> that
> >>>>>>> as
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
> >> this
> >>>>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
> >>>>>>>>>> disk_offering,
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
> >> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> >> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT
> execution
> >>>>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
> >>>>>>> integration)
> >>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check
> for
> >>>>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
> >>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
> >>>> expunge.
> >>>>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>>>>
> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >> http://solidfire.com/solution/overview/?video=play
> >>>>>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
I am never at a loss for an opinion. I some thoughts, but want to
confirm assumptions and ideas against the solidfire, disk_io_throttle,
and
object_store branches. I hope to collect them in a coherent form
tomorrow (5 June 2013).
Thanks,
-John
On Jun 4, 2013, at 5:29 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> "So, in essence, the SolidFire plugin introduces the notion of a managed
> iSCSI device and provisioned IOPS."
>
> Technically, the SolidFire plug-in just introduces the notion of
> provisioned storage IOPS.
>
> The storage framework that leverages the plug-in was incomplete, so I had
> to try to add in the notion of a managed iSCSI device.
>
> I appreciate all the time you've been spending on this. :) Do you have a
> recommendation as to how we should accomplish what you're looking for?
>
> Thanks!
>
>
> On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> So, in essence, the SolidFire plugin introduces the notion of a managed
>> iSCSI device and provisioned IOPS.
>>
>> I want to see a separation of the management capabilities (i.e. can a
>> device be managed/does an operator want it managed by CloudStack) from the
>> storage protocol. Ideally, we should end up with a semantic that will
>> allow any type of storage device to be managed. I also want to make
>> progress on decoupling the storage types from the hypervisor definitions.
>>
>> Thanks,
>> -John
>>
>> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> Hi John,
>>>
>>> No problem. Answers are below in red.
>>>
>>> Thanks!
>>>
>>>
>>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com> wrote:
>>>
>>>> Mike,
>>>>
>>>> Could you please answer the following questions for me with regards to
>> the
>>>> operation of the SolidFire plugin:
>>>>
>>>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>> Each SAN volume is equivalent to a single LUN (LUN 0).
>>> 1 SAN volume : 1 LUN
>>>
>>>> What is the cardinality between SAN Volumes and CloudStack Volumes?
>>> 1 SAN volume : 1 CloudStack volume
>>>
>>>> Are the LUN(s) created by the management server or externally by the
>>>> operator?
>>> When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) is
>>> created by the management server (via the plug-in) the first time the
>>> CloudStack volume is attached to a hypervisor.
>>>
>>> If you don't want to use the SolidFire plug-in, but still want to use a
>>> SolidFire volume (LUN), you can do this already today (prior to 4.2). The
>>> admin manually creates the SAN volume and - in this case - multiple VMs
>> and
>>> data disks can share this SAN volume. While you can do this today, it is
>>> not useful if you want to enforce storage QoS.
>>>
>>>> Are the SAN volumes by the management server or externally by the
>> operator?
>>> When the SolidFire plug-in is used, the SAN volumes are completely
>> managed
>>> by the management server (via the plug-in). There is no admin
>> interaction.
>>> This allows for a 1:1 mapping between a SAN volume and a CloudStack
>> volume,
>>> which is necessary for any storage vendor that supports true QoS.
>>>
>>>>
>>>> I would like to clarify how these pieces are related and expected to
>>>> operate.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>>> "In particular, how do we ensure that multiple VMs with provisioned
>> IOPS
>>>>> won't be cut off by the underlying storage."
>>>>>
>>>>> In the storage QoS world, we need to map a single SAN volume (LUN) to a
>>>>> single CloudStack volume. We cannot have multiple CloudStack volumes
>>>>> sharing a single SAN volume and still guarantee QoS.
>>>>>
>>>>> If the user wants to have a single SAN volume house more than one
>>>>> CloudStack volume, then can do that today without any of my plug-in
>> code.
>>>>>
>>>>>
>>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com
>>>>>> wrote:
>>>>>
>>>>>> "The administrator will allocate a SAN volume for CloudStack's use
>> onto
>>>>>> which CloudStack volumes will be created."
>>>>>>
>>>>>> I think we crossed e-mails. :)
>>>>>>
>>>>>> Check out my recent e-mail on this.
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> You are coming to the part which concerns me -- concepts from the
>>>>>>> hypervisor are leaking into storage layer.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> The weird part is that the iSCSI type is today only used (as far as
>> I
>>>>>>> know)
>>>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>>>>>>>>
>>>>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
>>>> type.
>>>>>>>>
>>>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>>>>>>> SharedMountPoint
>>>>>>>> type.
>>>>>>>>
>>>>>>>> So, I suppose mine and Edison's thinking here was to make a new type
>>>> of
>>>>>>>> storage to describe this dynamic ability Edison added into the
>> storage
>>>>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
>>>>>>> versus,
>>>>>>>> say, Dynamic_FC.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> "The storage device itself shouldn't know or care that it is being
>>>> used
>>>>>>>>> for a Xen SR -- simply be able to answer questions about it is
>>>>>>> storing."
>>>>>>>>>
>>>>>>>>> I see...so your concern here is that the SolidFire plug-in needs to
>>>>>>> call
>>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
>> treat
>>>>>>> it as
>>>>>>>>> such.
>>>>>>>>>
>>>>>>>>> I'm totally open to removing that constraint and just calling it
>>>> iSCSI
>>>>>>> or
>>>>>>>>> whatever. We would just need a way for the hypervisor attach logic
>> to
>>>>>>>>> detect this new requirement and perform the necessary activities.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> See my responses in-line.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I'm trying to picture this:
>>>>>>>>>>>
>>>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
>>>>>>> operator
>>>>>>>>>> may
>>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
>>>> shared
>>>>>>> by
>>>>>>>>>>> many services, and the operator has chosen to dedicate only a
>>>> portion
>>>>>>>>>> of it
>>>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>>>>>>>>>
>>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
>>>>>>> (before
>>>>>>>>>> an
>>>>>>>>>>> attach request to the hypervisor) would work unless the
>> hypervisor
>>>>>>>>>> consumes
>>>>>>>>>>> the SAN volume in the form of, say, an SR.
>>>>>>>>>>
>>>>>>>>>> My thinking is that, independent of CloudStack, an operator
>>>> allocates
>>>>>>> a
>>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
>>>>>>> simply
>>>>>>>>>> want to turn control of that LUN over to CloudStack, but not allow
>>>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> As the attach logic stands prior to my changes, we would be
>> passing
>>>>>>> in a
>>>>>>>>>>> SAN volume that does not have the necessary hypervisor support
>>>> (like
>>>>>>> an
>>>>>>>>>> SR)
>>>>>>>>>>> and the logic will fail.
>>>>>>>>>>>
>>>>>>>>>>> Are you thinking we should maybe have the storage framework
>> itself
>>>>>>>>>> detect
>>>>>>>>>>> that such a SAN volume needs support from the hypervisor side and
>>>>>>> have
>>>>>>>>>> it
>>>>>>>>>>> call into the agent code specifically to create the SR before the
>>>>>>> attach
>>>>>>>>>>> logic runs in the agent code?
>>>>>>>>>>
>>>>>>>>>> I think the hypervisor management plugin should have a rich enough
>>>>>>>>>> interface to storage to determine available for volume storage.
>> For
>>>>>>> Xen,
>>>>>>>>>> this interface would allow the interrogation of the device to
>>>>>>> determine the
>>>>>>>>>> SR is present. The storage device itself shouldn't know or care
>>>>>>> that it
>>>>>>>>>> is being used for a Xen SR -- simply be able to answer questions
>>>>>>> about it
>>>>>>>>>> is storing.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>>>
>>>>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
>>>>>>>>>> so-called
>>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows
>>>> up
>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>> applicable zone.
>>>>>>>>>>>>
>>>>>>>>>>>> * The admin creates a Disk Offering that references the storage
>>>> tag
>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>>>
>>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a new
>>>> row
>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>>>
>>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
>>>> disk).
>>>>>>>>>> This
>>>>>>>>>>>> leads to the storage framework calling the plug-in to create a
>> new
>>>>>>>>>> volume
>>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
>>>> updates
>>>>>>> the
>>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
>>>>>>>>>> volume).
>>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is in
>>>> the
>>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
>>>>>>> 'Ready'
>>>>>>>>>>>> state (even after a detach disk) and the plug-in code will not
>> be
>>>>>>>>>> called
>>>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>>>
>>>>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
>>>>>>> volume
>>>>>>>>>> to
>>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>>>> structure
>>>>>>>>>> (ex.
>>>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
>>>>>>>>>>>>> definitions, refers to the physical allocation on the device,
>>>>>>>>>> correct?"
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
>>>>>>> try to
>>>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What you say here may very well make sense, but I'm having a
>>>> hard
>>>>>>>>>> time
>>>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
>> was
>>>>>>> the
>>>>>>>>>>>>>> initial person to suggest the approach I took.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>>>> jburwell@basho.com
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It feels like we are combining two distinct concepts --
>> storage
>>>>>>>>>> device
>>>>>>>>>>>>>>> management and storage protocols. In both cases, we are
>>>>>>>>>> communicating with
>>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
>>>>>>> (Dynamic)
>>>>>>>>>> on the
>>>>>>>>>>>>>>> device while the other requires the volume to be volume to be
>>>>>>>>>> managed
>>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
>>>>>>> sync on
>>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
>>>> physical
>>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
>> able
>>>> to
>>>>>>>>>>>>>>> communicate with a storage device to move bits from one place
>>>> to
>>>>>>>>>> another,
>>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device
>> may
>>>> be
>>>>>>>>>> able to
>>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
>>>> iSCSI
>>>>>>>>>> device
>>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>>>> SolidFire
>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
>>>>>>> storing
>>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
>>>>>>> device,
>>>>>>>>>> an
>>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
>>>> the
>>>>>>>>>> device is
>>>>>>>>>>>>>>> shared by many services, and the operator has chosen to
>>>> dedicate
>>>>>>>>>> only a
>>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
>> appropriate
>>>>>>> to
>>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
>>>>>>> notion
>>>>>>>>>> of a
>>>>>>>>>>>>>>> storage device with an attached driver that indicates whether
>>>> or
>>>>>>>>>> not is
>>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>>>> representing a
>>>>>>>>>> physical
>>>>>>>>>>>>>>> allocation on a device separate that is associated with a
>>>> volume.
>>>>>>>>>> With
>>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
>>>>>>>>>> protocols they
>>>>>>>>>>>>>>> support and when they encounter a device managed by
>> CloudStack,
>>>>>>>>>> utilize the
>>>>>>>>>>>>>>> management operations exposed by the driver to automate
>>>>>>> allocation.
>>>>>>>>>> If
>>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
>>>>>>> drill
>>>>>>>>>> down to
>>>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
>> the
>>>>>>>>>> Dynamic
>>>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
>>>> Storage
>>>>>>>>>> based
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
>>>>>>> exist
>>>>>>>>>> on
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
>>>> volumes
>>>>>>> are
>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
>>>> Storage
>>>>>>>>>> based
>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
>>>>>>> system
>>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework). When
>> a
>>>>>>> user
>>>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
>>>> tells
>>>>>>>>>> the
>>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
>> attach
>>>>>>>>>> logic
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
>>>>>>> structure
>>>>>>>>>>>>>>> exists
>>>>>>>>>>>>>>>> at this point because the volume was just created. The
>>>>>>> hypervisor
>>>>>>>>>> data
>>>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> These are new terms, so I should probably have defined them
>>>> up
>>>>>>>>>> front
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>>>>>>>>>> creates a
>>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
>> created
>>>>>>> to
>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>>>>>>>>>> structure
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
>> this
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
>>>>>>> assumes
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
>>>>>>> attaches
>>>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead
>>>> of
>>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on
>> a
>>>>>>> SAN,
>>>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
>>>>>>> when an
>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>> volume is performed because these types of volumes have not
>>>>>>> been
>>>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
>> the
>>>>>>>>>> attach
>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
>> attaches
>>>>>>> the
>>>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
>>>> behavior
>>>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
>>>>>>> Another
>>>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
>>>>>>> storage
>>>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>>>>>>> XenServer
>>>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on
>> an
>>>>>>>>>> iSCSI
>>>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
>> hypervisor
>>>>>>> and
>>>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
>> CloudStack
>>>>>>> does
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
>> For
>>>>>>> all
>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
>>>> perform
>>>>>>>>>> this
>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference between
>>>> the
>>>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
>> overlay
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
>>>> up
>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants
>> to
>>>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
>>>>>>> would
>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>>>>>>> implement a
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>>>>>>> specific.
>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>>>>>>>>>> storage).
>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>>>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
>>>>>>> creation
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
>>>> him
>>>>>>> in
>>>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
>>>>>>> realized
>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless
>>>> the
>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
>>>> create
>>>>>>> the
>>>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
>>>>>>> Storage
>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see
>> the
>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
>> composed/driven
>>>>>>> by
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
>>>> per
>>>>>>>>>> their
>>>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
>>>> with a
>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>>>>>>> difficult to
>>>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>>>>>>>>>> XenServer,
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
>> then
>>>> go
>>>>>>>>>> into
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
>> volume
>>>>>>> (ex.
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
>>>> VMs
>>>>>>> and
>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need
>> no
>>>>>>>>>> longer
>>>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
>> between
>>>> a
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
>> storage
>>>>>>>>>> Quality
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>>>>>>> dynamically
>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
>>>> address
>>>>>>> is
>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
>> when
>>>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do
>> so.
>>>>>>> I've
>>>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
>>>> sandbox
>>>>>>>>>> and
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>>>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
>>>>>>> agent
>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
>> create a
>>>>>>>>>> volume
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>>>>>>> hypervisor,
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>>>>>>> invoked on
>>>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
>> create,
>>>>>>>>>> say, a
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>>>>>>> VMware) to
>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making
>>>> the
>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>>>>>>> hypervisors
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2
>> to
>>>>>>> get
>>>>>>>>>> KVM
>>>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>>>>>>> Depending on
>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working
>>>> for
>>>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards
>> to
>>>>>>> Disk
>>>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
>>>>>>> Damage
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
>> patch
>>>>>>> cycle
>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
>> MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
>>>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
>> non
>>>>>>> issue
>>>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
>> now,
>>>>>>>>>> users
>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>>>>>>> calculation.
>>>>>>>>>> Of
>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>>>>>>> individual
>>>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work
>> on
>>>> it
>>>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
>> features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying
>> to
>>>>>>>>>> adding
>>>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>>>>>>> statistics
>>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
>> you
>>>>>>> have
>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>>>>>>>>>> read/write
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
>>>> more
>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>>>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
>> imagine
>>>>>>> our
>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could
>> an
>>>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>>>>>>> supports
>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
>>>> through
>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>>>>>>> parameters
>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>>>> apologize
>>>>>>>>>> for
>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
>> with a
>>>>>>> set
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>>>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
>> better
>>>> to
>>>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider
>>>> the
>>>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>>>> response
>>>>>>> to
>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
>>>> think
>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but
>> I
>>>>>>> think
>>>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
>>>> have
>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
>>>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
>> policies
>>>> be
>>>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
>> purposes,
>>>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model
>>>> for
>>>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled?
>> In
>>>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
>> IOPS
>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>>>> deliver.
>>>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
>>>>>>> device
>>>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>>>> controlling
>>>>>>> the
>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
>>>> from
>>>>>>> the
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
>> Burwell
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
>>>>>>> merged
>>>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready
>> to
>>>>>>> turn
>>>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue
>>>> is
>>>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>>>> intersect/compliment
>>>>>>> the
>>>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
>> As I
>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
>>>> like
>>>>>>> to
>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
>> wido@widodh.nl
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
>> reason
>>>>>>> till
>>>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
>>>> mention
>>>>>>>>>> again
>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
>>>>>>> never
>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>>>> throttling
>>>>>>>>>> and
>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from
>>>> day
>>>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
>>>>>>> support
>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
>>>> because
>>>>>>> I
>>>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>>>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>>>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
>>>> now
>>>>>>> so
>>>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
>> before
>>>>>>> the
>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
>> instead
>>>>>>> of
>>>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
>> freeze
>>>>>>> date.
>>>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>>>> necessary
>>>>>>>>>> to do
>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>>>>>>>>>> association
>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>>>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
>> Finally I
>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
>> because
>>>>>>> the
>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
>>>>>>> bytes as
>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
>>>> disk
>>>>>>> I/O
>>>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>>>>>>>>>> situations
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
>>>> work.
>>>>>>> It
>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
>>>> freeze,
>>>>>>> I
>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But
>>>> if
>>>>>>> it
>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
>> release
>>>>>>> with
>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>>>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
>>>>>>> branch
>>>>>>>>>> into
>>>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master
>>>> in
>>>>>>> 48
>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
>>>>>>> storage
>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>>>> limitation
>>>>>>> of
>>>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
>>>> affect
>>>>>>>>>> the
>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
>>>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
>> and
>>>>>>> limit
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>>>>>>> difference
>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>>>>>>> different
>>>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>>>>>>>>>> collect
>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only
>> go
>>>>>>> for
>>>>>>>>>> a
>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
>>>>>>> then
>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
>>>>>>> would
>>>>>>>>>> set
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
>> point I
>>>>>>>>>> think
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
>>>> place
>>>>>>> to
>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>>>>>>> different
>>>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
>>>> that
>>>>>>> as
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
>> this
>>>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>>>>>>>>> disk_offering,
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
>> iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
>> https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>>>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>>>>>>> integration)
>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>>>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
>>>> expunge.
>>>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
>> Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>>>>>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"So, in essence, the SolidFire plugin introduces the notion of a managed
iSCSI device and provisioned IOPS."
Technically, the SolidFire plug-in just introduces the notion of
provisioned storage IOPS.
The storage framework that leverages the plug-in was incomplete, so I had
to try to add in the notion of a managed iSCSI device.
I appreciate all the time you've been spending on this. :) Do you have a
recommendation as to how we should accomplish what you're looking for?
Thanks!
On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> So, in essence, the SolidFire plugin introduces the notion of a managed
> iSCSI device and provisioned IOPS.
>
> I want to see a separation of the management capabilities (i.e. can a
> device be managed/does an operator want it managed by CloudStack) from the
> storage protocol. Ideally, we should end up with a semantic that will
> allow any type of storage device to be managed. I also want to make
> progress on decoupling the storage types from the hypervisor definitions.
>
> Thanks,
> -John
>
> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hi John,
> >
> > No problem. Answers are below in red.
> >
> > Thanks!
> >
> >
> > On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com> wrote:
> >
> >> Mike,
> >>
> >> Could you please answer the following questions for me with regards to
> the
> >> operation of the SolidFire plugin:
> >>
> >> What is the cardinality between iSCSI LUNs and SAN volumes?
> >>
> > Each SAN volume is equivalent to a single LUN (LUN 0).
> > 1 SAN volume : 1 LUN
> >
> >> What is the cardinality between SAN Volumes and CloudStack Volumes?
> >>
> > 1 SAN volume : 1 CloudStack volume
> >
> >> Are the LUN(s) created by the management server or externally by the
> >> operator?
> >>
> > When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) is
> > created by the management server (via the plug-in) the first time the
> > CloudStack volume is attached to a hypervisor.
> >
> > If you don't want to use the SolidFire plug-in, but still want to use a
> > SolidFire volume (LUN), you can do this already today (prior to 4.2). The
> > admin manually creates the SAN volume and - in this case - multiple VMs
> and
> > data disks can share this SAN volume. While you can do this today, it is
> > not useful if you want to enforce storage QoS.
> >
> >> Are the SAN volumes by the management server or externally by the
> operator?
> >>
> > When the SolidFire plug-in is used, the SAN volumes are completely
> managed
> > by the management server (via the plug-in). There is no admin
> interaction.
> > This allows for a 1:1 mapping between a SAN volume and a CloudStack
> volume,
> > which is necessary for any storage vendor that supports true QoS.
> >
> >>
> >> I would like to clarify how these pieces are related and expected to
> >> operate.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> "In particular, how do we ensure that multiple VMs with provisioned
> IOPS
> >>> won't be cut off by the underlying storage."
> >>>
> >>> In the storage QoS world, we need to map a single SAN volume (LUN) to a
> >>> single CloudStack volume. We cannot have multiple CloudStack volumes
> >>> sharing a single SAN volume and still guarantee QoS.
> >>>
> >>> If the user wants to have a single SAN volume house more than one
> >>> CloudStack volume, then can do that today without any of my plug-in
> code.
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com
> >>>> wrote:
> >>>
> >>>> "The administrator will allocate a SAN volume for CloudStack's use
> onto
> >>>> which CloudStack volumes will be created."
> >>>>
> >>>> I think we crossed e-mails. :)
> >>>>
> >>>> Check out my recent e-mail on this.
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> You are coming to the part which concerns me -- concepts from the
> >>>>> hypervisor are leaking into storage layer.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>>> The weird part is that the iSCSI type is today only used (as far as
> I
> >>>>> know)
> >>>>>> in regards to XenServer (when you have not PreSetup an SR).
> >>>>>>
> >>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
> >> type.
> >>>>>>
> >>>>>> If you want to use your iSCSI volume from KVM, it uses the
> >>>>> SharedMountPoint
> >>>>>> type.
> >>>>>>
> >>>>>> So, I suppose mine and Edison's thinking here was to make a new type
> >> of
> >>>>>> storage to describe this dynamic ability Edison added into the
> storage
> >>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
> >>>>> versus,
> >>>>>> say, Dynamic_FC.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> "The storage device itself shouldn't know or care that it is being
> >> used
> >>>>>>> for a Xen SR -- simply be able to answer questions about it is
> >>>>> storing."
> >>>>>>>
> >>>>>>> I see...so your concern here is that the SolidFire plug-in needs to
> >>>>> call
> >>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
> treat
> >>>>> it as
> >>>>>>> such.
> >>>>>>>
> >>>>>>> I'm totally open to removing that constraint and just calling it
> >> iSCSI
> >>>>> or
> >>>>>>> whatever. We would just need a way for the hypervisor attach logic
> to
> >>>>>>> detect this new requirement and perform the necessary activities.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> See my responses in-line.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I'm trying to picture this:
> >>>>>>>>>
> >>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
> >>>>> operator
> >>>>>>>> may
> >>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
> >> shared
> >>>>> by
> >>>>>>>>> many services, and the operator has chosen to dedicate only a
> >> portion
> >>>>>>>> of it
> >>>>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>>>>>>
> >>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
> >>>>> (before
> >>>>>>>> an
> >>>>>>>>> attach request to the hypervisor) would work unless the
> hypervisor
> >>>>>>>> consumes
> >>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>>>
> >>>>>>>> My thinking is that, independent of CloudStack, an operator
> >> allocates
> >>>>> a
> >>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
> >>>>> simply
> >>>>>>>> want to turn control of that LUN over to CloudStack, but not allow
> >>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> As the attach logic stands prior to my changes, we would be
> passing
> >>>>> in a
> >>>>>>>>> SAN volume that does not have the necessary hypervisor support
> >> (like
> >>>>> an
> >>>>>>>> SR)
> >>>>>>>>> and the logic will fail.
> >>>>>>>>>
> >>>>>>>>> Are you thinking we should maybe have the storage framework
> itself
> >>>>>>>> detect
> >>>>>>>>> that such a SAN volume needs support from the hypervisor side and
> >>>>> have
> >>>>>>>> it
> >>>>>>>>> call into the agent code specifically to create the SR before the
> >>>>> attach
> >>>>>>>>> logic runs in the agent code?
> >>>>>>>>
> >>>>>>>> I think the hypervisor management plugin should have a rich enough
> >>>>>>>> interface to storage to determine available for volume storage.
> For
> >>>>> Xen,
> >>>>>>>> this interface would allow the interrogation of the device to
> >>>>> determine the
> >>>>>>>> SR is present. The storage device itself shouldn't know or care
> >>>>> that it
> >>>>>>>> is being used for a Xen SR -- simply be able to answer questions
> >>>>> about it
> >>>>>>>> is storing.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>
> >>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
> >>>>>>>> so-called
> >>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows
> >> up
> >>>>> in
> >>>>>>>> the
> >>>>>>>>>> applicable zone.
> >>>>>>>>>>
> >>>>>>>>>> * The admin creates a Disk Offering that references the storage
> >> tag
> >>>>> of
> >>>>>>>> the
> >>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>
> >>>>>>>>>> * The end user creates a CloudStack volume. This leads to a new
> >> row
> >>>>> in
> >>>>>>>> the
> >>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>
> >>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
> >> disk).
> >>>>>>>> This
> >>>>>>>>>> leads to the storage framework calling the plug-in to create a
> new
> >>>>>>>> volume
> >>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
> >> updates
> >>>>> the
> >>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
> >>>>>>>> volume).
> >>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is in
> >> the
> >>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
> >>>>> 'Ready'
> >>>>>>>>>> state (even after a detach disk) and the plug-in code will not
> be
> >>>>>>>> called
> >>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>
> >>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
> >>>>> volume
> >>>>>>>> to
> >>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
> >> structure
> >>>>>>>> (ex.
> >>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
> >>>>>>>>>>> definitions, refers to the physical allocation on the device,
> >>>>>>>> correct?"
> >>>>>>>>>>>
> >>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>>>>>>>>
> >>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
> >>>>> try to
> >>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>
> >>>>>>>>>>>> What you say here may very well make sense, but I'm having a
> >> hard
> >>>>>>>> time
> >>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
> was
> >>>>> the
> >>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>>>
> >>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >> jburwell@basho.com
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It feels like we are combining two distinct concepts --
> storage
> >>>>>>>> device
> >>>>>>>>>>>>> management and storage protocols. In both cases, we are
> >>>>>>>> communicating with
> >>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
> >>>>> (Dynamic)
> >>>>>>>> on the
> >>>>>>>>>>>>> device while the other requires the volume to be volume to be
> >>>>>>>> managed
> >>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
> >>>>> sync on
> >>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
> >> physical
> >>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be
> able
> >> to
> >>>>>>>>>>>>> communicate with a storage device to move bits from one place
> >> to
> >>>>>>>> another,
> >>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device
> may
> >> be
> >>>>>>>> able to
> >>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
> >> iSCSI
> >>>>>>>> device
> >>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
> >> SolidFire
> >>>>>>>> iSCSI
> >>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
> >>>>> storing
> >>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
> >>>>> device,
> >>>>>>>> an
> >>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
> >> the
> >>>>>>>> device is
> >>>>>>>>>>>>> shared by many services, and the operator has chosen to
> >> dedicate
> >>>>>>>> only a
> >>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
> appropriate
> >>>>> to
> >>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
> >>>>> notion
> >>>>>>>> of a
> >>>>>>>>>>>>> storage device with an attached driver that indicates whether
> >> or
> >>>>>>>> not is
> >>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
> >> representing a
> >>>>>>>> physical
> >>>>>>>>>>>>> allocation on a device separate that is associated with a
> >> volume.
> >>>>>>>> With
> >>>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
> >>>>>>>> protocols they
> >>>>>>>>>>>>> support and when they encounter a device managed by
> CloudStack,
> >>>>>>>> utilize the
> >>>>>>>>>>>>> management operations exposed by the driver to automate
> >>>>> allocation.
> >>>>>>>> If
> >>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
> >>>>> drill
> >>>>>>>> down to
> >>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
> the
> >>>>>>>> Dynamic
> >>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
> >> Storage
> >>>>>>>> based
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
> >>>>> exist
> >>>>>>>> on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
> >> volumes
> >>>>> are
> >>>>>>>>>>>>> created
> >>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
> >> Storage
> >>>>>>>> based
> >>>>>>>>>>>>> on a
> >>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
> >>>>> system
> >>>>>>>>>>>>>> dynamically (as is enabled via the storage framework). When
> a
> >>>>> user
> >>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
> >> tells
> >>>>>>>> the
> >>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
> attach
> >>>>>>>> logic
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
> >>>>> structure
> >>>>>>>>>>>>> exists
> >>>>>>>>>>>>>> at this point because the volume was just created. The
> >>>>> hypervisor
> >>>>>>>> data
> >>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> These are new terms, so I should probably have defined them
> >> up
> >>>>>>>> front
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
> >>>>>>>> creates a
> >>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
> created
> >>>>> to
> >>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
> >>>>>>>> structure
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
> this
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
> >>>>> assumes
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
> >>>>> attaches
> >>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead
> >> of
> >>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on
> a
> >>>>> SAN,
> >>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
> >>>>> when an
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>> volume is performed because these types of volumes have not
> >>>>> been
> >>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
> the
> >>>>>>>> attach
> >>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
> attaches
> >>>>> the
> >>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
> >> behavior
> >>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
> >>>>> Another
> >>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
> >>>>> storage
> >>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
> >>>>> XenServer
> >>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on
> an
> >>>>>>>> iSCSI
> >>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
> hypervisor
> >>>>> and
> >>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
> CloudStack
> >>>>> does
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
> For
> >>>>> all
> >>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
> >> perform
> >>>>>>>> this
> >>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Reading through the code, what is the difference between
> >> the
> >>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs
> overlay
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
> >> up
> >>>>>>>> by
> >>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
> >>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants
> to
> >>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
> >>>>> would
> >>>>>>>> be
> >>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
> >>>>> implement a
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
> >>>>> specific.
> >>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
> >>>>>>>> zone-wide
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
> >>>>>>>> storage).
> >>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
> >>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
> >>>>> creation
> >>>>>>>> and
> >>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
> >> him
> >>>>> in
> >>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
> >>>>> realized
> >>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless
> >> the
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
> >> create
> >>>>> the
> >>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
> >>>>> Storage
> >>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see
> the
> >>>>>>>> Storage
> >>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> composed/driven
> >>>>> by
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
> >> per
> >>>>>>>> their
> >>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
> >> with a
> >>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
> >>>>> difficult to
> >>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
> >>>>>>>> SolidFire
> >>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
> >>>>>>>> XenServer,
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
> then
> >> go
> >>>>>>>> into
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
> volume
> >>>>> (ex.
> >>>>>>>> a
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
> >> VMs
> >>>>> and
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need
> no
> >>>>>>>> longer
> >>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
> between
> >> a
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
> storage
> >>>>>>>> Quality
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
> >>>>> dynamically
> >>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
> >> address
> >>>>> is
> >>>>>>>> in
> >>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
> when
> >>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do
> so.
> >>>>> I've
> >>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
> >> sandbox
> >>>>>>>> and
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
> >>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
> >>>>> agent
> >>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
> create a
> >>>>>>>> volume
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
> >>>>> hypervisor,
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
> >>>>> invoked on
> >>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
> create,
> >>>>>>>> say, a
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
> >>>>> VMware) to
> >>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making
> >> the
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
> >>>>> hypervisor
> >>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
> >>>>> hypervisors
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2
> to
> >>>>> get
> >>>>>>>> KVM
> >>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
> >>>>> Depending on
> >>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working
> >> for
> >>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards
> to
> >>>>> Disk
> >>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
> >>>>> Damage
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
> patch
> >>>>> cycle
> >>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
> >> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
> non
> >>>>> issue
> >>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
> now,
> >>>>>>>> users
> >>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
> >>>>> calculation.
> >>>>>>>> Of
> >>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
> >>>>> individual
> >>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work
> on
> >> it
> >>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying
> to
> >>>>>>>> adding
> >>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
> >>>>> statistics
> >>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
> you
> >>>>> have
> >>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
> >>>>>>>> read/write
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
> >> more
> >>>>>>>> for
> >>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
> >>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would
> imagine
> >>>>> our
> >>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could
> an
> >>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
> >>>>> supports
> >>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
> >> through
> >>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
> >>>>> parameters
> >>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
> >> apologize
> >>>>>>>> for
> >>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
> with a
> >>>>> set
> >>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
> >>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
> better
> >> to
> >>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider
> >> the
> >>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
> >> response
> >>>>> to
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
> >> think
> >>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but
> I
> >>>>> think
> >>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
> >> have
> >>>>>>>> have
> >>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
> >> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
> >>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
> policies
> >> be
> >>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
> purposes,
> >>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model
> >> for
> >>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> >>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled?
> In
> >>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
> IOPS
> >>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
> >> deliver.
> >>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
> >>>>>>>> storage
> >>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
> >>>>> device
> >>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
> >> controlling
> >>>>> the
> >>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
> >> from
> >>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
> Burwell
> >> <
> >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
> >>>>> merged
> >>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready
> to
> >>>>> turn
> >>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue
> >> is
> >>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
> >>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >> intersect/compliment
> >>>>> the
> >>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
> As I
> >>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
> >> like
> >>>>> to
> >>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
> >>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
> wido@widodh.nl
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
> >>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
> reason
> >>>>> till
> >>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
> >> mention
> >>>>>>>> again
> >>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
> >>>>> never
> >>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
> >> throttling
> >>>>>>>> and
> >>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from
> >> day
> >>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
> >>>>> support
> >>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
> >> because
> >>>>> I
> >>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
> >>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
> >>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
> >> now
> >>>>> so
> >>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
> before
> >>>>> the
> >>>>>>>> 4.2
> >>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
> instead
> >>>>> of
> >>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
> freeze
> >>>>> date.
> >>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
> >> necessary
> >>>>>>>> to do
> >>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> >>>>>>>> association
> >>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
> >>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning.
> Finally I
> >>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
> because
> >>>>> the
> >>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
> >>>>> bytes as
> >>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
> >> disk
> >>>>> I/O
> >>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
> >>>>>>>> situations
> >>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
> >> work.
> >>>>> It
> >>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
> >> freeze,
> >>>>> I
> >>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But
> >> if
> >>>>> it
> >>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
> release
> >>>>> with
> >>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
> >>>>> branch
> >>>>>>>> into
> >>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master
> >> in
> >>>>> 48
> >>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
> >>>>> storage
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
> >> limitation
> >>>>> of
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
> >> affect
> >>>>>>>> the
> >>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
> >> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
> and
> >>>>> limit
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
> >>>>> difference
> >>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
> >>>>> different
> >>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
> >>>>>>>> collect
> >>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only
> go
> >>>>> for
> >>>>>>>> a
> >>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
> >>>>> then
> >>>>>>>> Read
> >>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
> >>>>> would
> >>>>>>>> set
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
> point I
> >>>>>>>> think
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
> >> place
> >>>>> to
> >>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
> >>>>> different
> >>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
> >> that
> >>>>> as
> >>>>>>>> the
> >>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
> this
> >>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
> >>>>>>>> disk_offering,
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
> >>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
> >>>>> integration)
> >>>>>>>> is
> >>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
> >>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
> >>>>>>>> CloudStack
> >>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
> >> expunge.
> >>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
So, in essence, the SolidFire plugin introduces the notion of a managed iSCSI device and provisioned IOPS.
I want to see a separation of the management capabilities (i.e. can a device be managed/does an operator want it managed by CloudStack) from the storage protocol. Ideally, we should end up with a semantic that will allow any type of storage device to be managed. I also want to make progress on decoupling the storage types from the hypervisor definitions.
Thanks,
-John
On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Hi John,
>
> No problem. Answers are below in red.
>
> Thanks!
>
>
> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Could you please answer the following questions for me with regards to the
>> operation of the SolidFire plugin:
>>
>> What is the cardinality between iSCSI LUNs and SAN volumes?
>>
> Each SAN volume is equivalent to a single LUN (LUN 0).
> 1 SAN volume : 1 LUN
>
>> What is the cardinality between SAN Volumes and CloudStack Volumes?
>>
> 1 SAN volume : 1 CloudStack volume
>
>> Are the LUN(s) created by the management server or externally by the
>> operator?
>>
> When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) is
> created by the management server (via the plug-in) the first time the
> CloudStack volume is attached to a hypervisor.
>
> If you don't want to use the SolidFire plug-in, but still want to use a
> SolidFire volume (LUN), you can do this already today (prior to 4.2). The
> admin manually creates the SAN volume and - in this case - multiple VMs and
> data disks can share this SAN volume. While you can do this today, it is
> not useful if you want to enforce storage QoS.
>
>> Are the SAN volumes by the management server or externally by the operator?
>>
> When the SolidFire plug-in is used, the SAN volumes are completely managed
> by the management server (via the plug-in). There is no admin interaction.
> This allows for a 1:1 mapping between a SAN volume and a CloudStack volume,
> which is necessary for any storage vendor that supports true QoS.
>
>>
>> I would like to clarify how these pieces are related and expected to
>> operate.
>>
>> Thanks,
>> -John
>>
>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> "In particular, how do we ensure that multiple VMs with provisioned IOPS
>>> won't be cut off by the underlying storage."
>>>
>>> In the storage QoS world, we need to map a single SAN volume (LUN) to a
>>> single CloudStack volume. We cannot have multiple CloudStack volumes
>>> sharing a single SAN volume and still guarantee QoS.
>>>
>>> If the user wants to have a single SAN volume house more than one
>>> CloudStack volume, then can do that today without any of my plug-in code.
>>>
>>>
>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>>>> wrote:
>>>
>>>> "The administrator will allocate a SAN volume for CloudStack's use onto
>>>> which CloudStack volumes will be created."
>>>>
>>>> I think we crossed e-mails. :)
>>>>
>>>> Check out my recent e-mail on this.
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> You are coming to the part which concerns me -- concepts from the
>>>>> hypervisor are leaking into storage layer.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>
>>>>>> The weird part is that the iSCSI type is today only used (as far as I
>>>>> know)
>>>>>> in regards to XenServer (when you have not PreSetup an SR).
>>>>>>
>>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
>> type.
>>>>>>
>>>>>> If you want to use your iSCSI volume from KVM, it uses the
>>>>> SharedMountPoint
>>>>>> type.
>>>>>>
>>>>>> So, I suppose mine and Edison's thinking here was to make a new type
>> of
>>>>>> storage to describe this dynamic ability Edison added into the storage
>>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
>>>>> versus,
>>>>>> say, Dynamic_FC.
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com
>>>>>>> wrote:
>>>>>>
>>>>>>> "The storage device itself shouldn't know or care that it is being
>> used
>>>>>>> for a Xen SR -- simply be able to answer questions about it is
>>>>> storing."
>>>>>>>
>>>>>>> I see...so your concern here is that the SolidFire plug-in needs to
>>>>> call
>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to treat
>>>>> it as
>>>>>>> such.
>>>>>>>
>>>>>>> I'm totally open to removing that constraint and just calling it
>> iSCSI
>>>>> or
>>>>>>> whatever. We would just need a way for the hypervisor attach logic to
>>>>>>> detect this new requirement and perform the necessary activities.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> See my responses in-line.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I'm trying to picture this:
>>>>>>>>>
>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
>>>>> operator
>>>>>>>> may
>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
>> shared
>>>>> by
>>>>>>>>> many services, and the operator has chosen to dedicate only a
>> portion
>>>>>>>> of it
>>>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>>>>>>>
>>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
>>>>> (before
>>>>>>>> an
>>>>>>>>> attach request to the hypervisor) would work unless the hypervisor
>>>>>>>> consumes
>>>>>>>>> the SAN volume in the form of, say, an SR.
>>>>>>>>
>>>>>>>> My thinking is that, independent of CloudStack, an operator
>> allocates
>>>>> a
>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
>>>>> simply
>>>>>>>> want to turn control of that LUN over to CloudStack, but not allow
>>>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> As the attach logic stands prior to my changes, we would be passing
>>>>> in a
>>>>>>>>> SAN volume that does not have the necessary hypervisor support
>> (like
>>>>> an
>>>>>>>> SR)
>>>>>>>>> and the logic will fail.
>>>>>>>>>
>>>>>>>>> Are you thinking we should maybe have the storage framework itself
>>>>>>>> detect
>>>>>>>>> that such a SAN volume needs support from the hypervisor side and
>>>>> have
>>>>>>>> it
>>>>>>>>> call into the agent code specifically to create the SR before the
>>>>> attach
>>>>>>>>> logic runs in the agent code?
>>>>>>>>
>>>>>>>> I think the hypervisor management plugin should have a rich enough
>>>>>>>> interface to storage to determine available for volume storage. For
>>>>> Xen,
>>>>>>>> this interface would allow the interrogation of the device to
>>>>> determine the
>>>>>>>> SR is present. The storage device itself shouldn't know or care
>>>>> that it
>>>>>>>> is being used for a Xen SR -- simply be able to answer questions
>>>>> about it
>>>>>>>> is storing.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> So, the flow is as follows:
>>>>>>>>>>
>>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
>>>>>>>> so-called
>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows
>> up
>>>>> in
>>>>>>>> the
>>>>>>>>>> applicable zone.
>>>>>>>>>>
>>>>>>>>>> * The admin creates a Disk Offering that references the storage
>> tag
>>>>> of
>>>>>>>> the
>>>>>>>>>> newly created Primary Storage.
>>>>>>>>>>
>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a new
>> row
>>>>> in
>>>>>>>> the
>>>>>>>>>> cloud.volumes table.
>>>>>>>>>>
>>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
>> disk).
>>>>>>>> This
>>>>>>>>>> leads to the storage framework calling the plug-in to create a new
>>>>>>>> volume
>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
>> updates
>>>>> the
>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
>>>>>>>> volume).
>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is in
>> the
>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
>>>>> 'Ready'
>>>>>>>>>> state (even after a detach disk) and the plug-in code will not be
>>>>>>>> called
>>>>>>>>>> again to create this SAN volume.
>>>>>>>>>>
>>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
>>>>> volume
>>>>>>>> to
>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
>> structure
>>>>>>>> (ex.
>>>>>>>>>> an SR on XenServer).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
>>>>>>>>>>> definitions, refers to the physical allocation on the device,
>>>>>>>> correct?"
>>>>>>>>>>>
>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>>>>>>>>
>>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
>>>>> try to
>>>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>
>>>>>>>>>>>> What you say here may very well make sense, but I'm having a
>> hard
>>>>>>>> time
>>>>>>>>>>>> envisioning it.
>>>>>>>>>>>>
>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he was
>>>>> the
>>>>>>>>>>>> initial person to suggest the approach I took.
>>>>>>>>>>>>
>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
>> jburwell@basho.com
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> It feels like we are combining two distinct concepts -- storage
>>>>>>>> device
>>>>>>>>>>>>> management and storage protocols. In both cases, we are
>>>>>>>> communicating with
>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
>>>>> (Dynamic)
>>>>>>>> on the
>>>>>>>>>>>>> device while the other requires the volume to be volume to be
>>>>>>>> managed
>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
>>>>> sync on
>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
>> physical
>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be able
>> to
>>>>>>>>>>>>> communicate with a storage device to move bits from one place
>> to
>>>>>>>> another,
>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device may
>> be
>>>>>>>> able to
>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
>> iSCSI
>>>>>>>> device
>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
>> SolidFire
>>>>>>>> iSCSI
>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
>>>>> storing
>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
>>>>> device,
>>>>>>>> an
>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
>> the
>>>>>>>> device is
>>>>>>>>>>>>> shared by many services, and the operator has chosen to
>> dedicate
>>>>>>>> only a
>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems appropriate
>>>>> to
>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
>>>>> notion
>>>>>>>> of a
>>>>>>>>>>>>> storage device with an attached driver that indicates whether
>> or
>>>>>>>> not is
>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
>> representing a
>>>>>>>> physical
>>>>>>>>>>>>> allocation on a device separate that is associated with a
>> volume.
>>>>>>>> With
>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
>>>>>>>> protocols they
>>>>>>>>>>>>> support and when they encounter a device managed by CloudStack,
>>>>>>>> utilize the
>>>>>>>>>>>>> management operations exposed by the driver to automate
>>>>> allocation.
>>>>>>>> If
>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
>>>>> drill
>>>>>>>> down to
>>>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Here is the difference between the current iSCSI type and the
>>>>>>>> Dynamic
>>>>>>>>>>>>> type:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
>> Storage
>>>>>>>> based
>>>>>>>>>>>>> on
>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
>>>>> exist
>>>>>>>> on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
>> volumes
>>>>> are
>>>>>>>>>>>>> created
>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
>> Storage
>>>>>>>> based
>>>>>>>>>>>>> on a
>>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
>>>>> system
>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework). When a
>>>>> user
>>>>>>>>>>>>> wants to
>>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
>> tells
>>>>>>>> the
>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the attach
>>>>>>>> logic
>>>>>>>>>>>>> for
>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
>>>>> structure
>>>>>>>>>>>>> exists
>>>>>>>>>>>>>> at this point because the volume was just created. The
>>>>> hypervisor
>>>>>>>> data
>>>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> These are new terms, so I should probably have defined them
>> up
>>>>>>>> front
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>>>>>>>> creates a
>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is created
>>>>> to
>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>>>>>>>> structure
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
>>>>> assumes
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
>>>>> attaches
>>>>>>>>>>>>> the VDI
>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead
>> of
>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on a
>>>>> SAN,
>>>>>>>>>>>>> but the
>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
>>>>> when an
>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>> volume is performed because these types of volumes have not
>>>>> been
>>>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once the
>>>>>>>> attach
>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it attaches
>>>>> the
>>>>>>>>>>>>> one and
>>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
>> behavior
>>>>>>>>>>>>> works in
>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
>>>>> Another
>>>>>>>>>>>>> way to
>>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
>>>>> storage
>>>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>>>>> XenServer
>>>>>>>>>>>>> when you
>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on an
>>>>>>>> iSCSI
>>>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the hypervisor
>>>>> and
>>>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as CloudStack
>>>>> does
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer. For
>>>>> all
>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
>> perform
>>>>>>>> this
>>>>>>>>>>>>> step
>>>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference between
>> the
>>>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
>> up
>>>>>>>> by
>>>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants to
>>>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
>>>>> would
>>>>>>>> be
>>>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>>>>> implement a
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>>>>> specific.
>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>>>>>>>> zone-wide
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>>>>>>>> storage).
>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>>>>>>>> changes.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
>>>>> creation
>>>>>>>> and
>>>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
>> him
>>>>> in
>>>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
>>>>> realized
>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless
>> the
>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
>> create
>>>>> the
>>>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
>>>>> Storage
>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see the
>>>>>>>> Storage
>>>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be composed/driven
>>>>> by
>>>>>>>> the
>>>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
>> per
>>>>>>>> their
>>>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
>> with a
>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>>>>> difficult to
>>>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>>>>>>>> SolidFire
>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>>>>>>>> XenServer,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system, then
>> go
>>>>>>>> into
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the volume
>>>>> (ex.
>>>>>>>> a
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
>> VMs
>>>>> and
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
>>>>>>>> longer
>>>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between
>> a
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
>>>>>>>> Quality
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>>>>> dynamically
>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
>> address
>>>>> is
>>>>>>>> in
>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>>>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so.
>>>>> I've
>>>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
>> sandbox
>>>>>>>> and
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>>>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
>>>>> agent
>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
>>>>>>>> volume
>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>>>>> hypervisor,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>>>>> invoked on
>>>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and create,
>>>>>>>> say, a
>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>>>>> VMware) to
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making
>> the
>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>>>>> hypervisor
>>>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>>>>> hypervisors
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to
>>>>> get
>>>>>>>> KVM
>>>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>>>>> Depending on
>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working
>> for
>>>>>>>>>>>>> VMware in
>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to
>>>>> Disk
>>>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
>>>>> Damage
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch
>>>>> cycle
>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
>> mine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non
>>>>> issue
>>>>>>>>>>>>> as I
>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
>>>>>>>> users
>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>>>>>>>>>>> service_offering
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>>>>> calculation.
>>>>>>>> Of
>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>>>>> individual
>>>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on
>> it
>>>>>>>>>>>>> later. Do
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
>>>>>>>> adding
>>>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>>>>> statistics
>>>>>>>>>>>>> rather
>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you
>>>>> have
>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>>>>>>>> read/write
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
>> more
>>>>>>>> for
>>>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>>>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine
>>>>> our
>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>>>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>>>>> supports
>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
>> through
>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>>>>> parameters
>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
>> apologize
>>>>>>>> for
>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a
>>>>> set
>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>>>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better
>> to
>>>>>>>>>>>>> commit
>>>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider
>> the
>>>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
>> response
>>>>> to
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
>> think
>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I
>>>>> think
>>>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
>> have
>>>>>>>> have
>>>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
>> issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>>>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies
>> be
>>>>>>>>>>>>> captured
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model
>> for
>>>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>>>>>>>>>>>> particular,
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
>> deliver.
>>>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>>>>>>>> storage
>>>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
>>>>> device
>>>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
>> controlling
>>>>> the
>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
>> from
>>>>> the
>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell
>> <
>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
>>>>> merged
>>>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to
>>>>> turn
>>>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue
>> is
>>>>>>>>>>>>> addressed
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
>> intersect/compliment
>>>>> the
>>>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
>> like
>>>>> to
>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wido@widodh.nl
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason
>>>>> till
>>>>>>>>>>>>> Monday,
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
>> mention
>>>>>>>> again
>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
>>>>> never
>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
>> throttling
>>>>>>>> and
>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from
>> day
>>>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
>>>>> support
>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
>> because
>>>>> I
>>>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>>>>>>>> Therefore, we
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>>>>> backwards
>>>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
>> now
>>>>> so
>>>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before
>>>>> the
>>>>>>>> 4.2
>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead
>>>>> of
>>>>>>>>>>>>> Total
>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze
>>>>> date.
>>>>>>>>>>>>> But if
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
>> necessary
>>>>>>>> to do
>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>>>>>>>> association
>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>>>>> Hollander <
>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because
>>>>> the
>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
>>>>> bytes as
>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
>> disk
>>>>> I/O
>>>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>>>>>>>> situations
>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
>> work.
>>>>> It
>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
>> freeze,
>>>>> I
>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But
>> if
>>>>> it
>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release
>>>>> with
>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
>> wido@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
>>>>> branch
>>>>>>>> into
>>>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master
>> in
>>>>> 48
>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
>>>>> storage
>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
>> limitation
>>>>> of
>>>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
>> affect
>>>>>>>> the
>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
>> device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and
>>>>> limit
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>>>>> difference
>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>>>>> different
>>>>>>>>>>>>> rate
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>>>>>>>> collect
>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go
>>>>> for
>>>>>>>> a
>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
>>>>> then
>>>>>>>> Read
>>>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
>>>>> would
>>>>>>>> set
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
>>>>>>>> think
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
>> place
>>>>> to
>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>>>>> different
>>>>>>>>>>>>> values
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
>> that
>>>>> as
>>>>>>>> the
>>>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this
>>>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>>>>>>> disk_offering,
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>>>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>>>>> integration)
>>>>>>>> is
>>>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>>>>>>>> potential
>>>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>>>>>>>> CloudStack
>>>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
>> expunge.
>>>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
No problem. Answers are below in red.
Thanks!
On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Could you please answer the following questions for me with regards to the
> operation of the SolidFire plugin:
>
> What is the cardinality between iSCSI LUNs and SAN volumes?
>
Each SAN volume is equivalent to a single LUN (LUN 0).
1 SAN volume : 1 LUN
> What is the cardinality between SAN Volumes and CloudStack Volumes?
>
1 SAN volume : 1 CloudStack volume
> Are the LUN(s) created by the management server or externally by the
> operator?
>
When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) is
created by the management server (via the plug-in) the first time the
CloudStack volume is attached to a hypervisor.
If you don't want to use the SolidFire plug-in, but still want to use a
SolidFire volume (LUN), you can do this already today (prior to 4.2). The
admin manually creates the SAN volume and - in this case - multiple VMs and
data disks can share this SAN volume. While you can do this today, it is
not useful if you want to enforce storage QoS.
> Are the SAN volumes by the management server or externally by the operator?
>
When the SolidFire plug-in is used, the SAN volumes are completely managed
by the management server (via the plug-in). There is no admin interaction.
This allows for a 1:1 mapping between a SAN volume and a CloudStack volume,
which is necessary for any storage vendor that supports true QoS.
>
> I would like to clarify how these pieces are related and expected to
> operate.
>
> Thanks,
> -John
>
> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > "In particular, how do we ensure that multiple VMs with provisioned IOPS
> > won't be cut off by the underlying storage."
> >
> > In the storage QoS world, we need to map a single SAN volume (LUN) to a
> > single CloudStack volume. We cannot have multiple CloudStack volumes
> > sharing a single SAN volume and still guarantee QoS.
> >
> > If the user wants to have a single SAN volume house more than one
> > CloudStack volume, then can do that today without any of my plug-in code.
> >
> >
> > On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >> wrote:
> >
> >> "The administrator will allocate a SAN volume for CloudStack's use onto
> >> which CloudStack volumes will be created."
> >>
> >> I think we crossed e-mails. :)
> >>
> >> Check out my recent e-mail on this.
> >>
> >>
> >> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Mike,
> >>>
> >>> You are coming to the part which concerns me -- concepts from the
> >>> hypervisor are leaking into storage layer.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >>> wrote:
> >>>
> >>>> The weird part is that the iSCSI type is today only used (as far as I
> >>> know)
> >>>> in regards to XenServer (when you have not PreSetup an SR).
> >>>>
> >>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
> type.
> >>>>
> >>>> If you want to use your iSCSI volume from KVM, it uses the
> >>> SharedMountPoint
> >>>> type.
> >>>>
> >>>> So, I suppose mine and Edison's thinking here was to make a new type
> of
> >>>> storage to describe this dynamic ability Edison added into the storage
> >>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
> >>> versus,
> >>>> say, Dynamic_FC.
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com
> >>>>> wrote:
> >>>>
> >>>>> "The storage device itself shouldn't know or care that it is being
> used
> >>>>> for a Xen SR -- simply be able to answer questions about it is
> >>> storing."
> >>>>>
> >>>>> I see...so your concern here is that the SolidFire plug-in needs to
> >>> call
> >>>>> itself "Dynamic" storage so that the hypervisor logic knows to treat
> >>> it as
> >>>>> such.
> >>>>>
> >>>>> I'm totally open to removing that constraint and just calling it
> iSCSI
> >>> or
> >>>>> whatever. We would just need a way for the hypervisor attach logic to
> >>>>> detect this new requirement and perform the necessary activities.
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
> >>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> See my responses in-line.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I'm trying to picture this:
> >>>>>>>
> >>>>>>> "Finally, while CloudStack may be able to manage a device, an
> >>> operator
> >>>>>> may
> >>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
> shared
> >>> by
> >>>>>>> many services, and the operator has chosen to dedicate only a
> portion
> >>>>>> of it
> >>>>>>> to CloudStack). Does my reasoning make sense?"
> >>>>>>>
> >>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
> >>> (before
> >>>>>> an
> >>>>>>> attach request to the hypervisor) would work unless the hypervisor
> >>>>>> consumes
> >>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>
> >>>>>> My thinking is that, independent of CloudStack, an operator
> allocates
> >>> a
> >>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
> >>> simply
> >>>>>> want to turn control of that LUN over to CloudStack, but not allow
> >>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>
> >>>>>>>
> >>>>>>> As the attach logic stands prior to my changes, we would be passing
> >>> in a
> >>>>>>> SAN volume that does not have the necessary hypervisor support
> (like
> >>> an
> >>>>>> SR)
> >>>>>>> and the logic will fail.
> >>>>>>>
> >>>>>>> Are you thinking we should maybe have the storage framework itself
> >>>>>> detect
> >>>>>>> that such a SAN volume needs support from the hypervisor side and
> >>> have
> >>>>>> it
> >>>>>>> call into the agent code specifically to create the SR before the
> >>> attach
> >>>>>>> logic runs in the agent code?
> >>>>>>
> >>>>>> I think the hypervisor management plugin should have a rich enough
> >>>>>> interface to storage to determine available for volume storage. For
> >>> Xen,
> >>>>>> this interface would allow the interrogation of the device to
> >>> determine the
> >>>>>> SR is present. The storage device itself shouldn't know or care
> >>> that it
> >>>>>> is being used for a Xen SR -- simply be able to answer questions
> >>> about it
> >>>>>> is storing.
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>>> wrote:
> >>>>>>>
> >>>>>>>> So, the flow is as follows:
> >>>>>>>>
> >>>>>>>> * The admin registers the SolidFire driver (which is a type of
> >>>>>> so-called
> >>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows
> up
> >>> in
> >>>>>> the
> >>>>>>>> applicable zone.
> >>>>>>>>
> >>>>>>>> * The admin creates a Disk Offering that references the storage
> tag
> >>> of
> >>>>>> the
> >>>>>>>> newly created Primary Storage.
> >>>>>>>>
> >>>>>>>> * The end user creates a CloudStack volume. This leads to a new
> row
> >>> in
> >>>>>> the
> >>>>>>>> cloud.volumes table.
> >>>>>>>>
> >>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
> disk).
> >>>>>> This
> >>>>>>>> leads to the storage framework calling the plug-in to create a new
> >>>>>> volume
> >>>>>>>> on its storage system (in my case, a SAN). The plug-in also
> updates
> >>> the
> >>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
> >>>>>> volume).
> >>>>>>>> This plug-in code is only invoked if the CloudStack volume is in
> the
> >>>>>>>> 'Allocated' state. After the attach, the volume will be in the
> >>> 'Ready'
> >>>>>>>> state (even after a detach disk) and the plug-in code will not be
> >>>>>> called
> >>>>>>>> again to create this SAN volume.
> >>>>>>>>
> >>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
> >>> volume
> >>>>>> to
> >>>>>>>> attach needs "assistance" in the form of a hypervisor data
> structure
> >>>>>> (ex.
> >>>>>>>> an SR on XenServer).
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
> >>>>>>>>> definitions, refers to the physical allocation on the device,
> >>>>>> correct?"
> >>>>>>>>>
> >>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>>>>>>
> >>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
> >>> try to
> >>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi John,
> >>>>>>>>>>
> >>>>>>>>>> What you say here may very well make sense, but I'm having a
> hard
> >>>>>> time
> >>>>>>>>>> envisioning it.
> >>>>>>>>>>
> >>>>>>>>>> Perhaps we should draw Edison in on this conversation as he was
> >>> the
> >>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>
> >>>>>>>>>> What do you think?
> >>>>>>>>>>
> >>>>>>>>>> Thanks!
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> jburwell@basho.com
> >>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> It feels like we are combining two distinct concepts -- storage
> >>>>>> device
> >>>>>>>>>>> management and storage protocols. In both cases, we are
> >>>>>> communicating with
> >>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
> >>> (Dynamic)
> >>>>>> on the
> >>>>>>>>>>> device while the other requires the volume to be volume to be
> >>>>>> managed
> >>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
> >>> sync on
> >>>>>>>>>>> terminology, volume, in these definitions, refers to the
> physical
> >>>>>>>>>>> allocation on the device, correct? Minimally, we must be able
> to
> >>>>>>>>>>> communicate with a storage device to move bits from one place
> to
> >>>>>> another,
> >>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device may
> be
> >>>>>> able to
> >>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged
> iSCSI
> >>>>>> device
> >>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
> SolidFire
> >>>>>> iSCSI
> >>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
> >>> storing
> >>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
> >>> device,
> >>>>>> an
> >>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
> the
> >>>>>> device is
> >>>>>>>>>>> shared by many services, and the operator has chosen to
> dedicate
> >>>>>> only a
> >>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
> >>>>>>>>>>>
> >>>>>>>>>>> Assuming my thoughts above are reasonable, it seems appropriate
> >>> to
> >>>>>>>>>>> strip the management concerns from StoragePoolType, add the
> >>> notion
> >>>>>> of a
> >>>>>>>>>>> storage device with an attached driver that indicates whether
> or
> >>>>>> not is
> >>>>>>>>>>> managed by CloudStack, and establish a abstraction
> representing a
> >>>>>> physical
> >>>>>>>>>>> allocation on a device separate that is associated with a
> volume.
> >>>>>> With
> >>>>>>>>>>> these notions in place, hypervisor drivers can declare which
> >>>>>> protocols they
> >>>>>>>>>>> support and when they encounter a device managed by CloudStack,
> >>>>>> utilize the
> >>>>>>>>>>> management operations exposed by the driver to automate
> >>> allocation.
> >>>>>> If
> >>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
> >>> drill
> >>>>>> down to
> >>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Here is the difference between the current iSCSI type and the
> >>>>>> Dynamic
> >>>>>>>>>>> type:
> >>>>>>>>>>>>
> >>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
> Storage
> >>>>>> based
> >>>>>>>>>>> on
> >>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
> >>> exist
> >>>>>> on
> >>>>>>>>>>> the
> >>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
> volumes
> >>> are
> >>>>>>>>>>> created
> >>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
> Storage
> >>>>>> based
> >>>>>>>>>>> on a
> >>>>>>>>>>>> plug-in that will create and delete volumes on its storage
> >>> system
> >>>>>>>>>>>> dynamically (as is enabled via the storage framework). When a
> >>> user
> >>>>>>>>>>> wants to
> >>>>>>>>>>>> attach a CloudStack volume that was created, the framework
> tells
> >>>>>> the
> >>>>>>>>>>>> plug-in to create a new volume. After this is done, the attach
> >>>>>> logic
> >>>>>>>>>>> for
> >>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
> >>> structure
> >>>>>>>>>>> exists
> >>>>>>>>>>>> at this point because the volume was just created. The
> >>> hypervisor
> >>>>>> data
> >>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> These are new terms, so I should probably have defined them
> up
> >>>>>> front
> >>>>>>>>>>> for
> >>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
> >>>>>> creates a
> >>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is created
> >>> to
> >>>>>>>>>>> consume
> >>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
> >>>>>> structure
> >>>>>>>>>>> is
> >>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
> >>>>>>>>>>> hypervisor
> >>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
> >>> assumes
> >>>>>>>>>>> the
> >>>>>>>>>>>>> hypervisor data structure is already in place and simply
> >>> attaches
> >>>>>>>>>>> the VDI
> >>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead
> of
> >>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on a
> >>> SAN,
> >>>>>>>>>>> but the
> >>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
> >>> when an
> >>>>>>>>>>> attach
> >>>>>>>>>>>>> volume is performed because these types of volumes have not
> >>> been
> >>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once the
> >>>>>> attach
> >>>>>>>>>>> logic
> >>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it attaches
> >>> the
> >>>>>>>>>>> one and
> >>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>> jburwell@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The current implementation of the Dynamic type attach
> behavior
> >>>>>>>>>>> works in
> >>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
> >>> Another
> >>>>>>>>>>> way to
> >>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
> >>> storage
> >>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
> >>> XenServer
> >>>>>>>>>>> when you
> >>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on an
> >>>>>> iSCSI
> >>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>> This allows you to skip the step of going to the hypervisor
> >>> and
> >>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>> storage repository based on that iSCSI target as CloudStack
> >>> does
> >>>>>>>>>>> that
> >>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>> for you. I think this is only supported for XenServer. For
> >>> all
> >>>>>>>>>>> other
> >>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
> perform
> >>>>>> this
> >>>>>>>>>>> step
> >>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Reading through the code, what is the difference between
> the
> >>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set
> up
> >>>>>> by
> >>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
> >>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> For example, let's say another storage company wants to
> >>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
> >>> would
> >>>>>> be
> >>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
> >>> implement a
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
> >>> specific.
> >>>>>>>>>>> It is
> >>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
> >>>>>> zone-wide
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
> >>>>>> storage).
> >>>>>>>>>>> This
> >>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
> >>>>>> changes.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
> >>> creation
> >>>>>> and
> >>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
> him
> >>> in
> >>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
> >>> realized
> >>>>>>>>>>> there
> >>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless
> the
> >>>>>>>>>>> attach
> >>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
> create
> >>> the
> >>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
> >>> Storage
> >>>>>>>>>>> layer
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see the
> >>>>>> Storage
> >>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>> expose a set of operations that can be composed/driven
> >>> by
> >>>>>> the
> >>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
> per
> >>>>>> their
> >>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
> with a
> >>>>>>>>>>> massive
> >>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
> >>> difficult to
> >>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
> >>>>>> SolidFire
> >>>>>>>>>>> code
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
> >>>>>> XenServer,
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system, then
> go
> >>>>>> into
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the volume
> >>> (ex.
> >>>>>> a
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
> VMs
> >>> and
> >>>>>>>>>>> data
> >>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
> >>>>>> longer
> >>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between
> a
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
> >>>>>> Quality
> >>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
> >>> dynamically
> >>>>>>>>>>> create
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
> address
> >>> is
> >>>>>> in
> >>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
> >>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so.
> >>> I've
> >>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
> sandbox
> >>>>>> and
> >>>>>>>>>>> can
> >>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
> >>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
> >>> agent
> >>>>>>>>>>> code).
> >>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
> >>>>>> volume
> >>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
> >>> hypervisor,
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
> >>> invoked on
> >>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and create,
> >>>>>> say, a
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
> >>> VMware) to
> >>>>>>>>>>> make
> >>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making
> the
> >>>>>>>>>>> attach
> >>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
> >>> hypervisor
> >>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
> >>> hypervisors
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to
> >>> get
> >>>>>> KVM
> >>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
> >>> Depending on
> >>>>>>>>>>> what
> >>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working
> for
> >>>>>>>>>>> VMware in
> >>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to
> >>> Disk
> >>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
> >>> Damage
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch
> >>> cycle
> >>>>>>>>>>> post
> >>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non
> >>> issue
> >>>>>>>>>>> as I
> >>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
> >>>>>> users
> >>>>>>>>>>> need
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>>>>>>>>> service_offering
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
> >>> calculation.
> >>>>>> Of
> >>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
> >>> individual
> >>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on
> it
> >>>>>>>>>>> later. Do
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
> >>>>>> adding
> >>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
> >>> statistics
> >>>>>>>>>>> rather
> >>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you
> >>> have
> >>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
> >>>>>> read/write
> >>>>>>>>>>> for
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
> more
> >>>>>> for
> >>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
> >>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine
> >>> our
> >>>>>>>>>>> users
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
> >>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
> >>> supports
> >>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
> through
> >>>>>>>>>>> command
> >>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
> >>> parameters
> >>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I
> apologize
> >>>>>> for
> >>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a
> >>> set
> >>>>>> of
> >>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
> >>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better
> to
> >>>>>>>>>>> commit
> >>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider
> the
> >>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
> response
> >>> to
> >>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I
> think
> >>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I
> >>> think
> >>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
> have
> >>>>>> have
> >>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
> >>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies
> be
> >>>>>>>>>>> captured
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
> >>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model
> for
> >>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> >>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
> >>>>>>>>>>> particular,
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
> >>>>>>>>>>> configured
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
> deliver.
> >>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
> >>>>>> storage
> >>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
> >>> device
> >>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
> controlling
> >>> the
> >>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
> from
> >>> the
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell
> <
> >>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
> >>> merged
> >>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to
> >>> turn
> >>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue
> is
> >>>>>>>>>>> addressed
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
> >>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> intersect/compliment
> >>> the
> >>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
> >>>>>>>>>>> understand
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would
> like
> >>> to
> >>>>>>>>>>> ensure
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
> >>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wido@widodh.nl
> >
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
> >>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason
> >>> till
> >>>>>>>>>>> Monday,
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
> mention
> >>>>>> again
> >>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
> >>> never
> >>>>>>>>>>> release
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
> throttling
> >>>>>> and
> >>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from
> day
> >>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
> >>> support
> >>>>>>>>>>> read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
> because
> >>> I
> >>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
> >>>>>> Therefore, we
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
> >>> backwards
> >>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
> now
> >>> so
> >>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before
> >>> the
> >>>>>> 4.2
> >>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead
> >>> of
> >>>>>>>>>>> Total
> >>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze
> >>> date.
> >>>>>>>>>>> But if
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
> necessary
> >>>>>> to do
> >>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> >>>>>> association
> >>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
> >>> Hollander <
> >>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
> >>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because
> >>> the
> >>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
> >>> bytes as
> >>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
> disk
> >>> I/O
> >>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
> >>>>>> situations
> >>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
> work.
> >>> It
> >>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
> freeze,
> >>> I
> >>>>>> will
> >>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But
> if
> >>> it
> >>>>>>>>>>> goes
> >>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release
> >>> with
> >>>>>>>>>>> only
> >>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
> >>> branch
> >>>>>> into
> >>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master
> in
> >>> 48
> >>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
> >>> storage
> >>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
> limitation
> >>> of
> >>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
> affect
> >>>>>> the
> >>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and
> >>> limit
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
> >>> difference
> >>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
> >>> different
> >>>>>>>>>>> rate
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
> >>>>>> collect
> >>>>>>>>>>> both
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go
> >>> for
> >>>>>> a
> >>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
> >>> then
> >>>>>> Read
> >>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
> >>> would
> >>>>>> set
> >>>>>>>>>>> a
> >>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
> >>>>>> think
> >>>>>>>>>>> it
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
> place
> >>> to
> >>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
> >>> different
> >>>>>>>>>>> values
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
> that
> >>> as
> >>>>>> the
> >>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this
> >>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
> >>>>>> disk_offering,
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
> >>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
> >>> integration)
> >>>>>> is
> >>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
> >>>>>> potential
> >>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
> >>>>>> CloudStack
> >>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
> expunge.
> >>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Could you please answer the following questions for me with regards to the operation of the SolidFire plugin:
What is the cardinality between iSCSI LUNs and SAN volumes?
What is the cardinality between SAN Volumes and CloudStack Volumes?
Are the LUN(s) created by the management server or externally by the operator?
Are the SAN volumes by the management server or externally by the operator?
I would like to clarify how these pieces are related and expected to operate.
Thanks,
-John
On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> "In particular, how do we ensure that multiple VMs with provisioned IOPS
> won't be cut off by the underlying storage."
>
> In the storage QoS world, we need to map a single SAN volume (LUN) to a
> single CloudStack volume. We cannot have multiple CloudStack volumes
> sharing a single SAN volume and still guarantee QoS.
>
> If the user wants to have a single SAN volume house more than one
> CloudStack volume, then can do that today without any of my plug-in code.
>
>
> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
>> wrote:
>
>> "The administrator will allocate a SAN volume for CloudStack's use onto
>> which CloudStack volumes will be created."
>>
>> I think we crossed e-mails. :)
>>
>> Check out my recent e-mail on this.
>>
>>
>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> You are coming to the part which concerns me -- concepts from the
>>> hypervisor are leaking into storage layer.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>>> The weird part is that the iSCSI type is today only used (as far as I
>>> know)
>>>> in regards to XenServer (when you have not PreSetup an SR).
>>>>
>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs type.
>>>>
>>>> If you want to use your iSCSI volume from KVM, it uses the
>>> SharedMountPoint
>>>> type.
>>>>
>>>> So, I suppose mine and Edison's thinking here was to make a new type of
>>>> storage to describe this dynamic ability Edison added into the storage
>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
>>> versus,
>>>> say, Dynamic_FC.
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com
>>>>> wrote:
>>>>
>>>>> "The storage device itself shouldn't know or care that it is being used
>>>>> for a Xen SR -- simply be able to answer questions about it is
>>> storing."
>>>>>
>>>>> I see...so your concern here is that the SolidFire plug-in needs to
>>> call
>>>>> itself "Dynamic" storage so that the hypervisor logic knows to treat
>>> it as
>>>>> such.
>>>>>
>>>>> I'm totally open to removing that constraint and just calling it iSCSI
>>> or
>>>>> whatever. We would just need a way for the hypervisor attach logic to
>>>>> detect this new requirement and perform the necessary activities.
>>>>>
>>>>>
>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> See my responses in-line.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I'm trying to picture this:
>>>>>>>
>>>>>>> "Finally, while CloudStack may be able to manage a device, an
>>> operator
>>>>>> may
>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is shared
>>> by
>>>>>>> many services, and the operator has chosen to dedicate only a portion
>>>>>> of it
>>>>>>> to CloudStack). Does my reasoning make sense?"
>>>>>>>
>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
>>> (before
>>>>>> an
>>>>>>> attach request to the hypervisor) would work unless the hypervisor
>>>>>> consumes
>>>>>>> the SAN volume in the form of, say, an SR.
>>>>>>
>>>>>> My thinking is that, independent of CloudStack, an operator allocates
>>> a
>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
>>> simply
>>>>>> want to turn control of that LUN over to CloudStack, but not allow
>>>>>> CloudStack to allocate anymore LUNs.
>>>>>>
>>>>>>>
>>>>>>> As the attach logic stands prior to my changes, we would be passing
>>> in a
>>>>>>> SAN volume that does not have the necessary hypervisor support (like
>>> an
>>>>>> SR)
>>>>>>> and the logic will fail.
>>>>>>>
>>>>>>> Are you thinking we should maybe have the storage framework itself
>>>>>> detect
>>>>>>> that such a SAN volume needs support from the hypervisor side and
>>> have
>>>>>> it
>>>>>>> call into the agent code specifically to create the SR before the
>>> attach
>>>>>>> logic runs in the agent code?
>>>>>>
>>>>>> I think the hypervisor management plugin should have a rich enough
>>>>>> interface to storage to determine available for volume storage. For
>>> Xen,
>>>>>> this interface would allow the interrogation of the device to
>>> determine the
>>>>>> SR is present. The storage device itself shouldn't know or care
>>> that it
>>>>>> is being used for a Xen SR -- simply be able to answer questions
>>> about it
>>>>>> is storing.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com
>>>>>>>> wrote:
>>>>>>>
>>>>>>>> So, the flow is as follows:
>>>>>>>>
>>>>>>>> * The admin registers the SolidFire driver (which is a type of
>>>>>> so-called
>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows up
>>> in
>>>>>> the
>>>>>>>> applicable zone.
>>>>>>>>
>>>>>>>> * The admin creates a Disk Offering that references the storage tag
>>> of
>>>>>> the
>>>>>>>> newly created Primary Storage.
>>>>>>>>
>>>>>>>> * The end user creates a CloudStack volume. This leads to a new row
>>> in
>>>>>> the
>>>>>>>> cloud.volumes table.
>>>>>>>>
>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach disk).
>>>>>> This
>>>>>>>> leads to the storage framework calling the plug-in to create a new
>>>>>> volume
>>>>>>>> on its storage system (in my case, a SAN). The plug-in also updates
>>> the
>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
>>>>>> volume).
>>>>>>>> This plug-in code is only invoked if the CloudStack volume is in the
>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
>>> 'Ready'
>>>>>>>> state (even after a detach disk) and the plug-in code will not be
>>>>>> called
>>>>>>>> again to create this SAN volume.
>>>>>>>>
>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
>>> volume
>>>>>> to
>>>>>>>> attach needs "assistance" in the form of a hypervisor data structure
>>>>>> (ex.
>>>>>>>> an SR on XenServer).
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
>>>>>>>>> definitions, refers to the physical allocation on the device,
>>>>>> correct?"
>>>>>>>>>
>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>>>>>>
>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
>>> try to
>>>>>>>>> use 'CloudStack volume'.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi John,
>>>>>>>>>>
>>>>>>>>>> What you say here may very well make sense, but I'm having a hard
>>>>>> time
>>>>>>>>>> envisioning it.
>>>>>>>>>>
>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he was
>>> the
>>>>>>>>>> initial person to suggest the approach I took.
>>>>>>>>>>
>>>>>>>>>> What do you think?
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> It feels like we are combining two distinct concepts -- storage
>>>>>> device
>>>>>>>>>>> management and storage protocols. In both cases, we are
>>>>>> communicating with
>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
>>> (Dynamic)
>>>>>> on the
>>>>>>>>>>> device while the other requires the volume to be volume to be
>>>>>> managed
>>>>>>>>>>> outside of the CloudStack context. To ensure that we are in
>>> sync on
>>>>>>>>>>> terminology, volume, in these definitions, refers to the physical
>>>>>>>>>>> allocation on the device, correct? Minimally, we must be able to
>>>>>>>>>>> communicate with a storage device to move bits from one place to
>>>>>> another,
>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device may be
>>>>>> able to
>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
>>>>>> device
>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed SolidFire
>>>>>> iSCSI
>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
>>> storing
>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a
>>> device,
>>>>>> an
>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
>>>>>> device is
>>>>>>>>>>> shared by many services, and the operator has chosen to dedicate
>>>>>> only a
>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>>>>>>>>
>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems appropriate
>>> to
>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
>>> notion
>>>>>> of a
>>>>>>>>>>> storage device with an attached driver that indicates whether or
>>>>>> not is
>>>>>>>>>>> managed by CloudStack, and establish a abstraction representing a
>>>>>> physical
>>>>>>>>>>> allocation on a device separate that is associated with a volume.
>>>>>> With
>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
>>>>>> protocols they
>>>>>>>>>>> support and when they encounter a device managed by CloudStack,
>>>>>> utilize the
>>>>>>>>>>> management operations exposed by the driver to automate
>>> allocation.
>>>>>> If
>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
>>> drill
>>>>>> down to
>>>>>>>>>>> a more detailed design.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Here is the difference between the current iSCSI type and the
>>>>>> Dynamic
>>>>>>>>>>> type:
>>>>>>>>>>>>
>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary Storage
>>>>>> based
>>>>>>>>>>> on
>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
>>> exist
>>>>>> on
>>>>>>>>>>> the
>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack volumes
>>> are
>>>>>>>>>>> created
>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>>>>>>>>>>>
>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary Storage
>>>>>> based
>>>>>>>>>>> on a
>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
>>> system
>>>>>>>>>>>> dynamically (as is enabled via the storage framework). When a
>>> user
>>>>>>>>>>> wants to
>>>>>>>>>>>> attach a CloudStack volume that was created, the framework tells
>>>>>> the
>>>>>>>>>>>> plug-in to create a new volume. After this is done, the attach
>>>>>> logic
>>>>>>>>>>> for
>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
>>> structure
>>>>>>>>>>> exists
>>>>>>>>>>>> at this point because the volume was just created. The
>>> hypervisor
>>>>>> data
>>>>>>>>>>>> structure must be created.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> These are new terms, so I should probably have defined them up
>>>>>> front
>>>>>>>>>>> for
>>>>>>>>>>>>> you. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>>>>>> creates a
>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is created
>>> to
>>>>>>>>>>> consume
>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>>>>>> structure
>>>>>>>>>>> is
>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>>>>>>>>>>> hypervisor
>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
>>> assumes
>>>>>>>>>>> the
>>>>>>>>>>>>> hypervisor data structure is already in place and simply
>>> attaches
>>>>>>>>>>> the VDI
>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on a
>>> SAN,
>>>>>>>>>>> but the
>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
>>> when an
>>>>>>>>>>> attach
>>>>>>>>>>>>> volume is performed because these types of volumes have not
>>> been
>>>>>>>>>>> pre-hooked
>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once the
>>>>>> attach
>>>>>>>>>>> logic
>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it attaches
>>> the
>>>>>>>>>>> one and
>>>>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>>> jburwell@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The current implementation of the Dynamic type attach behavior
>>>>>>>>>>> works in
>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
>>> Another
>>>>>>>>>>> way to
>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
>>> storage
>>>>>>>>>>> pool type?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>>> XenServer
>>>>>>>>>>> when you
>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on an
>>>>>> iSCSI
>>>>>>>>>>>>>> target.
>>>>>>>>>>>>>>> This allows you to skip the step of going to the hypervisor
>>> and
>>>>>>>>>>>>>> creating a
>>>>>>>>>>>>>>> storage repository based on that iSCSI target as CloudStack
>>> does
>>>>>>>>>>> that
>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer. For
>>> all
>>>>>>>>>>> other
>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and perform
>>>>>> this
>>>>>>>>>>> step
>>>>>>>>>>>>>>> manually.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Reading through the code, what is the difference between the
>>>>>>>>>>> ISCSI and
>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up
>>>>>> by
>>>>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>>>>> SolidFire)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants to
>>>>>>>>>>> implement a
>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
>>> would
>>>>>> be
>>>>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>>> implement a
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>>>>>>>>> hypervisor-attach
>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>>> specific.
>>>>>>>>>>> It is
>>>>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>>>>>> zone-wide
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>>>>>> storage).
>>>>>>>>>>> This
>>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>>>>>> changes.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
>>> creation
>>>>>> and
>>>>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with him
>>> in
>>>>>>>>>>> Portland
>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
>>> realized
>>>>>>>>>>> there
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
>>>>>>>>>>> attach
>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and create
>>> the
>>>>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
>>> Storage
>>>>>>>>>>> layer
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see the
>>>>>> Storage
>>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be composed/driven
>>> by
>>>>>> the
>>>>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures per
>>>>>> their
>>>>>>>>>>>>>> needs.
>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>>>>>>>>>>> massive
>>>>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
>>> difficult to
>>>>>>>>>>>>>> maintain
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>>>>>> SolidFire
>>>>>>>>>>> code
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>>>>>> XenServer,
>>>>>>>>>>> the
>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system, then go
>>>>>> into
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the volume
>>> (ex.
>>>>>> a
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs
>>> and
>>>>>>>>>>> data
>>>>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
>>>>>> longer
>>>>>>>>>>> be so
>>>>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
>>>>>> Quality
>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>>> dynamically
>>>>>>>>>>> create
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not address
>>> is
>>>>>> in
>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>>>>>>>>>>> performing an
>>>>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so.
>>> I've
>>>>>>>>>>> got
>>>>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox
>>>>>> and
>>>>>>>>>>> can
>>>>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>>> agnostic.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
>>> agent
>>>>>>>>>>> code).
>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
>>>>>> volume
>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>>> hypervisor,
>>>>>>>>>>> the
>>>>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>>> invoked on
>>>>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and create,
>>>>>> say, a
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>>> VMware) to
>>>>>>>>>>> make
>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making the
>>>>>>>>>>> attach
>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>>> hypervisor
>>>>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>>> hypervisors
>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to
>>> get
>>>>>> KVM
>>>>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>>> Depending on
>>>>>>>>>>> what
>>>>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>>>>>>>>>> VMware in
>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to
>>> Disk
>>>>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
>>> Damage
>>>>>> <
>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch
>>> cycle
>>>>>>>>>>> post
>>>>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non
>>> issue
>>>>>>>>>>> as I
>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
>>>>>> users
>>>>>>>>>>> need
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>>>>>>>>> service_offering
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>>> calculation.
>>>>>> Of
>>>>>>>>>>> course
>>>>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>>> individual
>>>>>>>>>>> volume,
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>>>>>>>>>> later. Do
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
>>>>>> adding
>>>>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>>> statistics
>>>>>>>>>>> rather
>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you
>>> have
>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>>>>>> read/write
>>>>>>>>>>> for
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more
>>>>>> for
>>>>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>>> operations (
>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine
>>> our
>>>>>>>>>>> users
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>>>>>>>>>>> operator
>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>>> supports
>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>>>>>>>>>> command
>>>>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>>> parameters
>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize
>>>>>> for
>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a
>>> set
>>>>>> of
>>>>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>>>>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>>>>>>>>>>> commit
>>>>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>>>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response
>>> to
>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I
>>> think
>>>>>>>>>>> we need
>>>>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have
>>>>>> have
>>>>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>>> following:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>>>>>>>>>> captured
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>>>>>>>>>> expressing
>>>>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>>>>>>>>>> reasoning
>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>>>>>>>>>> particular,
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>>>>>>>>>> configured
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>>>>>>>>>> Another
>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>>>>>> storage
>>>>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
>>> device
>>>>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling
>>> the
>>>>>>>>>>> max
>>>>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from
>>> the
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
>>> merged
>>>>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to
>>> turn
>>>>>>>>>>> over to
>>>>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>>>>>>>>>> addressed
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>>>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment
>>> the
>>>>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>>>>>>>>>> understand
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like
>>> to
>>>>>>>>>>> ensure
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>>>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason
>>> till
>>>>>>>>>>> Monday,
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
>>>>>> again
>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
>>> never
>>>>>>>>>>> release
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling
>>>>>> and
>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day
>>>>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
>>> support
>>>>>>>>>>> read
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because
>>> I
>>>>>>>>>>> don't see
>>>>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>>>>>> Therefore, we
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>>> backwards
>>>>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now
>>> so
>>>>>>>>>>> that it
>>>>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before
>>> the
>>>>>> 4.2
>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead
>>> of
>>>>>>>>>>> Total
>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze
>>> date.
>>>>>>>>>>> But if
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary
>>>>>> to do
>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>>>>>> association
>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>>> Hollander <
>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because
>>> the
>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
>>> bytes as
>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk
>>> I/O
>>>>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>>>>>> situations
>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work.
>>> It
>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze,
>>> I
>>>>>> will
>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if
>>> it
>>>>>>>>>>> goes
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release
>>> with
>>>>>>>>>>> only
>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
>>> branch
>>>>>> into
>>>>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in
>>> 48
>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
>>> storage
>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation
>>> of
>>>>>>>>>>> device
>>>>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect
>>>>>> the
>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and
>>> limit
>>>>>>>>>>> the
>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>>> difference
>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>>> different
>>>>>>>>>>> rate
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>>>>>> collect
>>>>>>>>>>> both
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go
>>> for
>>>>>> a
>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
>>> then
>>>>>> Read
>>>>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
>>> would
>>>>>> set
>>>>>>>>>>> a
>>>>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
>>>>>> think
>>>>>>>>>>> it
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place
>>> to
>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>>> different
>>>>>>>>>>> values
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that
>>> as
>>>>>> the
>>>>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this
>>> way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>>>>> disk_offering,
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>>>>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>>> integration)
>>>>>> is
>>>>>>>>>>>>>> included
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>>>>>> potential
>>>>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>>>>>> CloudStack
>>>>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>>>>>>>>>> migrate,
>>>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"In particular, how do we ensure that multiple VMs with provisioned IOPS
won't be cut off by the underlying storage."
In the storage QoS world, we need to map a single SAN volume (LUN) to a
single CloudStack volume. We cannot have multiple CloudStack volumes
sharing a single SAN volume and still guarantee QoS.
If the user wants to have a single SAN volume house more than one
CloudStack volume, then can do that today without any of my plug-in code.
On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> "The administrator will allocate a SAN volume for CloudStack's use onto
> which CloudStack volumes will be created."
>
> I think we crossed e-mails. :)
>
> Check out my recent e-mail on this.
>
>
> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> You are coming to the part which concerns me -- concepts from the
>> hypervisor are leaking into storage layer.
>>
>> Thanks,
>> -John
>>
>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > The weird part is that the iSCSI type is today only used (as far as I
>> know)
>> > in regards to XenServer (when you have not PreSetup an SR).
>> >
>> > If you want to use your iSCSI volume from VMware, it uses the vmfs type.
>> >
>> > If you want to use your iSCSI volume from KVM, it uses the
>> SharedMountPoint
>> > type.
>> >
>> > So, I suppose mine and Edison's thinking here was to make a new type of
>> > storage to describe this dynamic ability Edison added into the storage
>> > framework. Maybe it should be more specificy, though: Dynamic_iSCSI
>> versus,
>> > say, Dynamic_FC.
>> >
>> >
>> > On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>> >> wrote:
>> >
>> >> "The storage device itself shouldn't know or care that it is being used
>> >> for a Xen SR -- simply be able to answer questions about it is
>> storing."
>> >>
>> >> I see...so your concern here is that the SolidFire plug-in needs to
>> call
>> >> itself "Dynamic" storage so that the hypervisor logic knows to treat
>> it as
>> >> such.
>> >>
>> >> I'm totally open to removing that constraint and just calling it iSCSI
>> or
>> >> whatever. We would just need a way for the hypervisor attach logic to
>> >> detect this new requirement and perform the necessary activities.
>> >>
>> >>
>> >> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >>
>> >>> Mike,
>> >>>
>> >>> See my responses in-line.
>> >>>
>> >>> Thanks,
>> >>> -John
>> >>>
>> >>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >>> wrote:
>> >>>
>> >>>> I'm trying to picture this:
>> >>>>
>> >>>> "Finally, while CloudStack may be able to manage a device, an
>> operator
>> >>> may
>> >>>> chose to leave it unmanaged by CloudStack (e.g. the device is shared
>> by
>> >>>> many services, and the operator has chosen to dedicate only a portion
>> >>> of it
>> >>>> to CloudStack). Does my reasoning make sense?"
>> >>>>
>> >>>> I guess I'm not sure how creating a SAN volume via the plug-in
>> (before
>> >>> an
>> >>>> attach request to the hypervisor) would work unless the hypervisor
>> >>> consumes
>> >>>> the SAN volume in the form of, say, an SR.
>> >>>
>> >>> My thinking is that, independent of CloudStack, an operator allocates
>> a
>> >>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
>> simply
>> >>> want to turn control of that LUN over to CloudStack, but not allow
>> >>> CloudStack to allocate anymore LUNs.
>> >>>
>> >>>>
>> >>>> As the attach logic stands prior to my changes, we would be passing
>> in a
>> >>>> SAN volume that does not have the necessary hypervisor support (like
>> an
>> >>> SR)
>> >>>> and the logic will fail.
>> >>>>
>> >>>> Are you thinking we should maybe have the storage framework itself
>> >>> detect
>> >>>> that such a SAN volume needs support from the hypervisor side and
>> have
>> >>> it
>> >>>> call into the agent code specifically to create the SR before the
>> attach
>> >>>> logic runs in the agent code?
>> >>>
>> >>> I think the hypervisor management plugin should have a rich enough
>> >>> interface to storage to determine available for volume storage. For
>> Xen,
>> >>> this interface would allow the interrogation of the device to
>> determine the
>> >>> SR is present. The storage device itself shouldn't know or care
>> that it
>> >>> is being used for a Xen SR -- simply be able to answer questions
>> about it
>> >>> is storing.
>> >>>
>> >>>>
>> >>>>
>> >>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>> >>> mike.tutkowski@solidfire.com
>> >>>>> wrote:
>> >>>>
>> >>>>> So, the flow is as follows:
>> >>>>>
>> >>>>> * The admin registers the SolidFire driver (which is a type of
>> >>> so-called
>> >>>>> Dynamic storage). Once this is done, a new Primary Storage shows up
>> in
>> >>> the
>> >>>>> applicable zone.
>> >>>>>
>> >>>>> * The admin creates a Disk Offering that references the storage tag
>> of
>> >>> the
>> >>>>> newly created Primary Storage.
>> >>>>>
>> >>>>> * The end user creates a CloudStack volume. This leads to a new row
>> in
>> >>> the
>> >>>>> cloud.volumes table.
>> >>>>>
>> >>>>> * The end user attaches the CloudStack volume to a VM (attach disk).
>> >>> This
>> >>>>> leads to the storage framework calling the plug-in to create a new
>> >>> volume
>> >>>>> on its storage system (in my case, a SAN). The plug-in also updates
>> the
>> >>>>> cloud.volumes row with applicable data (like the IQN of the SAN
>> >>> volume).
>> >>>>> This plug-in code is only invoked if the CloudStack volume is in the
>> >>>>> 'Allocated' state. After the attach, the volume will be in the
>> 'Ready'
>> >>>>> state (even after a detach disk) and the plug-in code will not be
>> >>> called
>> >>>>> again to create this SAN volume.
>> >>>>>
>> >>>>> * The hypervisor-attach logic is run and detects the CloudStack
>> volume
>> >>> to
>> >>>>> attach needs "assistance" in the form of a hypervisor data structure
>> >>> (ex.
>> >>>>> an SR on XenServer).
>> >>>>>
>> >>>>>
>> >>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>
>> >>>>>> "To ensure that we are in sync on terminology, volume, in these
>> >>>>>> definitions, refers to the physical allocation on the device,
>> >>> correct?"
>> >>>>>>
>> >>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>> >>>>>>
>> >>>>>> To refer to the 'volume' the end user can make in CloudStack, I
>> try to
>> >>>>>> use 'CloudStack volume'.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>
>> >>>>>>> Hi John,
>> >>>>>>>
>> >>>>>>> What you say here may very well make sense, but I'm having a hard
>> >>> time
>> >>>>>>> envisioning it.
>> >>>>>>>
>> >>>>>>> Perhaps we should draw Edison in on this conversation as he was
>> the
>> >>>>>>> initial person to suggest the approach I took.
>> >>>>>>>
>> >>>>>>> What do you think?
>> >>>>>>>
>> >>>>>>> Thanks!
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
>> >>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> It feels like we are combining two distinct concepts -- storage
>> >>> device
>> >>>>>>>> management and storage protocols. In both cases, we are
>> >>> communicating with
>> >>>>>>>> ISCSI, but one allows the system to create/delete volumes
>> (Dynamic)
>> >>> on the
>> >>>>>>>> device while the other requires the volume to be volume to be
>> >>> managed
>> >>>>>>>> outside of the CloudStack context. To ensure that we are in
>> sync on
>> >>>>>>>> terminology, volume, in these definitions, refers to the physical
>> >>>>>>>> allocation on the device, correct? Minimally, we must be able to
>> >>>>>>>> communicate with a storage device to move bits from one place to
>> >>> another,
>> >>>>>>>> read bits, delete bits, etc. Optionally, a storage device may be
>> >>> able to
>> >>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
>> >>> device
>> >>>>>>>> onto which we store a Xen SR, and we can have a managed SolidFire
>> >>> iSCSI
>> >>>>>>>> device on which CloudStack is capable of allocating LUNs and
>> storing
>> >>>>>>>> volumes. Finally, while CloudStack may be able to manage a
>> device,
>> >>> an
>> >>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
>> >>> device is
>> >>>>>>>> shared by many services, and the operator has chosen to dedicate
>> >>> only a
>> >>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>> >>>>>>>>
>> >>>>>>>> Assuming my thoughts above are reasonable, it seems appropriate
>> to
>> >>>>>>>> strip the management concerns from StoragePoolType, add the
>> notion
>> >>> of a
>> >>>>>>>> storage device with an attached driver that indicates whether or
>> >>> not is
>> >>>>>>>> managed by CloudStack, and establish a abstraction representing a
>> >>> physical
>> >>>>>>>> allocation on a device separate that is associated with a volume.
>> >>> With
>> >>>>>>>> these notions in place, hypervisor drivers can declare which
>> >>> protocols they
>> >>>>>>>> support and when they encounter a device managed by CloudStack,
>> >>> utilize the
>> >>>>>>>> management operations exposed by the driver to automate
>> allocation.
>> >>> If
>> >>>>>>>> these thoughts/concepts make sense, then we can sit down and
>> drill
>> >>> down to
>> >>>>>>>> a more detailed design.
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>
>> >>>>>>>>> Here is the difference between the current iSCSI type and the
>> >>> Dynamic
>> >>>>>>>> type:
>> >>>>>>>>>
>> >>>>>>>>> iSCSI type: The admin has to go in and create a Primary Storage
>> >>> based
>> >>>>>>>> on
>> >>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
>> exist
>> >>> on
>> >>>>>>>> the
>> >>>>>>>>> storage system (it is pre-allocated). Future CloudStack volumes
>> are
>> >>>>>>>> created
>> >>>>>>>>> as VDIs on the SR that was created behind the scenes.
>> >>>>>>>>>
>> >>>>>>>>> Dynamic type: The admin has to go in and create Primary Storage
>> >>> based
>> >>>>>>>> on a
>> >>>>>>>>> plug-in that will create and delete volumes on its storage
>> system
>> >>>>>>>>> dynamically (as is enabled via the storage framework). When a
>> user
>> >>>>>>>> wants to
>> >>>>>>>>> attach a CloudStack volume that was created, the framework tells
>> >>> the
>> >>>>>>>>> plug-in to create a new volume. After this is done, the attach
>> >>> logic
>> >>>>>>>> for
>> >>>>>>>>> the hypervisor in question is called. No hypervisor data
>> structure
>> >>>>>>>> exists
>> >>>>>>>>> at this point because the volume was just created. The
>> hypervisor
>> >>> data
>> >>>>>>>>> structure must be created.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> These are new terms, so I should probably have defined them up
>> >>> front
>> >>>>>>>> for
>> >>>>>>>>>> you. :)
>> >>>>>>>>>>
>> >>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>> >>> creates a
>> >>>>>>>>>> volume on a SAN), then a hypervisor data structure is created
>> to
>> >>>>>>>> consume
>> >>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>> >>> structure
>> >>>>>>>> is
>> >>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>> >>>>>>>> hypervisor
>> >>>>>>>>>> data structure as needed. In these cases, the attach logic
>> assumes
>> >>>>>>>> the
>> >>>>>>>>>> hypervisor data structure is already in place and simply
>> attaches
>> >>>>>>>> the VDI
>> >>>>>>>>>> on the hypervisor data structure to the VM in question.
>> >>>>>>>>>>
>> >>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>> >>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on a
>> SAN,
>> >>>>>>>> but the
>> >>>>>>>>>> SAN itself). The hypervisor data structure must be created
>> when an
>> >>>>>>>> attach
>> >>>>>>>>>> volume is performed because these types of volumes have not
>> been
>> >>>>>>>> pre-hooked
>> >>>>>>>>>> up to such a hypervisor data structure by an admin. Once the
>> >>> attach
>> >>>>>>>> logic
>> >>>>>>>>>> creates, say, an SR on XenServer for this volume, it attaches
>> the
>> >>>>>>>> one and
>> >>>>>>>>>> only VDI within the SR to the VM in question.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
>> jburwell@basho.com>
>> >>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Mike,
>> >>>>>>>>>>>
>> >>>>>>>>>>> The current implementation of the Dynamic type attach behavior
>> >>>>>>>> works in
>> >>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
>> Another
>> >>>>>>>> way to
>> >>>>>>>>>>> ask the question -- what is the definition of a Dynamic
>> storage
>> >>>>>>>> pool type?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> -John
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
>> XenServer
>> >>>>>>>> when you
>> >>>>>>>>>>>> want to set up Primary Storage that is directly based on an
>> >>> iSCSI
>> >>>>>>>>>>> target.
>> >>>>>>>>>>>> This allows you to skip the step of going to the hypervisor
>> and
>> >>>>>>>>>>> creating a
>> >>>>>>>>>>>> storage repository based on that iSCSI target as CloudStack
>> does
>> >>>>>>>> that
>> >>>>>>>>>>> part
>> >>>>>>>>>>>> for you. I think this is only supported for XenServer. For
>> all
>> >>>>>>>> other
>> >>>>>>>>>>>> hypervisors, you must first go to the hypervisor and perform
>> >>> this
>> >>>>>>>> step
>> >>>>>>>>>>>> manually.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I don't really know what RBD is.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>> >>> jburwell@basho.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Reading through the code, what is the difference between the
>> >>>>>>>> ISCSI and
>> >>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> >>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> This new type of storage is defined in the
>> >>>>>>>> Storage.StoragePoolType
>> >>>>>>>>>>> class
>> >>>>>>>>>>>>>> (called Dynamic):
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> public static enum StoragePoolType {
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Filesystem(false), // local directory
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> ISO(false), // for iso image
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> CLVM(true),
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> RBD(true),
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> SharedMountPoint(true),
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up
>> >>> by
>> >>>>>>>>>>>>>> customers.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> OCFS2(true),
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>> >>> SolidFire)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> boolean shared;
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> this.shared = shared;
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> public boolean isShared() {
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> return shared;
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> }
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> For example, let's say another storage company wants to
>> >>>>>>>> implement a
>> >>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
>> would
>> >>> be
>> >>>>>>>>>>> dynamic,
>> >>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
>> implement a
>> >>>>>>>> storage
>> >>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>> >>>>>>>> hypervisor-attach
>> >>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>> to support their plug-in.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
>> specific.
>> >>>>>>>> It is
>> >>>>>>>>>>> not.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>> >>> zone-wide
>> >>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>> >>> storage).
>> >>>>>>>> This
>> >>>>>>>>>>>>> kind of
>> >>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>> >>> changes.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Edison created a new framework that supported the
>> creation
>> >>> and
>> >>>>>>>>>>> deletion
>> >>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with him
>> in
>> >>>>>>>> Portland
>> >>>>>>>>>>>>> back
>> >>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
>> realized
>> >>>>>>>> there
>> >>>>>>>>>>> was
>> >>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
>> >>>>>>>> attach
>> >>>>>>>>>>> logic
>> >>>>>>>>>>>>> was
>> >>>>>>>>>>>>>>>> changed to recognize this new type of storage and create
>> the
>> >>>>>>>>>>>>> appropriate
>> >>>>>>>>>>>>>>>> hypervisor data structure.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>> >>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
>> Storage
>> >>>>>>>> layer
>> >>>>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>> understand or care about details. I expect to see the
>> >>> Storage
>> >>>>>>>>>>>>> services
>> >>>>>>>>>>>>>>>>> expose a set of operations that can be composed/driven
>> by
>> >>> the
>> >>>>>>>>>>>>> Hypervisor
>> >>>>>>>>>>>>>>>>> implementations to allocate space/create structures per
>> >>> their
>> >>>>>>>>>>> needs.
>> >>>>>>>>>>>>> If
>> >>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>> >>>>>>>> massive
>> >>>>>>>>>>>>> n-to-n
>> >>>>>>>>>>>>>>>>> problem that will make the system increasingly
>> difficult to
>> >>>>>>>>>>> maintain
>> >>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>> >>> SolidFire
>> >>>>>>>> code
>> >>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>> >>> XenServer,
>> >>>>>>>> the
>> >>>>>>>>>>> admin
>> >>>>>>>>>>>>>>>>> had
>> >>>>>>>>>>>>>>>>>> to first create a volume on the storage system, then go
>> >>> into
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>> to set up a data structure to make use of the volume
>> (ex.
>> >>> a
>> >>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs
>> and
>> >>>>>>>> data
>> >>>>>>>>>>> disks
>> >>>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>> shared this storage system's volume.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
>> >>> longer
>> >>>>>>>> be so
>> >>>>>>>>>>>>>>>>> static
>> >>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
>> >>>>>>>> storage
>> >>>>>>>>>>>>> system's
>> >>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
>> >>> Quality
>> >>>>>>>> of
>> >>>>>>>>>>>>>>>>> Service).
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
>> dynamically
>> >>>>>>>> create
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>> delete
>> >>>>>>>>>>>>>>>>>> volumes as needed.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> The problem that the storage framework did not address
>> is
>> >>> in
>> >>>>>>>>>>> creating
>> >>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>> >>>>>>>> performing an
>> >>>>>>>>>>>>>>>>>> attach/detach.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so.
>> I've
>> >>>>>>>> got
>> >>>>>>>>>>>>>>>>> XenServer
>> >>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox
>> >>> and
>> >>>>>>>> can
>> >>>>>>>>>>>>> submit
>> >>>>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Does that help a bit? :)
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
>> agnostic.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
>> agent
>> >>>>>>>> code).
>> >>>>>>>>>>> The
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
>> >>> volume
>> >>>>>>>> as
>> >>>>>>>>>>>>>>>>> needed,
>> >>>>>>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
>> hypervisor,
>> >>>>>>>> the
>> >>>>>>>>>>> attach
>> >>>>>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
>> invoked on
>> >>>>>>>>>>> zone-wide
>> >>>>>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>> (where the volume has just been created) and create,
>> >>> say, a
>> >>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
>> VMware) to
>> >>>>>>>> make
>> >>>>>>>>>>> use
>> >>>>>>>>>>>>>>>>> of the
>> >>>>>>>>>>>>>>>>>>> volume that was just created.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making the
>> >>>>>>>> attach
>> >>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>> in the agent code.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Does that clear it up?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>> >>>>>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
>> hypervisor
>> >>>>>>>>>>> specific?
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
>> hypervisors
>> >>>>>>>> that
>> >>>>>>>>>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to
>> get
>> >>> KVM
>> >>>>>>>> in.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
>> Depending on
>> >>>>>>>> what
>> >>>>>>>>>>> we do
>> >>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>> >>>>>>>> VMware in
>> >>>>>>>>>>> my
>> >>>>>>>>>>>>>>>>>>>> sandbox,
>> >>>>>>>>>>>>>>>>>>>>> as well.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to
>> Disk
>> >>>>>>>>>>> Offerings.
>> >>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>> plan to
>> >>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
>> Damage
>> >>> <
>> >>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>> >>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch
>> cycle
>> >>>>>>>> post
>> >>>>>>>>>>> 4.2?
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non
>> issue
>> >>>>>>>> as I
>> >>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>> >>>>>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> John,
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
>> >>> users
>> >>>>>>>> need
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>>>> calculate
>> >>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>> >>>>>>>> service_offering
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
>> calculation.
>> >>> Of
>> >>>>>>>> course
>> >>>>>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>>>>> better
>> >>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
>> individual
>> >>>>>>>> volume,
>> >>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>> >>>>>>>> later. Do
>> >>>>>>>>>>> you
>> >>>>>>>>>>>>>>>>>>>> think it
>> >>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>> >>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
>> >>> adding
>> >>>>>>>> more.
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
>> statistics
>> >>>>>>>> rather
>> >>>>>>>>>>> than
>> >>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you
>> have
>> >>>>>>>> time.
>> >>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>> >>> read/write
>> >>>>>>>> for
>> >>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more
>> >>> for
>> >>>>>>>>>>> volumes
>> >>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
>> operations (
>> >>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine
>> our
>> >>>>>>>> users
>> >>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>> like
>> >>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>> >>>>>>>> operator
>> >>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>>>>> such a
>> >>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
>> supports
>> >>>>>>>> setting
>> >>>>>>>>>>>>>>>>> IOPS/BPS
>> >>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>> >>>>>>>> command
>> >>>>>>>>>>> line.
>> >>>>>>>>>>>>>>>>>>>>>> However,
>> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
>> parameters
>> >>>>>>>> of a
>> >>>>>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>>>>>>>>> offering
>> >>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize
>> >>> for
>> >>>>>>>> being
>> >>>>>>>>>>>>>>>>>> unclear.
>> >>>>>>>>>>>>>>>>>>>>>> Can
>> >>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a
>> set
>> >>> of
>> >>>>>>>>>>>>>>>>> interfaces
>> >>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>> >>> resource?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>> >>>>>>>> commit
>> >>>>>>>>>>> Mike's
>> >>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>> >>>>>>>>>>> limitation in
>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response
>> to
>> >>>>>>>> Mike
>> >>>>>>>>>>>>>>>>> regarding
>> >>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
>> >>> that
>> >>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I
>> think
>> >>>>>>>> we need
>> >>>>>>>>>>>>>>>>> closer
>> >>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have
>> >>> have
>> >>>>>>>>>>> solidfire
>> >>>>>>>>>>>>>>>>>> track
>> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
>> following:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>> >>>>>>>> captured
>> >>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>> common
>> >>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>> >>>>>>>> expressing
>> >>>>>>>>>>>>>>>>>> offerings).
>> >>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>> >>>>>>>> reasoning
>> >>>>>>>>>>>>>>>>> about
>> >>>>>>>>>>>>>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>> >>>>>>>> configurations
>> >>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>> >>>>>>>> particular,
>> >>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>> >>>>>>>> configured
>> >>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>> a VM
>> >>>>>>>>>>>>>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>> >>>>>>>> Another
>> >>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>> >>> storage
>> >>>>>>>>>>> device at
>> >>>>>>>>>>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
>> device
>> >>>>>>>>>>> introduces
>> >>>>>>>>>>>>>>>>>> logical
>> >>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling
>> the
>> >>>>>>>> max
>> >>>>>>>>>>>>>>>>> number of
>> >>>>>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from
>> the
>> >>>>>>>> storage
>> >>>>>>>>>>>>>>>>> system
>> >>>>>>>>>>>>>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>> >>>>>>>>>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
>> merged
>> >>>>>>>> until all
>> >>>>>>>>>>>>>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to
>> turn
>> >>>>>>>> over to
>> >>>>>>>>>>>>>>>>> test.
>> >>>>>>>>>>>>>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>> >>>>>>>> addressed
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>> >>>>>>>> satisfied.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment
>> the
>> >>>>>>>>>>> SolidFire
>> >>>>>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>> (
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>> >>>>>>>> understand
>> >>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like
>> to
>> >>>>>>>> ensure
>> >>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>> >>>>>>>> SolidFire are
>> >>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> >>>>>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason
>> till
>> >>>>>>>> Monday,
>> >>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
>> >>> again
>> >>>>>>>>>>> that we
>> >>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
>> never
>> >>>>>>>> release
>> >>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>> version
>> >>>>>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling
>> >>> and
>> >>>>>>>> that's
>> >>>>>>>>>>>>>>>>> great!
>> >>>>>>>>>>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day
>> >>> #1.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
>> support
>> >>>>>>>> read
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because
>> I
>> >>>>>>>> don't see
>> >>>>>>>>>>>>>>>>> how we
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>> >>> Therefore, we
>> >>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
>> backwards
>> >>>>>>>>>>>>>>>>> compatibility.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now
>> so
>> >>>>>>>> that it
>> >>>>>>>>>>>>>>>>> will go
>> >>>>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before
>> the
>> >>> 4.2
>> >>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead
>> of
>> >>>>>>>> Total
>> >>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze
>> date.
>> >>>>>>>> But if
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>> window
>> >>>>>>>>>>>>>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary
>> >>> to do
>> >>>>>>>>>>> that.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>> >>> association
>> >>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
>> Hollander <
>> >>>>>>>>>>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>> >>>>>>>> decided to
>> >>>>>>>>>>>>>>>>> ignore
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because
>> the
>> >>>>>>>> network
>> >>>>>>>>>>>>>>>>>> throttling
>> >>>>>>>>>>>>>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
>> bytes as
>> >>>>>>>> well.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk
>> I/O
>> >>>>>>>>>>> completely
>> >>>>>>>>>>>>>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>> >>> situations
>> >>>>>>>> then
>> >>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work.
>> It
>> >>>>>>>> could be
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze,
>> I
>> >>> will
>> >>>>>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if
>> it
>> >>>>>>>> goes
>> >>>>>>>>>>> into
>> >>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release
>> with
>> >>>>>>>> only
>> >>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>> IOps
>> >>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
>> branch
>> >>> into
>> >>>>>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in
>> 48
>> >>>>>>>> hours.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
>> storage
>> >>>>>>>> device
>> >>>>>>>>>>>>>>>>> (local
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation
>> of
>> >>>>>>>> device
>> >>>>>>>>>>>>>>>>> (such as
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect
>> >>> the
>> >>>>>>>> disk
>> >>>>>>>>>>>>>>>>>>>> performance
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and
>> limit
>> >>>>>>>> the
>> >>>>>>>>>>> disk
>> >>>>>>>>>>>>>>>>> I/O
>> >>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
>> difference
>> >>>>>>>> between
>> >>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
>> different
>> >>>>>>>> rate
>> >>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>> >>> collect
>> >>>>>>>> both
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go
>> for
>> >>> a
>> >>>>>>>> global
>> >>>>>>>>>>>>>>>>> value.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
>> then
>> >>> Read
>> >>>>>>>>>>> IOps,
>> >>>>>>>>>>>>>>>>> so it
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
>> would
>> >>> set
>> >>>>>>>> a
>> >>>>>>>>>>> lower
>> >>>>>>>>>>>>>>>>>> value
>> >>>>>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
>> >>> think
>> >>>>>>>> it
>> >>>>>>>>>>> would
>> >>>>>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place
>> to
>> >>>>>>>> support
>> >>>>>>>>>>>>>>>>> both,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
>> different
>> >>>>>>>> values
>> >>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that
>> as
>> >>> the
>> >>>>>>>>>>> total
>> >>>>>>>>>>>>>>>>>> limit.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this
>> way?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>> >>> disk_offering,
>> >>>>>>>> and
>> >>>>>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>>>>>>>>>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>
>> >>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>> >>> success?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
>> integration)
>> >>> is
>> >>>>>>>>>>> included
>> >>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>> >>> potential
>> >>>>>>>>>>>>>>>>> regressions?
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>> >>> CloudStack
>> >>>>>>>> UI.
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>> >>>>>>>> migrate,
>> >>>>>>>>>>>>>>>>> restore
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>> >>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>
>> >>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>> >>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>> cloud<
>> >>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the cloud<
>> >>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the cloud<
>> >>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the cloud<
>> >>> http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the
>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>> *â„¢*
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >> *â„¢*
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"The administrator will allocate a SAN volume for CloudStack's use onto
which CloudStack volumes will be created."
I think we crossed e-mails. :)
Check out my recent e-mail on this.
On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> You are coming to the part which concerns me -- concepts from the
> hypervisor are leaking into storage layer.
>
> Thanks,
> -John
>
> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > The weird part is that the iSCSI type is today only used (as far as I
> know)
> > in regards to XenServer (when you have not PreSetup an SR).
> >
> > If you want to use your iSCSI volume from VMware, it uses the vmfs type.
> >
> > If you want to use your iSCSI volume from KVM, it uses the
> SharedMountPoint
> > type.
> >
> > So, I suppose mine and Edison's thinking here was to make a new type of
> > storage to describe this dynamic ability Edison added into the storage
> > framework. Maybe it should be more specificy, though: Dynamic_iSCSI
> versus,
> > say, Dynamic_FC.
> >
> >
> > On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >> wrote:
> >
> >> "The storage device itself shouldn't know or care that it is being used
> >> for a Xen SR -- simply be able to answer questions about it is storing."
> >>
> >> I see...so your concern here is that the SolidFire plug-in needs to call
> >> itself "Dynamic" storage so that the hypervisor logic knows to treat it
> as
> >> such.
> >>
> >> I'm totally open to removing that constraint and just calling it iSCSI
> or
> >> whatever. We would just need a way for the hypervisor attach logic to
> >> detect this new requirement and perform the necessary activities.
> >>
> >>
> >> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Mike,
> >>>
> >>> See my responses in-line.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >>> wrote:
> >>>
> >>>> I'm trying to picture this:
> >>>>
> >>>> "Finally, while CloudStack may be able to manage a device, an operator
> >>> may
> >>>> chose to leave it unmanaged by CloudStack (e.g. the device is shared
> by
> >>>> many services, and the operator has chosen to dedicate only a portion
> >>> of it
> >>>> to CloudStack). Does my reasoning make sense?"
> >>>>
> >>>> I guess I'm not sure how creating a SAN volume via the plug-in (before
> >>> an
> >>>> attach request to the hypervisor) would work unless the hypervisor
> >>> consumes
> >>>> the SAN volume in the form of, say, an SR.
> >>>
> >>> My thinking is that, independent of CloudStack, an operator allocates a
> >>> chunk of a SAN to CloudStack, and exposes it through a LUN. They
> simply
> >>> want to turn control of that LUN over to CloudStack, but not allow
> >>> CloudStack to allocate anymore LUNs.
> >>>
> >>>>
> >>>> As the attach logic stands prior to my changes, we would be passing
> in a
> >>>> SAN volume that does not have the necessary hypervisor support (like
> an
> >>> SR)
> >>>> and the logic will fail.
> >>>>
> >>>> Are you thinking we should maybe have the storage framework itself
> >>> detect
> >>>> that such a SAN volume needs support from the hypervisor side and have
> >>> it
> >>>> call into the agent code specifically to create the SR before the
> attach
> >>>> logic runs in the agent code?
> >>>
> >>> I think the hypervisor management plugin should have a rich enough
> >>> interface to storage to determine available for volume storage. For
> Xen,
> >>> this interface would allow the interrogation of the device to
> determine the
> >>> SR is present. The storage device itself shouldn't know or care that
> it
> >>> is being used for a Xen SR -- simply be able to answer questions about
> it
> >>> is storing.
> >>>
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com
> >>>>> wrote:
> >>>>
> >>>>> So, the flow is as follows:
> >>>>>
> >>>>> * The admin registers the SolidFire driver (which is a type of
> >>> so-called
> >>>>> Dynamic storage). Once this is done, a new Primary Storage shows up
> in
> >>> the
> >>>>> applicable zone.
> >>>>>
> >>>>> * The admin creates a Disk Offering that references the storage tag
> of
> >>> the
> >>>>> newly created Primary Storage.
> >>>>>
> >>>>> * The end user creates a CloudStack volume. This leads to a new row
> in
> >>> the
> >>>>> cloud.volumes table.
> >>>>>
> >>>>> * The end user attaches the CloudStack volume to a VM (attach disk).
> >>> This
> >>>>> leads to the storage framework calling the plug-in to create a new
> >>> volume
> >>>>> on its storage system (in my case, a SAN). The plug-in also updates
> the
> >>>>> cloud.volumes row with applicable data (like the IQN of the SAN
> >>> volume).
> >>>>> This plug-in code is only invoked if the CloudStack volume is in the
> >>>>> 'Allocated' state. After the attach, the volume will be in the
> 'Ready'
> >>>>> state (even after a detach disk) and the plug-in code will not be
> >>> called
> >>>>> again to create this SAN volume.
> >>>>>
> >>>>> * The hypervisor-attach logic is run and detects the CloudStack
> volume
> >>> to
> >>>>> attach needs "assistance" in the form of a hypervisor data structure
> >>> (ex.
> >>>>> an SR on XenServer).
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>>> "To ensure that we are in sync on terminology, volume, in these
> >>>>>> definitions, refers to the physical allocation on the device,
> >>> correct?"
> >>>>>>
> >>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>>>
> >>>>>> To refer to the 'volume' the end user can make in CloudStack, I try
> to
> >>>>>> use 'CloudStack volume'.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>
> >>>>>>> Hi John,
> >>>>>>>
> >>>>>>> What you say here may very well make sense, but I'm having a hard
> >>> time
> >>>>>>> envisioning it.
> >>>>>>>
> >>>>>>> Perhaps we should draw Edison in on this conversation as he was the
> >>>>>>> initial person to suggest the approach I took.
> >>>>>>>
> >>>>>>> What do you think?
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> It feels like we are combining two distinct concepts -- storage
> >>> device
> >>>>>>>> management and storage protocols. In both cases, we are
> >>> communicating with
> >>>>>>>> ISCSI, but one allows the system to create/delete volumes
> (Dynamic)
> >>> on the
> >>>>>>>> device while the other requires the volume to be volume to be
> >>> managed
> >>>>>>>> outside of the CloudStack context. To ensure that we are in sync
> on
> >>>>>>>> terminology, volume, in these definitions, refers to the physical
> >>>>>>>> allocation on the device, correct? Minimally, we must be able to
> >>>>>>>> communicate with a storage device to move bits from one place to
> >>> another,
> >>>>>>>> read bits, delete bits, etc. Optionally, a storage device may be
> >>> able to
> >>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
> >>> device
> >>>>>>>> onto which we store a Xen SR, and we can have a managed SolidFire
> >>> iSCSI
> >>>>>>>> device on which CloudStack is capable of allocating LUNs and
> storing
> >>>>>>>> volumes. Finally, while CloudStack may be able to manage a
> device,
> >>> an
> >>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
> >>> device is
> >>>>>>>> shared by many services, and the operator has chosen to dedicate
> >>> only a
> >>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
> >>>>>>>>
> >>>>>>>> Assuming my thoughts above are reasonable, it seems appropriate to
> >>>>>>>> strip the management concerns from StoragePoolType, add the notion
> >>> of a
> >>>>>>>> storage device with an attached driver that indicates whether or
> >>> not is
> >>>>>>>> managed by CloudStack, and establish a abstraction representing a
> >>> physical
> >>>>>>>> allocation on a device separate that is associated with a volume.
> >>> With
> >>>>>>>> these notions in place, hypervisor drivers can declare which
> >>> protocols they
> >>>>>>>> support and when they encounter a device managed by CloudStack,
> >>> utilize the
> >>>>>>>> management operations exposed by the driver to automate
> allocation.
> >>> If
> >>>>>>>> these thoughts/concepts make sense, then we can sit down and drill
> >>> down to
> >>>>>>>> a more detailed design.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>
> >>>>>>>>> Here is the difference between the current iSCSI type and the
> >>> Dynamic
> >>>>>>>> type:
> >>>>>>>>>
> >>>>>>>>> iSCSI type: The admin has to go in and create a Primary Storage
> >>> based
> >>>>>>>> on
> >>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
> exist
> >>> on
> >>>>>>>> the
> >>>>>>>>> storage system (it is pre-allocated). Future CloudStack volumes
> are
> >>>>>>>> created
> >>>>>>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>>>>>
> >>>>>>>>> Dynamic type: The admin has to go in and create Primary Storage
> >>> based
> >>>>>>>> on a
> >>>>>>>>> plug-in that will create and delete volumes on its storage system
> >>>>>>>>> dynamically (as is enabled via the storage framework). When a
> user
> >>>>>>>> wants to
> >>>>>>>>> attach a CloudStack volume that was created, the framework tells
> >>> the
> >>>>>>>>> plug-in to create a new volume. After this is done, the attach
> >>> logic
> >>>>>>>> for
> >>>>>>>>> the hypervisor in question is called. No hypervisor data
> structure
> >>>>>>>> exists
> >>>>>>>>> at this point because the volume was just created. The hypervisor
> >>> data
> >>>>>>>>> structure must be created.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> These are new terms, so I should probably have defined them up
> >>> front
> >>>>>>>> for
> >>>>>>>>>> you. :)
> >>>>>>>>>>
> >>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
> >>> creates a
> >>>>>>>>>> volume on a SAN), then a hypervisor data structure is created to
> >>>>>>>> consume
> >>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
> >>> structure
> >>>>>>>> is
> >>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
> >>>>>>>> hypervisor
> >>>>>>>>>> data structure as needed. In these cases, the attach logic
> assumes
> >>>>>>>> the
> >>>>>>>>>> hypervisor data structure is already in place and simply
> attaches
> >>>>>>>> the VDI
> >>>>>>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>>>>>
> >>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
> >>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on a
> SAN,
> >>>>>>>> but the
> >>>>>>>>>> SAN itself). The hypervisor data structure must be created when
> an
> >>>>>>>> attach
> >>>>>>>>>> volume is performed because these types of volumes have not been
> >>>>>>>> pre-hooked
> >>>>>>>>>> up to such a hypervisor data structure by an admin. Once the
> >>> attach
> >>>>>>>> logic
> >>>>>>>>>> creates, say, an SR on XenServer for this volume, it attaches
> the
> >>>>>>>> one and
> >>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> jburwell@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Mike,
> >>>>>>>>>>>
> >>>>>>>>>>> The current implementation of the Dynamic type attach behavior
> >>>>>>>> works in
> >>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
> Another
> >>>>>>>> way to
> >>>>>>>>>>> ask the question -- what is the definition of a Dynamic storage
> >>>>>>>> pool type?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> -John
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
> >>>>>>>> when you
> >>>>>>>>>>>> want to set up Primary Storage that is directly based on an
> >>> iSCSI
> >>>>>>>>>>> target.
> >>>>>>>>>>>> This allows you to skip the step of going to the hypervisor
> and
> >>>>>>>>>>> creating a
> >>>>>>>>>>>> storage repository based on that iSCSI target as CloudStack
> does
> >>>>>>>> that
> >>>>>>>>>>> part
> >>>>>>>>>>>> for you. I think this is only supported for XenServer. For all
> >>>>>>>> other
> >>>>>>>>>>>> hypervisors, you must first go to the hypervisor and perform
> >>> this
> >>>>>>>> step
> >>>>>>>>>>>> manually.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>> jburwell@basho.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Reading through the code, what is the difference between the
> >>>>>>>> ISCSI and
> >>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>> class
> >>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> RBD(true),
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up
> >>> by
> >>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
> >>> SolidFire)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> boolean shared;
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> return shared;
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> For example, let's say another storage company wants to
> >>>>>>>> implement a
> >>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
> would
> >>> be
> >>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>> zone-wide storage, as well. They would need only implement
> a
> >>>>>>>> storage
> >>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>>>>>> hypervisor-attach
> >>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
> specific.
> >>>>>>>> It is
> >>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
> >>> zone-wide
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
> >>> storage).
> >>>>>>>> This
> >>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
> >>> changes.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Edison created a new framework that supported the creation
> >>> and
> >>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with him
> in
> >>>>>>>> Portland
> >>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
> realized
> >>>>>>>> there
> >>>>>>>>>>> was
> >>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
> >>>>>>>> attach
> >>>>>>>>>>> logic
> >>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>> changed to recognize this new type of storage and create
> the
> >>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
> Storage
> >>>>>>>> layer
> >>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> understand or care about details. I expect to see the
> >>> Storage
> >>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>> expose a set of operations that can be composed/driven by
> >>> the
> >>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>> implementations to allocate space/create structures per
> >>> their
> >>>>>>>>>>> needs.
> >>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
> >>>>>>>> massive
> >>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>> problem that will make the system increasingly difficult
> to
> >>>>>>>>>>> maintain
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
> >>> SolidFire
> >>>>>>>> code
> >>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
> >>> XenServer,
> >>>>>>>> the
> >>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>> to first create a volume on the storage system, then go
> >>> into
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>> to set up a data structure to make use of the volume
> (ex.
> >>> a
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs
> and
> >>>>>>>> data
> >>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
> >>> longer
> >>>>>>>> be so
> >>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
> >>>>>>>> storage
> >>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
> >>> Quality
> >>>>>>>> of
> >>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
> dynamically
> >>>>>>>> create
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The problem that the storage framework did not address
> is
> >>> in
> >>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
> >>>>>>>> performing an
> >>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so.
> I've
> >>>>>>>> got
> >>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox
> >>> and
> >>>>>>>> can
> >>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
> agnostic.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
> >>>>>>>> code).
> >>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
> >>> volume
> >>>>>>>> as
> >>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
> hypervisor,
> >>>>>>>> the
> >>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked
> on
> >>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> (where the volume has just been created) and create,
> >>> say, a
> >>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware)
> to
> >>>>>>>> make
> >>>>>>>>>>> use
> >>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making the
> >>>>>>>> attach
> >>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
> hypervisor
> >>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
> hypervisors
> >>>>>>>> that
> >>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get
> >>> KVM
> >>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending
> on
> >>>>>>>> what
> >>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
> >>>>>>>> VMware in
> >>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to
> Disk
> >>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
> Damage
> >>> <
> >>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch
> cycle
> >>>>>>>> post
> >>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non
> issue
> >>>>>>>> as I
> >>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
> >>> users
> >>>>>>>> need
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>>>>>> service_offering
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation.
> >>> Of
> >>>>>>>> course
> >>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
> individual
> >>>>>>>> volume,
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
> >>>>>>>> later. Do
> >>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
> >>> adding
> >>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
> statistics
> >>>>>>>> rather
> >>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you
> have
> >>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
> >>> read/write
> >>>>>>>> for
> >>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more
> >>> for
> >>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
> operations (
> >>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
> >>>>>>>> users
> >>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
> >>>>>>>> operator
> >>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
> >>>>>>>> setting
> >>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
> >>>>>>>> command
> >>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
> parameters
> >>>>>>>> of a
> >>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize
> >>> for
> >>>>>>>> being
> >>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a
> set
> >>> of
> >>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
> >>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
> >>>>>>>> commit
> >>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
> >>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response
> to
> >>>>>>>> Mike
> >>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
> >>> that
> >>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I
> think
> >>>>>>>> we need
> >>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have
> >>> have
> >>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
> >>>>>>>> captured
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
> >>>>>>>> expressing
> >>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
> >>>>>>>> reasoning
> >>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> >>>>>>>> configurations
> >>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
> >>>>>>>> particular,
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
> >>>>>>>> configured
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
> >>>>>>>> Another
> >>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
> >>> storage
> >>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
> >>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling
> the
> >>>>>>>> max
> >>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from
> the
> >>>>>>>> storage
> >>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
> >>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
> >>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to
> turn
> >>>>>>>> over to
> >>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
> >>>>>>>> addressed
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
> >>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment
> the
> >>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
> >>>>>>>> understand
> >>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like
> to
> >>>>>>>> ensure
> >>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
> >>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason
> till
> >>>>>>>> Monday,
> >>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
> >>> again
> >>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
> never
> >>>>>>>> release
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling
> >>> and
> >>>>>>>> that's
> >>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day
> >>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
> support
> >>>>>>>> read
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
> >>>>>>>> don't see
> >>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
> >>> Therefore, we
> >>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
> backwards
> >>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now
> so
> >>>>>>>> that it
> >>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the
> >>> 4.2
> >>>>>>>>>>> release
> >>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
> >>>>>>>> Total
> >>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze
> date.
> >>>>>>>> But if
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary
> >>> to do
> >>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> >>> association
> >>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
> Hollander <
> >>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
> >>>>>>>> decided to
> >>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
> >>>>>>>> network
> >>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes
> as
> >>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk
> I/O
> >>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
> >>> situations
> >>>>>>>> then
> >>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work.
> It
> >>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I
> >>> will
> >>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if
> it
> >>>>>>>> goes
> >>>>>>>>>>> into
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release
> with
> >>>>>>>> only
> >>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch
> >>> into
> >>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in
> 48
> >>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
> storage
> >>>>>>>> device
> >>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation
> of
> >>>>>>>> device
> >>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect
> >>> the
> >>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and
> limit
> >>>>>>>> the
> >>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
> difference
> >>>>>>>> between
> >>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
> different
> >>>>>>>> rate
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
> >>> collect
> >>>>>>>> both
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go
> for
> >>> a
> >>>>>>>> global
> >>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then
> >>> Read
> >>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would
> >>> set
> >>>>>>>> a
> >>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
> >>> think
> >>>>>>>> it
> >>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place
> to
> >>>>>>>> support
> >>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
> different
> >>>>>>>> values
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that
> as
> >>> the
> >>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this
> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
> >>> disk_offering,
> >>>>>>>> and
> >>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
> >>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration)
> >>> is
> >>>>>>>>>>> included
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
> >>> potential
> >>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
> >>> CloudStack
> >>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
> >>>>>>>> migrate,
> >>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>> cloud<
> >>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>> cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >>> http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
You are coming to the part which concerns me -- concepts from the hypervisor are leaking into storage layer.
Thanks,
-John
On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> The weird part is that the iSCSI type is today only used (as far as I know)
> in regards to XenServer (when you have not PreSetup an SR).
>
> If you want to use your iSCSI volume from VMware, it uses the vmfs type.
>
> If you want to use your iSCSI volume from KVM, it uses the SharedMountPoint
> type.
>
> So, I suppose mine and Edison's thinking here was to make a new type of
> storage to describe this dynamic ability Edison added into the storage
> framework. Maybe it should be more specificy, though: Dynamic_iSCSI versus,
> say, Dynamic_FC.
>
>
> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
>> wrote:
>
>> "The storage device itself shouldn't know or care that it is being used
>> for a Xen SR -- simply be able to answer questions about it is storing."
>>
>> I see...so your concern here is that the SolidFire plug-in needs to call
>> itself "Dynamic" storage so that the hypervisor logic knows to treat it as
>> such.
>>
>> I'm totally open to removing that constraint and just calling it iSCSI or
>> whatever. We would just need a way for the hypervisor attach logic to
>> detect this new requirement and perform the necessary activities.
>>
>>
>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> See my responses in-line.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>>> I'm trying to picture this:
>>>>
>>>> "Finally, while CloudStack may be able to manage a device, an operator
>>> may
>>>> chose to leave it unmanaged by CloudStack (e.g. the device is shared by
>>>> many services, and the operator has chosen to dedicate only a portion
>>> of it
>>>> to CloudStack). Does my reasoning make sense?"
>>>>
>>>> I guess I'm not sure how creating a SAN volume via the plug-in (before
>>> an
>>>> attach request to the hypervisor) would work unless the hypervisor
>>> consumes
>>>> the SAN volume in the form of, say, an SR.
>>>
>>> My thinking is that, independent of CloudStack, an operator allocates a
>>> chunk of a SAN to CloudStack, and exposes it through a LUN. They simply
>>> want to turn control of that LUN over to CloudStack, but not allow
>>> CloudStack to allocate anymore LUNs.
>>>
>>>>
>>>> As the attach logic stands prior to my changes, we would be passing in a
>>>> SAN volume that does not have the necessary hypervisor support (like an
>>> SR)
>>>> and the logic will fail.
>>>>
>>>> Are you thinking we should maybe have the storage framework itself
>>> detect
>>>> that such a SAN volume needs support from the hypervisor side and have
>>> it
>>>> call into the agent code specifically to create the SR before the attach
>>>> logic runs in the agent code?
>>>
>>> I think the hypervisor management plugin should have a rich enough
>>> interface to storage to determine available for volume storage. For Xen,
>>> this interface would allow the interrogation of the device to determine the
>>> SR is present. The storage device itself shouldn't know or care that it
>>> is being used for a Xen SR -- simply be able to answer questions about it
>>> is storing.
>>>
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com
>>>>> wrote:
>>>>
>>>>> So, the flow is as follows:
>>>>>
>>>>> * The admin registers the SolidFire driver (which is a type of
>>> so-called
>>>>> Dynamic storage). Once this is done, a new Primary Storage shows up in
>>> the
>>>>> applicable zone.
>>>>>
>>>>> * The admin creates a Disk Offering that references the storage tag of
>>> the
>>>>> newly created Primary Storage.
>>>>>
>>>>> * The end user creates a CloudStack volume. This leads to a new row in
>>> the
>>>>> cloud.volumes table.
>>>>>
>>>>> * The end user attaches the CloudStack volume to a VM (attach disk).
>>> This
>>>>> leads to the storage framework calling the plug-in to create a new
>>> volume
>>>>> on its storage system (in my case, a SAN). The plug-in also updates the
>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
>>> volume).
>>>>> This plug-in code is only invoked if the CloudStack volume is in the
>>>>> 'Allocated' state. After the attach, the volume will be in the 'Ready'
>>>>> state (even after a detach disk) and the plug-in code will not be
>>> called
>>>>> again to create this SAN volume.
>>>>>
>>>>> * The hypervisor-attach logic is run and detects the CloudStack volume
>>> to
>>>>> attach needs "assistance" in the form of a hypervisor data structure
>>> (ex.
>>>>> an SR on XenServer).
>>>>>
>>>>>
>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> "To ensure that we are in sync on terminology, volume, in these
>>>>>> definitions, refers to the physical allocation on the device,
>>> correct?"
>>>>>>
>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>>>
>>>>>> To refer to the 'volume' the end user can make in CloudStack, I try to
>>>>>> use 'CloudStack volume'.
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> Hi John,
>>>>>>>
>>>>>>> What you say here may very well make sense, but I'm having a hard
>>> time
>>>>>>> envisioning it.
>>>>>>>
>>>>>>> Perhaps we should draw Edison in on this conversation as he was the
>>>>>>> initial person to suggest the approach I took.
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> It feels like we are combining two distinct concepts -- storage
>>> device
>>>>>>>> management and storage protocols. In both cases, we are
>>> communicating with
>>>>>>>> ISCSI, but one allows the system to create/delete volumes (Dynamic)
>>> on the
>>>>>>>> device while the other requires the volume to be volume to be
>>> managed
>>>>>>>> outside of the CloudStack context. To ensure that we are in sync on
>>>>>>>> terminology, volume, in these definitions, refers to the physical
>>>>>>>> allocation on the device, correct? Minimally, we must be able to
>>>>>>>> communicate with a storage device to move bits from one place to
>>> another,
>>>>>>>> read bits, delete bits, etc. Optionally, a storage device may be
>>> able to
>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
>>> device
>>>>>>>> onto which we store a Xen SR, and we can have a managed SolidFire
>>> iSCSI
>>>>>>>> device on which CloudStack is capable of allocating LUNs and storing
>>>>>>>> volumes. Finally, while CloudStack may be able to manage a device,
>>> an
>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
>>> device is
>>>>>>>> shared by many services, and the operator has chosen to dedicate
>>> only a
>>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>>>>>
>>>>>>>> Assuming my thoughts above are reasonable, it seems appropriate to
>>>>>>>> strip the management concerns from StoragePoolType, add the notion
>>> of a
>>>>>>>> storage device with an attached driver that indicates whether or
>>> not is
>>>>>>>> managed by CloudStack, and establish a abstraction representing a
>>> physical
>>>>>>>> allocation on a device separate that is associated with a volume.
>>> With
>>>>>>>> these notions in place, hypervisor drivers can declare which
>>> protocols they
>>>>>>>> support and when they encounter a device managed by CloudStack,
>>> utilize the
>>>>>>>> management operations exposed by the driver to automate allocation.
>>> If
>>>>>>>> these thoughts/concepts make sense, then we can sit down and drill
>>> down to
>>>>>>>> a more detailed design.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>
>>>>>>>>> Here is the difference between the current iSCSI type and the
>>> Dynamic
>>>>>>>> type:
>>>>>>>>>
>>>>>>>>> iSCSI type: The admin has to go in and create a Primary Storage
>>> based
>>>>>>>> on
>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must exist
>>> on
>>>>>>>> the
>>>>>>>>> storage system (it is pre-allocated). Future CloudStack volumes are
>>>>>>>> created
>>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>>>>>>>>
>>>>>>>>> Dynamic type: The admin has to go in and create Primary Storage
>>> based
>>>>>>>> on a
>>>>>>>>> plug-in that will create and delete volumes on its storage system
>>>>>>>>> dynamically (as is enabled via the storage framework). When a user
>>>>>>>> wants to
>>>>>>>>> attach a CloudStack volume that was created, the framework tells
>>> the
>>>>>>>>> plug-in to create a new volume. After this is done, the attach
>>> logic
>>>>>>>> for
>>>>>>>>> the hypervisor in question is called. No hypervisor data structure
>>>>>>>> exists
>>>>>>>>> at this point because the volume was just created. The hypervisor
>>> data
>>>>>>>>> structure must be created.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> These are new terms, so I should probably have defined them up
>>> front
>>>>>>>> for
>>>>>>>>>> you. :)
>>>>>>>>>>
>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>>> creates a
>>>>>>>>>> volume on a SAN), then a hypervisor data structure is created to
>>>>>>>> consume
>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>>> structure
>>>>>>>> is
>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>>>>>>>> hypervisor
>>>>>>>>>> data structure as needed. In these cases, the attach logic assumes
>>>>>>>> the
>>>>>>>>>> hypervisor data structure is already in place and simply attaches
>>>>>>>> the VDI
>>>>>>>>>> on the hypervisor data structure to the VM in question.
>>>>>>>>>>
>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN,
>>>>>>>> but the
>>>>>>>>>> SAN itself). The hypervisor data structure must be created when an
>>>>>>>> attach
>>>>>>>>>> volume is performed because these types of volumes have not been
>>>>>>>> pre-hooked
>>>>>>>>>> up to such a hypervisor data structure by an admin. Once the
>>> attach
>>>>>>>> logic
>>>>>>>>>> creates, say, an SR on XenServer for this volume, it attaches the
>>>>>>>> one and
>>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mike,
>>>>>>>>>>>
>>>>>>>>>>> The current implementation of the Dynamic type attach behavior
>>>>>>>> works in
>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference. Another
>>>>>>>> way to
>>>>>>>>>>> ask the question -- what is the definition of a Dynamic storage
>>>>>>>> pool type?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>>
>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
>>>>>>>> when you
>>>>>>>>>>>> want to set up Primary Storage that is directly based on an
>>> iSCSI
>>>>>>>>>>> target.
>>>>>>>>>>>> This allows you to skip the step of going to the hypervisor and
>>>>>>>>>>> creating a
>>>>>>>>>>>> storage repository based on that iSCSI target as CloudStack does
>>>>>>>> that
>>>>>>>>>>> part
>>>>>>>>>>>> for you. I think this is only supported for XenServer. For all
>>>>>>>> other
>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and perform
>>> this
>>>>>>>> step
>>>>>>>>>>>> manually.
>>>>>>>>>>>>
>>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>>> jburwell@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Reading through the code, what is the difference between the
>>>>>>>> ISCSI and
>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>>> Storage.StoragePoolType
>>>>>>>>>>> class
>>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up
>>> by
>>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>>> SolidFire)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> For example, let's say another storage company wants to
>>>>>>>> implement a
>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It would
>>> be
>>>>>>>>>>> dynamic,
>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only implement a
>>>>>>>> storage
>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>>>>>> hypervisor-attach
>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific.
>>>>>>>> It is
>>>>>>>>>>> not.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>>> zone-wide
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>>> storage).
>>>>>>>> This
>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>>> changes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Edison created a new framework that supported the creation
>>> and
>>>>>>>>>>> deletion
>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
>>>>>>>> Portland
>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We realized
>>>>>>>> there
>>>>>>>>>>> was
>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
>>>>>>>> attach
>>>>>>>>>>> logic
>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>> changed to recognize this new type of storage and create the
>>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the Storage
>>>>>>>> layer
>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> understand or care about details. I expect to see the
>>> Storage
>>>>>>>>>>>>> services
>>>>>>>>>>>>>>>>> expose a set of operations that can be composed/driven by
>>> the
>>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>>> implementations to allocate space/create structures per
>>> their
>>>>>>>>>>> needs.
>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>>>>>>>> massive
>>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>>> problem that will make the system increasingly difficult to
>>>>>>>>>>> maintain
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>>> SolidFire
>>>>>>>> code
>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>>> XenServer,
>>>>>>>> the
>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>> to first create a volume on the storage system, then go
>>> into
>>>>>>>> the
>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the volume (ex.
>>> a
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
>>>>>>>> data
>>>>>>>>>>> disks
>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
>>> longer
>>>>>>>> be so
>>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
>>>>>>>> storage
>>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
>>> Quality
>>>>>>>> of
>>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
>>>>>>>> create
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The problem that the storage framework did not address is
>>> in
>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>>>>>>>> performing an
>>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so. I've
>>>>>>>> got
>>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox
>>> and
>>>>>>>> can
>>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>>>>>>>> code).
>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
>>> volume
>>>>>>>> as
>>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a hypervisor,
>>>>>>>> the
>>>>>>>>>>> attach
>>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and create,
>>> say, a
>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
>>>>>>>> make
>>>>>>>>>>> use
>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making the
>>>>>>>> attach
>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
>>>>>>>> that
>>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get
>>> KVM
>>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
>>>>>>>> what
>>>>>>>>>>> we do
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>>>>>>> VMware in
>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage
>>> <
>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
>>>>>>>> post
>>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue
>>>>>>>> as I
>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
>>> users
>>>>>>>> need
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>>>>>> service_offering
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation.
>>> Of
>>>>>>>> course
>>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>>>>>>>> volume,
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>>>>>>> later. Do
>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
>>> adding
>>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>>>>>>>> rather
>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>>> read/write
>>>>>>>> for
>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more
>>> for
>>>>>>>>>>> volumes
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
>>>>>>>> users
>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>>>>>>>> operator
>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>>>>>>>> setting
>>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>>>>>>> command
>>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters
>>>>>>>> of a
>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize
>>> for
>>>>>>>> being
>>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set
>>> of
>>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>>> resource?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>>>>>>>> commit
>>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
>>>>>>>> Mike
>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
>>> that
>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think
>>>>>>>> we need
>>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have
>>> have
>>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>>>>>>> captured
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>>>>>>> expressing
>>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>>>>>>> reasoning
>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>>>>>>> configurations
>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>>>>>>> particular,
>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>>>>>>> configured
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>>>>>>> Another
>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>>> storage
>>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the
>>>>>>>> max
>>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>>>>>>>> storage
>>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>>>>>>>> over to
>>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>>>>>>> addressed
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>>>>>>> understand
>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>>>>>>>> ensure
>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>>>>>>>> Monday,
>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
>>> again
>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>>>>>>>> release
>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling
>>> and
>>>>>>>> that's
>>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day
>>> #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
>>>>>>>> read
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>>>>>>>> don't see
>>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>>> Therefore, we
>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>>>>>>>> that it
>>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the
>>> 4.2
>>>>>>>>>>> release
>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
>>>>>>>> Total
>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
>>>>>>>> But if
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary
>>> to do
>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>>> association
>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>>>>>>>> decided to
>>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>>>>>>>> network
>>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>>> situations
>>>>>>>> then
>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I
>>> will
>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
>>>>>>>> goes
>>>>>>>>>>> into
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
>>>>>>>> only
>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch
>>> into
>>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>>>>>>>> device
>>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>>>>>>>> device
>>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect
>>> the
>>>>>>>> disk
>>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit
>>>>>>>> the
>>>>>>>>>>> disk
>>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>>>>>>>> between
>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
>>>>>>>> rate
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>>> collect
>>>>>>>> both
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for
>>> a
>>>>>>>> global
>>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then
>>> Read
>>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would
>>> set
>>>>>>>> a
>>>>>>>>>>> lower
>>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
>>> think
>>>>>>>> it
>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>>>>>>>> support
>>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>>>>>>>> values
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as
>>> the
>>>>>>>>>>> total
>>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>>> disk_offering,
>>>>>>>> and
>>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration)
>>> is
>>>>>>>>>>> included
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>>> potential
>>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>>> CloudStack
>>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>>>>>>> migrate,
>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
The weird part is that the iSCSI type is today only used (as far as I know)
in regards to XenServer (when you have not PreSetup an SR).
If you want to use your iSCSI volume from VMware, it uses the vmfs type.
If you want to use your iSCSI volume from KVM, it uses the SharedMountPoint
type.
So, I suppose mine and Edison's thinking here was to make a new type of
storage to describe this dynamic ability Edison added into the storage
framework. Maybe it should be more specificy, though: Dynamic_iSCSI versus,
say, Dynamic_FC.
On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> "The storage device itself shouldn't know or care that it is being used
> for a Xen SR -- simply be able to answer questions about it is storing."
>
> I see...so your concern here is that the SolidFire plug-in needs to call
> itself "Dynamic" storage so that the hypervisor logic knows to treat it as
> such.
>
> I'm totally open to removing that constraint and just calling it iSCSI or
> whatever. We would just need a way for the hypervisor attach logic to
> detect this new requirement and perform the necessary activities.
>
>
> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> See my responses in-line.
>>
>> Thanks,
>> -John
>>
>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > I'm trying to picture this:
>> >
>> > "Finally, while CloudStack may be able to manage a device, an operator
>> may
>> > chose to leave it unmanaged by CloudStack (e.g. the device is shared by
>> > many services, and the operator has chosen to dedicate only a portion
>> of it
>> > to CloudStack). Does my reasoning make sense?"
>> >
>> > I guess I'm not sure how creating a SAN volume via the plug-in (before
>> an
>> > attach request to the hypervisor) would work unless the hypervisor
>> consumes
>> > the SAN volume in the form of, say, an SR.
>>
>> My thinking is that, independent of CloudStack, an operator allocates a
>> chunk of a SAN to CloudStack, and exposes it through a LUN. They simply
>> want to turn control of that LUN over to CloudStack, but not allow
>> CloudStack to allocate anymore LUNs.
>>
>> >
>> > As the attach logic stands prior to my changes, we would be passing in a
>> > SAN volume that does not have the necessary hypervisor support (like an
>> SR)
>> > and the logic will fail.
>> >
>> > Are you thinking we should maybe have the storage framework itself
>> detect
>> > that such a SAN volume needs support from the hypervisor side and have
>> it
>> > call into the agent code specifically to create the SR before the attach
>> > logic runs in the agent code?
>>
>> I think the hypervisor management plugin should have a rich enough
>> interface to storage to determine available for volume storage. For Xen,
>> this interface would allow the interrogation of the device to determine the
>> SR is present. The storage device itself shouldn't know or care that it
>> is being used for a Xen SR -- simply be able to answer questions about it
>> is storing.
>>
>> >
>> >
>> > On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>> >> wrote:
>> >
>> >> So, the flow is as follows:
>> >>
>> >> * The admin registers the SolidFire driver (which is a type of
>> so-called
>> >> Dynamic storage). Once this is done, a new Primary Storage shows up in
>> the
>> >> applicable zone.
>> >>
>> >> * The admin creates a Disk Offering that references the storage tag of
>> the
>> >> newly created Primary Storage.
>> >>
>> >> * The end user creates a CloudStack volume. This leads to a new row in
>> the
>> >> cloud.volumes table.
>> >>
>> >> * The end user attaches the CloudStack volume to a VM (attach disk).
>> This
>> >> leads to the storage framework calling the plug-in to create a new
>> volume
>> >> on its storage system (in my case, a SAN). The plug-in also updates the
>> >> cloud.volumes row with applicable data (like the IQN of the SAN
>> volume).
>> >> This plug-in code is only invoked if the CloudStack volume is in the
>> >> 'Allocated' state. After the attach, the volume will be in the 'Ready'
>> >> state (even after a detach disk) and the plug-in code will not be
>> called
>> >> again to create this SAN volume.
>> >>
>> >> * The hypervisor-attach logic is run and detects the CloudStack volume
>> to
>> >> attach needs "assistance" in the form of a hypervisor data structure
>> (ex.
>> >> an SR on XenServer).
>> >>
>> >>
>> >> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com> wrote:
>> >>
>> >>> "To ensure that we are in sync on terminology, volume, in these
>> >>> definitions, refers to the physical allocation on the device,
>> correct?"
>> >>>
>> >>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>> >>>
>> >>> To refer to the 'volume' the end user can make in CloudStack, I try to
>> >>> use 'CloudStack volume'.
>> >>>
>> >>>
>> >>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> >>> mike.tutkowski@solidfire.com> wrote:
>> >>>
>> >>>> Hi John,
>> >>>>
>> >>>> What you say here may very well make sense, but I'm having a hard
>> time
>> >>>> envisioning it.
>> >>>>
>> >>>> Perhaps we should draw Edison in on this conversation as he was the
>> >>>> initial person to suggest the approach I took.
>> >>>>
>> >>>> What do you think?
>> >>>>
>> >>>> Thanks!
>> >>>>
>> >>>>
>> >>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
>> >wrote:
>> >>>>
>> >>>>> Mike,
>> >>>>>
>> >>>>> It feels like we are combining two distinct concepts -- storage
>> device
>> >>>>> management and storage protocols. In both cases, we are
>> communicating with
>> >>>>> ISCSI, but one allows the system to create/delete volumes (Dynamic)
>> on the
>> >>>>> device while the other requires the volume to be volume to be
>> managed
>> >>>>> outside of the CloudStack context. To ensure that we are in sync on
>> >>>>> terminology, volume, in these definitions, refers to the physical
>> >>>>> allocation on the device, correct? Minimally, we must be able to
>> >>>>> communicate with a storage device to move bits from one place to
>> another,
>> >>>>> read bits, delete bits, etc. Optionally, a storage device may be
>> able to
>> >>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
>> device
>> >>>>> onto which we store a Xen SR, and we can have a managed SolidFire
>> iSCSI
>> >>>>> device on which CloudStack is capable of allocating LUNs and storing
>> >>>>> volumes. Finally, while CloudStack may be able to manage a device,
>> an
>> >>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
>> device is
>> >>>>> shared by many services, and the operator has chosen to dedicate
>> only a
>> >>>>> portion of it to CloudStack). Does my reasoning make sense?
>> >>>>>
>> >>>>> Assuming my thoughts above are reasonable, it seems appropriate to
>> >>>>> strip the management concerns from StoragePoolType, add the notion
>> of a
>> >>>>> storage device with an attached driver that indicates whether or
>> not is
>> >>>>> managed by CloudStack, and establish a abstraction representing a
>> physical
>> >>>>> allocation on a device separate that is associated with a volume.
>> With
>> >>>>> these notions in place, hypervisor drivers can declare which
>> protocols they
>> >>>>> support and when they encounter a device managed by CloudStack,
>> utilize the
>> >>>>> management operations exposed by the driver to automate allocation.
>> If
>> >>>>> these thoughts/concepts make sense, then we can sit down and drill
>> down to
>> >>>>> a more detailed design.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> -John
>> >>>>>
>> >>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>
>> >>>>>> Here is the difference between the current iSCSI type and the
>> Dynamic
>> >>>>> type:
>> >>>>>>
>> >>>>>> iSCSI type: The admin has to go in and create a Primary Storage
>> based
>> >>>>> on
>> >>>>>> the iSCSI type. At this point in time, the iSCSI volume must exist
>> on
>> >>>>> the
>> >>>>>> storage system (it is pre-allocated). Future CloudStack volumes are
>> >>>>> created
>> >>>>>> as VDIs on the SR that was created behind the scenes.
>> >>>>>>
>> >>>>>> Dynamic type: The admin has to go in and create Primary Storage
>> based
>> >>>>> on a
>> >>>>>> plug-in that will create and delete volumes on its storage system
>> >>>>>> dynamically (as is enabled via the storage framework). When a user
>> >>>>> wants to
>> >>>>>> attach a CloudStack volume that was created, the framework tells
>> the
>> >>>>>> plug-in to create a new volume. After this is done, the attach
>> logic
>> >>>>> for
>> >>>>>> the hypervisor in question is called. No hypervisor data structure
>> >>>>> exists
>> >>>>>> at this point because the volume was just created. The hypervisor
>> data
>> >>>>>> structure must be created.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com
>> >>>>>>> wrote:
>> >>>>>>
>> >>>>>>> These are new terms, so I should probably have defined them up
>> front
>> >>>>> for
>> >>>>>>> you. :)
>> >>>>>>>
>> >>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>> creates a
>> >>>>>>> volume on a SAN), then a hypervisor data structure is created to
>> >>>>> consume
>> >>>>>>> the storage (ex. XenServer SR), then that hypervisor data
>> structure
>> >>>>> is
>> >>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>> >>>>> hypervisor
>> >>>>>>> data structure as needed. In these cases, the attach logic assumes
>> >>>>> the
>> >>>>>>> hypervisor data structure is already in place and simply attaches
>> >>>>> the VDI
>> >>>>>>> on the hypervisor data structure to the VM in question.
>> >>>>>>>
>> >>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>> >>>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN,
>> >>>>> but the
>> >>>>>>> SAN itself). The hypervisor data structure must be created when an
>> >>>>> attach
>> >>>>>>> volume is performed because these types of volumes have not been
>> >>>>> pre-hooked
>> >>>>>>> up to such a hypervisor data structure by an admin. Once the
>> attach
>> >>>>> logic
>> >>>>>>> creates, say, an SR on XenServer for this volume, it attaches the
>> >>>>> one and
>> >>>>>>> only VDI within the SR to the VM in question.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>> >>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Mike,
>> >>>>>>>>
>> >>>>>>>> The current implementation of the Dynamic type attach behavior
>> >>>>> works in
>> >>>>>>>> terms of Xen ISCSI which why I ask about the difference. Another
>> >>>>> way to
>> >>>>>>>> ask the question -- what is the definition of a Dynamic storage
>> >>>>> pool type?
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> -John
>> >>>>>>>>
>> >>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
>> >>>>> when you
>> >>>>>>>>> want to set up Primary Storage that is directly based on an
>> iSCSI
>> >>>>>>>> target.
>> >>>>>>>>> This allows you to skip the step of going to the hypervisor and
>> >>>>>>>> creating a
>> >>>>>>>>> storage repository based on that iSCSI target as CloudStack does
>> >>>>> that
>> >>>>>>>> part
>> >>>>>>>>> for you. I think this is only supported for XenServer. For all
>> >>>>> other
>> >>>>>>>>> hypervisors, you must first go to the hypervisor and perform
>> this
>> >>>>> step
>> >>>>>>>>> manually.
>> >>>>>>>>>
>> >>>>>>>>> I don't really know what RBD is.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
>> jburwell@basho.com>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Mike,
>> >>>>>>>>>>
>> >>>>>>>>>> Reading through the code, what is the difference between the
>> >>>>> ISCSI and
>> >>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> -John
>> >>>>>>>>>>
>> >>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> >>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> This new type of storage is defined in the
>> >>>>> Storage.StoragePoolType
>> >>>>>>>> class
>> >>>>>>>>>>> (called Dynamic):
>> >>>>>>>>>>>
>> >>>>>>>>>>> public static enum StoragePoolType {
>> >>>>>>>>>>>
>> >>>>>>>>>>> Filesystem(false), // local directory
>> >>>>>>>>>>>
>> >>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>> >>>>>>>>>>>
>> >>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>> >>>>>>>>>>>
>> >>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>>>>>>>>>
>> >>>>>>>>>>> ISO(false), // for iso image
>> >>>>>>>>>>>
>> >>>>>>>>>>> LVM(false), // XenServer local LVM SR
>> >>>>>>>>>>>
>> >>>>>>>>>>> CLVM(true),
>> >>>>>>>>>>>
>> >>>>>>>>>>> RBD(true),
>> >>>>>>>>>>>
>> >>>>>>>>>>> SharedMountPoint(true),
>> >>>>>>>>>>>
>> >>>>>>>>>>> VMFS(true), // VMware VMFS storage
>> >>>>>>>>>>>
>> >>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up
>> by
>> >>>>>>>>>>> customers.
>> >>>>>>>>>>>
>> >>>>>>>>>>> EXT(false), // XenServer local EXT SR
>> >>>>>>>>>>>
>> >>>>>>>>>>> OCFS2(true),
>> >>>>>>>>>>>
>> >>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>> SolidFire)
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> boolean shared;
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> StoragePoolType(boolean shared) {
>> >>>>>>>>>>>
>> >>>>>>>>>>> this.shared = shared;
>> >>>>>>>>>>>
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> public boolean isShared() {
>> >>>>>>>>>>>
>> >>>>>>>>>>> return shared;
>> >>>>>>>>>>>
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> For example, let's say another storage company wants to
>> >>>>> implement a
>> >>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It would
>> be
>> >>>>>>>> dynamic,
>> >>>>>>>>>>>> zone-wide storage, as well. They would need only implement a
>> >>>>> storage
>> >>>>>>>>>>>> plug-in as I've made the necessary changes to the
>> >>>>> hypervisor-attach
>> >>>>>>>>>> logic
>> >>>>>>>>>>>> to support their plug-in.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific.
>> >>>>> It is
>> >>>>>>>> not.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
>> zone-wide
>> >>>>>>>> storage
>> >>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
>> storage).
>> >>>>> This
>> >>>>>>>>>> kind of
>> >>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>> changes.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Edison created a new framework that supported the creation
>> and
>> >>>>>>>> deletion
>> >>>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
>> >>>>> Portland
>> >>>>>>>>>> back
>> >>>>>>>>>>>>> in April, we realized that it was not complete. We realized
>> >>>>> there
>> >>>>>>>> was
>> >>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
>> >>>>> attach
>> >>>>>>>> logic
>> >>>>>>>>>> was
>> >>>>>>>>>>>>> changed to recognize this new type of storage and create the
>> >>>>>>>>>> appropriate
>> >>>>>>>>>>>>> hypervisor data structure.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>> >>>>> jburwell@basho.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> It is generally odd to me that any operation in the Storage
>> >>>>> layer
>> >>>>>>>>>> would
>> >>>>>>>>>>>>>> understand or care about details. I expect to see the
>> Storage
>> >>>>>>>>>> services
>> >>>>>>>>>>>>>> expose a set of operations that can be composed/driven by
>> the
>> >>>>>>>>>> Hypervisor
>> >>>>>>>>>>>>>> implementations to allocate space/create structures per
>> their
>> >>>>>>>> needs.
>> >>>>>>>>>> If
>> >>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>> >>>>> massive
>> >>>>>>>>>> n-to-n
>> >>>>>>>>>>>>>> problem that will make the system increasingly difficult to
>> >>>>>>>> maintain
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific
>> SolidFire
>> >>>>> code
>> >>>>>>>> is
>> >>>>>>>>>>>>>> located in the CitrixResourceBase class?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> To delve into this in a bit more detail:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
>> XenServer,
>> >>>>> the
>> >>>>>>>> admin
>> >>>>>>>>>>>>>> had
>> >>>>>>>>>>>>>>> to first create a volume on the storage system, then go
>> into
>> >>>>> the
>> >>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>> to set up a data structure to make use of the volume (ex.
>> a
>> >>>>>>>> storage
>> >>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
>> >>>>> data
>> >>>>>>>> disks
>> >>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>> shared this storage system's volume.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> With Edison's new storage framework, storage need no
>> longer
>> >>>>> be so
>> >>>>>>>>>>>>>> static
>> >>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
>> >>>>> storage
>> >>>>>>>>>> system's
>> >>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
>> Quality
>> >>>>> of
>> >>>>>>>>>>>>>> Service).
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
>> >>>>> create
>> >>>>>>>> and
>> >>>>>>>>>>>>>> delete
>> >>>>>>>>>>>>>>> volumes as needed.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> The problem that the storage framework did not address is
>> in
>> >>>>>>>> creating
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>> >>>>> performing an
>> >>>>>>>>>>>>>>> attach/detach.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so. I've
>> >>>>> got
>> >>>>>>>>>>>>>> XenServer
>> >>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox
>> and
>> >>>>> can
>> >>>>>>>>>> submit
>> >>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Does that help a bit? :)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>> >>>>> code).
>> >>>>>>>> The
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
>> volume
>> >>>>> as
>> >>>>>>>>>>>>>> needed,
>> >>>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>>> when the time comes to attach the volume to a hypervisor,
>> >>>>> the
>> >>>>>>>> attach
>> >>>>>>>>>>>>>>> logic
>> >>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>> >>>>>>>> zone-wide
>> >>>>>>>>>>>>>>> storage
>> >>>>>>>>>>>>>>>> (where the volume has just been created) and create,
>> say, a
>> >>>>>>>> storage
>> >>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
>> >>>>> make
>> >>>>>>>> use
>> >>>>>>>>>>>>>> of the
>> >>>>>>>>>>>>>>>> volume that was just created.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I've been spending most of my time recently making the
>> >>>>> attach
>> >>>>>>>> logic
>> >>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>> in the agent code.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Does that clear it up?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks!
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>> >>>>>>>> jburwell@basho.com>
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>> >>>>>>>> specific?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
>> >>>>> that
>> >>>>>>>>>>>>>>> CloudStack
>> >>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get
>> KVM
>> >>>>> in.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
>> >>>>> what
>> >>>>>>>> we do
>> >>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>> >>>>> VMware in
>> >>>>>>>> my
>> >>>>>>>>>>>>>>>>> sandbox,
>> >>>>>>>>>>>>>>>>>> as well.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>> >>>>>>>> Offerings.
>> >>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>> plan to
>> >>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage
>> <
>> >>>>>>>>>>>>>>> kelcey@bbits.ca
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
>> >>>>> post
>> >>>>>>>> 4.2?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>> >>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue
>> >>>>> as I
>> >>>>>>>> don't
>> >>>>>>>>>>>>>>>>> support
>> >>>>>>>>>>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>> >>>>>>>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> John,
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now,
>> users
>> >>>>> need
>> >>>>>>>> to
>> >>>>>>>>>>>>>>>>> calculate
>> >>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>> >>>>> service_offering
>> >>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation.
>> Of
>> >>>>> course
>> >>>>>>>>>>>>>> it is
>> >>>>>>>>>>>>>>>>>>> better
>> >>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>> >>>>> volume,
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>> >>>>> later. Do
>> >>>>>>>> you
>> >>>>>>>>>>>>>>>>> think it
>> >>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>> >>>>> ustcweizhou@gmail.com
>> >>>>>>>>>
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to
>> adding
>> >>>>> more.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>> >>>>> rather
>> >>>>>>>> than
>> >>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>> >>>>> time.
>> >>>>>>>>>>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>> read/write
>> >>>>> for
>> >>>>>>>> an
>> >>>>>>>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more
>> for
>> >>>>>>>> volumes
>> >>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>> >>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
>> >>>>> users
>> >>>>>>>> would
>> >>>>>>>>>>>>>> like
>> >>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>> >>>>> operator
>> >>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>>>>>> such a
>> >>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>> >>>>> setting
>> >>>>>>>>>>>>>> IOPS/BPS
>> >>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>> >>>>> command
>> >>>>>>>> line.
>> >>>>>>>>>>>>>>>>>>> However,
>> >>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters
>> >>>>> of a
>> >>>>>>>>>>>>>> created
>> >>>>>>>>>>>>>>>>>>>> offering
>> >>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize
>> for
>> >>>>> being
>> >>>>>>>>>>>>>>> unclear.
>> >>>>>>>>>>>>>>>>>>> Can
>> >>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set
>> of
>> >>>>>>>>>>>>>> interfaces
>> >>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>> resource?
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>> >>>>> commit
>> >>>>>>>> Mike's
>> >>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>> >>>>>>>> limitation in
>> >>>>>>>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
>> >>>>> Mike
>> >>>>>>>>>>>>>> regarding
>> >>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
>> that
>> >>>>>>>>>>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think
>> >>>>> we need
>> >>>>>>>>>>>>>> closer
>> >>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have
>> have
>> >>>>>>>> solidfire
>> >>>>>>>>>>>>>>> track
>> >>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>> >>>>> captured
>> >>>>>>>> in
>> >>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>> common
>> >>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>> >>>>> expressing
>> >>>>>>>>>>>>>>> offerings).
>> >>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>> >>>>> reasoning
>> >>>>>>>>>>>>>> about
>> >>>>>>>>>>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>> >>>>> configurations
>> >>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>> >>>>> particular,
>> >>>>>>>> a
>> >>>>>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>> >>>>> configured
>> >>>>>>>> for
>> >>>>>>>>>>>>>> a VM
>> >>>>>>>>>>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>> >>>>> Another
>> >>>>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
>> storage
>> >>>>>>>> device at
>> >>>>>>>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>> >>>>>>>> introduces
>> >>>>>>>>>>>>>>> logical
>> >>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the
>> >>>>> max
>> >>>>>>>>>>>>>> number of
>> >>>>>>>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>> >>>>> storage
>> >>>>>>>>>>>>>> system
>> >>>>>>>>>>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>> >>>>>>>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>> >>>>> until all
>> >>>>>>>>>>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>> >>>>> over to
>> >>>>>>>>>>>>>> test.
>> >>>>>>>>>>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>> >>>>> addressed
>> >>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>> >>>>> satisfied.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>> >>>>>>>> SolidFire
>> >>>>>>>>>>>>>>> patch
>> >>>>>>>>>>>>>>>>> (
>> >>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>> >>>>> understand
>> >>>>>>>> it
>> >>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>> >>>>> ensure
>> >>>>>>>> we
>> >>>>>>>>>>>>>> don't
>> >>>>>>>>>>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>> >>>>> SolidFire are
>> >>>>>>>>>>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> >>>>>>>> ustcweizhou@gmail.com
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>> >>>>> Monday,
>> >>>>>>>> I
>> >>>>>>>>>>>>>> will
>> >>>>>>>>>>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
>> again
>> >>>>>>>> that we
>> >>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>> >>>>> release
>> >>>>>>>> a
>> >>>>>>>>>>>>>>> version
>> >>>>>>>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling
>> and
>> >>>>> that's
>> >>>>>>>>>>>>>> great!
>> >>>>>>>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day
>> #1.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
>> >>>>> read
>> >>>>>>>> and
>> >>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>> >>>>> don't see
>> >>>>>>>>>>>>>> how we
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
>> Therefore, we
>> >>>>>>>> would
>> >>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>> >>>>>>>>>>>>>> compatibility.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>> >>>>> that it
>> >>>>>>>>>>>>>> will go
>> >>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the
>> 4.2
>> >>>>>>>> release
>> >>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
>> >>>>> Total
>> >>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
>> >>>>> But if
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>>> window
>> >>>>>>>>>>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary
>> to do
>> >>>>>>>> that.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>> association
>> >>>>>>>> between
>> >>>>>>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>> >>>>>>>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>> >>>>> decided to
>> >>>>>>>>>>>>>> ignore
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>> >>>>> network
>> >>>>>>>>>>>>>>> throttling
>> >>>>>>>>>>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>> >>>>> well.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>> >>>>>>>> completely
>> >>>>>>>>>>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
>> situations
>> >>>>> then
>> >>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>> >>>>> could be
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I
>> will
>> >>>>>>>>>>>>>> implement
>> >>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
>> >>>>> goes
>> >>>>>>>> into
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
>> >>>>> only
>> >>>>>>>> total
>> >>>>>>>>>>>>>>> IOps
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch
>> into
>> >>>>>>>> master.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>> >>>>> hours.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>> >>>>> device
>> >>>>>>>>>>>>>> (local
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>> >>>>> device
>> >>>>>>>>>>>>>> (such as
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect
>> the
>> >>>>> disk
>> >>>>>>>>>>>>>>>>> performance
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit
>> >>>>> the
>> >>>>>>>> disk
>> >>>>>>>>>>>>>> I/O
>> >>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>> >>>>> between
>> >>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
>> >>>>> rate
>> >>>>>>>> for
>> >>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
>> collect
>> >>>>> both
>> >>>>>>>> the
>> >>>>>>>>>>>>>> Read
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for
>> a
>> >>>>> global
>> >>>>>>>>>>>>>> value.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then
>> Read
>> >>>>>>>> IOps,
>> >>>>>>>>>>>>>> so it
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would
>> set
>> >>>>> a
>> >>>>>>>> lower
>> >>>>>>>>>>>>>>> value
>> >>>>>>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I
>> think
>> >>>>> it
>> >>>>>>>> would
>> >>>>>>>>>>>>>> be
>> >>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>> >>>>> support
>> >>>>>>>>>>>>>> both,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>> >>>>> values
>> >>>>>>>> for
>> >>>>>>>>>>>>>>> read
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as
>> the
>> >>>>>>>> total
>> >>>>>>>>>>>>>>> limit.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
>> disk_offering,
>> >>>>> and
>> >>>>>>>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>>>>>>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>> success?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration)
>> is
>> >>>>>>>> included
>> >>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>> potential
>> >>>>>>>>>>>>>> regressions?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
>> CloudStack
>> >>>>> UI.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>> >>>>> migrate,
>> >>>>>>>>>>>>>> restore
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> git diff
>> >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>> >>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>>>>>>> https://cwiki.
>> >>>>>>>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>> >>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>>>>>> cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>> >>>>>>
>> >>>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the
>> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>> *â„¢*
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >> *â„¢*
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
To go back to my parlance earlier, a SolidFire is an unmanaged, iSCSI device. The administrator will allocate a SAN volume for CloudStack's use onto which CloudStack volumes will be created.
I want to see the hypervisor and storage code further decoupled. As I read the code, we only check for an SR/create it for Dynamic devices. To my mind, this operation should be protocol agnostic. The Xen plugin should interrogate the storage device to determine whether or not the SR is present, and if it is not, create it. Ditto with attach, the differences between the protocols should be further normalized across the protocols such that once the SR is mounted (which will have some protocol specific elements), the attach operation becomes protocol agnostic.
In terms of the provisioned IOPS (which is where we started ;), how do we ensure that the IOPS being provisioned through a hypervisor can actually be fulfilled through by storage. In particular, how do we ensure that multiple VMs with provisioned IOPS won't be cut off by the underlying storage. Additionally, it seems that hypervisors want to discretely express IOPS for read and write operations. I think we need to be consistent across the system, and I lean towards the discrete expression because it exposes the most capability across the system.
Thanks,
-John
On Jun 4, 2013, at 3:27 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> "The storage device itself shouldn't know or care that it is being used for
> a Xen SR -- simply be able to answer questions about it is storing."
>
> I see...so your concern here is that the SolidFire plug-in needs to call
> itself "Dynamic" storage so that the hypervisor logic knows to treat it as
> such.
>
> I'm totally open to removing that constraint and just calling it iSCSI or
> whatever. We would just need a way for the hypervisor attach logic to
> detect this new requirement and perform the necessary activities.
>
>
> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> See my responses in-line.
>>
>> Thanks,
>> -John
>>
>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> I'm trying to picture this:
>>>
>>> "Finally, while CloudStack may be able to manage a device, an operator
>> may
>>> chose to leave it unmanaged by CloudStack (e.g. the device is shared by
>>> many services, and the operator has chosen to dedicate only a portion of
>> it
>>> to CloudStack). Does my reasoning make sense?"
>>>
>>> I guess I'm not sure how creating a SAN volume via the plug-in (before an
>>> attach request to the hypervisor) would work unless the hypervisor
>> consumes
>>> the SAN volume in the form of, say, an SR.
>>
>> My thinking is that, independent of CloudStack, an operator allocates a
>> chunk of a SAN to CloudStack, and exposes it through a LUN. They simply
>> want to turn control of that LUN over to CloudStack, but not allow
>> CloudStack to allocate anymore LUNs.
>>
>>>
>>> As the attach logic stands prior to my changes, we would be passing in a
>>> SAN volume that does not have the necessary hypervisor support (like an
>> SR)
>>> and the logic will fail.
>>>
>>> Are you thinking we should maybe have the storage framework itself detect
>>> that such a SAN volume needs support from the hypervisor side and have it
>>> call into the agent code specifically to create the SR before the attach
>>> logic runs in the agent code?
>>
>> I think the hypervisor management plugin should have a rich enough
>> interface to storage to determine available for volume storage. For Xen,
>> this interface would allow the interrogation of the device to determine the
>> SR is present. The storage device itself shouldn't know or care that it
>> is being used for a Xen SR -- simply be able to answer questions about it
>> is storing.
>>
>>>
>>>
>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>>>> wrote:
>>>
>>>> So, the flow is as follows:
>>>>
>>>> * The admin registers the SolidFire driver (which is a type of so-called
>>>> Dynamic storage). Once this is done, a new Primary Storage shows up in
>> the
>>>> applicable zone.
>>>>
>>>> * The admin creates a Disk Offering that references the storage tag of
>> the
>>>> newly created Primary Storage.
>>>>
>>>> * The end user creates a CloudStack volume. This leads to a new row in
>> the
>>>> cloud.volumes table.
>>>>
>>>> * The end user attaches the CloudStack volume to a VM (attach disk).
>> This
>>>> leads to the storage framework calling the plug-in to create a new
>> volume
>>>> on its storage system (in my case, a SAN). The plug-in also updates the
>>>> cloud.volumes row with applicable data (like the IQN of the SAN volume).
>>>> This plug-in code is only invoked if the CloudStack volume is in the
>>>> 'Allocated' state. After the attach, the volume will be in the 'Ready'
>>>> state (even after a detach disk) and the plug-in code will not be called
>>>> again to create this SAN volume.
>>>>
>>>> * The hypervisor-attach logic is run and detects the CloudStack volume
>> to
>>>> attach needs "assistance" in the form of a hypervisor data structure
>> (ex.
>>>> an SR on XenServer).
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> "To ensure that we are in sync on terminology, volume, in these
>>>>> definitions, refers to the physical allocation on the device, correct?"
>>>>>
>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>>>
>>>>> To refer to the 'volume' the end user can make in CloudStack, I try to
>>>>> use 'CloudStack volume'.
>>>>>
>>>>>
>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> What you say here may very well make sense, but I'm having a hard time
>>>>>> envisioning it.
>>>>>>
>>>>>> Perhaps we should draw Edison in on this conversation as he was the
>>>>>> initial person to suggest the approach I took.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> It feels like we are combining two distinct concepts -- storage
>> device
>>>>>>> management and storage protocols. In both cases, we are
>> communicating with
>>>>>>> ISCSI, but one allows the system to create/delete volumes (Dynamic)
>> on the
>>>>>>> device while the other requires the volume to be volume to be managed
>>>>>>> outside of the CloudStack context. To ensure that we are in sync on
>>>>>>> terminology, volume, in these definitions, refers to the physical
>>>>>>> allocation on the device, correct? Minimally, we must be able to
>>>>>>> communicate with a storage device to move bits from one place to
>> another,
>>>>>>> read bits, delete bits, etc. Optionally, a storage device may be
>> able to
>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
>> device
>>>>>>> onto which we store a Xen SR, and we can have a managed SolidFire
>> iSCSI
>>>>>>> device on which CloudStack is capable of allocating LUNs and storing
>>>>>>> volumes. Finally, while CloudStack may be able to manage a device,
>> an
>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
>> device is
>>>>>>> shared by many services, and the operator has chosen to dedicate
>> only a
>>>>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>>>>
>>>>>>> Assuming my thoughts above are reasonable, it seems appropriate to
>>>>>>> strip the management concerns from StoragePoolType, add the notion
>> of a
>>>>>>> storage device with an attached driver that indicates whether or not
>> is
>>>>>>> managed by CloudStack, and establish a abstraction representing a
>> physical
>>>>>>> allocation on a device separate that is associated with a volume.
>> With
>>>>>>> these notions in place, hypervisor drivers can declare which
>> protocols they
>>>>>>> support and when they encounter a device managed by CloudStack,
>> utilize the
>>>>>>> management operations exposed by the driver to automate allocation.
>> If
>>>>>>> these thoughts/concepts make sense, then we can sit down and drill
>> down to
>>>>>>> a more detailed design.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> Here is the difference between the current iSCSI type and the
>> Dynamic
>>>>>>> type:
>>>>>>>>
>>>>>>>> iSCSI type: The admin has to go in and create a Primary Storage
>> based
>>>>>>> on
>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must exist
>> on
>>>>>>> the
>>>>>>>> storage system (it is pre-allocated). Future CloudStack volumes are
>>>>>>> created
>>>>>>>> as VDIs on the SR that was created behind the scenes.
>>>>>>>>
>>>>>>>> Dynamic type: The admin has to go in and create Primary Storage
>> based
>>>>>>> on a
>>>>>>>> plug-in that will create and delete volumes on its storage system
>>>>>>>> dynamically (as is enabled via the storage framework). When a user
>>>>>>> wants to
>>>>>>>> attach a CloudStack volume that was created, the framework tells the
>>>>>>>> plug-in to create a new volume. After this is done, the attach logic
>>>>>>> for
>>>>>>>> the hypervisor in question is called. No hypervisor data structure
>>>>>>> exists
>>>>>>>> at this point because the volume was just created. The hypervisor
>> data
>>>>>>>> structure must be created.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> These are new terms, so I should probably have defined them up
>> front
>>>>>>> for
>>>>>>>>> you. :)
>>>>>>>>>
>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
>> creates a
>>>>>>>>> volume on a SAN), then a hypervisor data structure is created to
>>>>>>> consume
>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data structure
>>>>>>> is
>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>>>>>>> hypervisor
>>>>>>>>> data structure as needed. In these cases, the attach logic assumes
>>>>>>> the
>>>>>>>>> hypervisor data structure is already in place and simply attaches
>>>>>>> the VDI
>>>>>>>>> on the hypervisor data structure to the VM in question.
>>>>>>>>>
>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN,
>>>>>>> but the
>>>>>>>>> SAN itself). The hypervisor data structure must be created when an
>>>>>>> attach
>>>>>>>>> volume is performed because these types of volumes have not been
>>>>>>> pre-hooked
>>>>>>>>> up to such a hypervisor data structure by an admin. Once the attach
>>>>>>> logic
>>>>>>>>> creates, say, an SR on XenServer for this volume, it attaches the
>>>>>>> one and
>>>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> The current implementation of the Dynamic type attach behavior
>>>>>>> works in
>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference. Another
>>>>>>> way to
>>>>>>>>>> ask the question -- what is the definition of a Dynamic storage
>>>>>>> pool type?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
>>>>>>> when you
>>>>>>>>>>> want to set up Primary Storage that is directly based on an iSCSI
>>>>>>>>>> target.
>>>>>>>>>>> This allows you to skip the step of going to the hypervisor and
>>>>>>>>>> creating a
>>>>>>>>>>> storage repository based on that iSCSI target as CloudStack does
>>>>>>> that
>>>>>>>>>> part
>>>>>>>>>>> for you. I think this is only supported for XenServer. For all
>>>>>>> other
>>>>>>>>>>> hypervisors, you must first go to the hypervisor and perform this
>>>>>>> step
>>>>>>>>>>> manually.
>>>>>>>>>>>
>>>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jburwell@basho.com
>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Reading through the code, what is the difference between the
>>>>>>> ISCSI and
>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> This new type of storage is defined in the
>>>>>>> Storage.StoragePoolType
>>>>>>>>>> class
>>>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>>>
>>>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>>>
>>>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>>>
>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>>>
>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>>>>>>>>>>>
>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>>>
>>>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>>>
>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>>>
>>>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>>>
>>>>>>>>>>>>> RBD(true),
>>>>>>>>>>>>>
>>>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>>>
>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>>>
>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>>>>>>>>>>>>> customers.
>>>>>>>>>>>>>
>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>>>
>>>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
>> SolidFire)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> boolean shared;
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>>>
>>>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>>>
>>>>>>>>>>>>> return shared;
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> For example, let's say another storage company wants to
>>>>>>> implement a
>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It would
>> be
>>>>>>>>>> dynamic,
>>>>>>>>>>>>>> zone-wide storage, as well. They would need only implement a
>>>>>>> storage
>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>>>>> hypervisor-attach
>>>>>>>>>>>> logic
>>>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific.
>>>>>>> It is
>>>>>>>>>> not.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>>>>>>>>>> storage
>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of storage).
>>>>>>> This
>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
>> changes.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Edison created a new framework that supported the creation
>> and
>>>>>>>>>> deletion
>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
>>>>>>> Portland
>>>>>>>>>>>> back
>>>>>>>>>>>>>>> in April, we realized that it was not complete. We realized
>>>>>>> there
>>>>>>>>>> was
>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
>>>>>>> attach
>>>>>>>>>> logic
>>>>>>>>>>>> was
>>>>>>>>>>>>>>> changed to recognize this new type of storage and create the
>>>>>>>>>>>> appropriate
>>>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the Storage
>>>>>>> layer
>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> understand or care about details. I expect to see the
>> Storage
>>>>>>>>>>>> services
>>>>>>>>>>>>>>>> expose a set of operations that can be composed/driven by
>> the
>>>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>>>> implementations to allocate space/create structures per
>> their
>>>>>>>>>> needs.
>>>>>>>>>>>> If
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>>>>>>> massive
>>>>>>>>>>>> n-to-n
>>>>>>>>>>>>>>>> problem that will make the system increasingly difficult to
>>>>>>>>>> maintain
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
>>>>>>> code
>>>>>>>>>> is
>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer,
>>>>>>> the
>>>>>>>>>> admin
>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>> to first create a volume on the storage system, then go
>> into
>>>>>>> the
>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> to set up a data structure to make use of the volume (ex. a
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
>>>>>>> data
>>>>>>>>>> disks
>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no longer
>>>>>>> be so
>>>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
>>>>>>> storage
>>>>>>>>>>>> system's
>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
>> Quality
>>>>>>> of
>>>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
>>>>>>> create
>>>>>>>>>> and
>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The problem that the storage framework did not address is
>> in
>>>>>>>>>> creating
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>>>>>>> performing an
>>>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so. I've
>>>>>>> got
>>>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and
>>>>>>> can
>>>>>>>>>>>> submit
>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>>>>>>> code).
>>>>>>>>>> The
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
>> volume
>>>>>>> as
>>>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a hypervisor,
>>>>>>> the
>>>>>>>>>> attach
>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>>>>>>>>>> zone-wide
>>>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> (where the volume has just been created) and create, say,
>> a
>>>>>>>>>> storage
>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
>>>>>>> make
>>>>>>>>>> use
>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making the
>>>>>>> attach
>>>>>>>>>> logic
>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>>>>>>>>>> specific?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
>>>>>>> that
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get
>> KVM
>>>>>>> in.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
>>>>>>> what
>>>>>>>>>> we do
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>>>>>> VMware in
>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>>>>>>>>> Offerings.
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
>>>>>>> post
>>>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue
>>>>>>> as I
>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
>>>>>>> need
>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>>>>> service_offering
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
>>>>>>> course
>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>>>>>>> volume,
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>>>>>> later. Do
>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
>>>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>>>>>>> rather
>>>>>>>>>> than
>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
>> read/write
>>>>>>> for
>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>>>>>>>>>> volumes
>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
>>>>>>> users
>>>>>>>>>> would
>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>>>>>>> operator
>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>>>>>>> setting
>>>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>>>>>> command
>>>>>>>>>> line.
>>>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters
>>>>>>> of a
>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
>>>>>>> being
>>>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set
>> of
>>>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
>> resource?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>>>>>>> commit
>>>>>>>>>> Mike's
>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
>>>>>>> Mike
>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
>> that
>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think
>>>>>>> we need
>>>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have have
>>>>>>>>>> solidfire
>>>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>>>>>> captured
>>>>>>>>>> in
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>>>>>> expressing
>>>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>>>>>> reasoning
>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>>>>>> configurations
>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>>>>>> particular,
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>>>>>> configured
>>>>>>>>>> for
>>>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>>>>>> Another
>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>>>>>>>>>> device at
>>>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>>>>>>>>> introduces
>>>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the
>>>>>>> max
>>>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>>>>>>> storage
>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>>>>>>> until all
>>>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>>>>>>> over to
>>>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>>>>>> addressed
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>>>>>>>>>> SolidFire
>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>>>>>> understand
>>>>>>>>>> it
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>>>>>>> ensure
>>>>>>>>>> we
>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>>>>>>> Monday,
>>>>>>>>>> I
>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
>> again
>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>>>>>>> release
>>>>>>>>>> a
>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
>>>>>>> that's
>>>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
>>>>>>> read
>>>>>>>>>> and
>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>>>>>>> don't see
>>>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore,
>> we
>>>>>>>>>> would
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>>>>>>> that it
>>>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the
>> 4.2
>>>>>>>>>> release
>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
>>>>>>> Total
>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
>>>>>>> But if
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to
>> do
>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
>> association
>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>>>>>>> decided to
>>>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>>>>>>> network
>>>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>>>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
>>>>>>> then
>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>>>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I
>> will
>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
>>>>>>> goes
>>>>>>>>>> into
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
>>>>>>> only
>>>>>>>>>> total
>>>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch
>> into
>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>>>>>>> device
>>>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>>>>>>> device
>>>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
>>>>>>> disk
>>>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit
>>>>>>> the
>>>>>>>>>> disk
>>>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>>>>>>> between
>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
>>>>>>> rate
>>>>>>>>>> for
>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
>>>>>>> both
>>>>>>>>>> the
>>>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
>>>>>>> global
>>>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then
>> Read
>>>>>>>>>> IOps,
>>>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would
>> set
>>>>>>> a
>>>>>>>>>> lower
>>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think
>>>>>>> it
>>>>>>>>>> would
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>>>>>>> support
>>>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>>>>>>> values
>>>>>>>>>> for
>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as
>> the
>>>>>>>>>> total
>>>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering,
>>>>>>> and
>>>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
>> success?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>>>>>>>>>> included
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
>> potential
>>>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack
>>>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>>>>>> migrate,
>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>>>> https://cwiki.
>>>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"The storage device itself shouldn't know or care that it is being used for
a Xen SR -- simply be able to answer questions about it is storing."
I see...so your concern here is that the SolidFire plug-in needs to call
itself "Dynamic" storage so that the hypervisor logic knows to treat it as
such.
I'm totally open to removing that constraint and just calling it iSCSI or
whatever. We would just need a way for the hypervisor attach logic to
detect this new requirement and perform the necessary activities.
On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> See my responses in-line.
>
> Thanks,
> -John
>
> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > I'm trying to picture this:
> >
> > "Finally, while CloudStack may be able to manage a device, an operator
> may
> > chose to leave it unmanaged by CloudStack (e.g. the device is shared by
> > many services, and the operator has chosen to dedicate only a portion of
> it
> > to CloudStack). Does my reasoning make sense?"
> >
> > I guess I'm not sure how creating a SAN volume via the plug-in (before an
> > attach request to the hypervisor) would work unless the hypervisor
> consumes
> > the SAN volume in the form of, say, an SR.
>
> My thinking is that, independent of CloudStack, an operator allocates a
> chunk of a SAN to CloudStack, and exposes it through a LUN. They simply
> want to turn control of that LUN over to CloudStack, but not allow
> CloudStack to allocate anymore LUNs.
>
> >
> > As the attach logic stands prior to my changes, we would be passing in a
> > SAN volume that does not have the necessary hypervisor support (like an
> SR)
> > and the logic will fail.
> >
> > Are you thinking we should maybe have the storage framework itself detect
> > that such a SAN volume needs support from the hypervisor side and have it
> > call into the agent code specifically to create the SR before the attach
> > logic runs in the agent code?
>
> I think the hypervisor management plugin should have a rich enough
> interface to storage to determine available for volume storage. For Xen,
> this interface would allow the interrogation of the device to determine the
> SR is present. The storage device itself shouldn't know or care that it
> is being used for a Xen SR -- simply be able to answer questions about it
> is storing.
>
> >
> >
> > On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >> wrote:
> >
> >> So, the flow is as follows:
> >>
> >> * The admin registers the SolidFire driver (which is a type of so-called
> >> Dynamic storage). Once this is done, a new Primary Storage shows up in
> the
> >> applicable zone.
> >>
> >> * The admin creates a Disk Offering that references the storage tag of
> the
> >> newly created Primary Storage.
> >>
> >> * The end user creates a CloudStack volume. This leads to a new row in
> the
> >> cloud.volumes table.
> >>
> >> * The end user attaches the CloudStack volume to a VM (attach disk).
> This
> >> leads to the storage framework calling the plug-in to create a new
> volume
> >> on its storage system (in my case, a SAN). The plug-in also updates the
> >> cloud.volumes row with applicable data (like the IQN of the SAN volume).
> >> This plug-in code is only invoked if the CloudStack volume is in the
> >> 'Allocated' state. After the attach, the volume will be in the 'Ready'
> >> state (even after a detach disk) and the plug-in code will not be called
> >> again to create this SAN volume.
> >>
> >> * The hypervisor-attach logic is run and detects the CloudStack volume
> to
> >> attach needs "assistance" in the form of a hypervisor data structure
> (ex.
> >> an SR on XenServer).
> >>
> >>
> >> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> "To ensure that we are in sync on terminology, volume, in these
> >>> definitions, refers to the physical allocation on the device, correct?"
> >>>
> >>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>
> >>> To refer to the 'volume' the end user can make in CloudStack, I try to
> >>> use 'CloudStack volume'.
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com> wrote:
> >>>
> >>>> Hi John,
> >>>>
> >>>> What you say here may very well make sense, but I'm having a hard time
> >>>> envisioning it.
> >>>>
> >>>> Perhaps we should draw Edison in on this conversation as he was the
> >>>> initial person to suggest the approach I took.
> >>>>
> >>>> What do you think?
> >>>>
> >>>> Thanks!
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jburwell@basho.com
> >wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> It feels like we are combining two distinct concepts -- storage
> device
> >>>>> management and storage protocols. In both cases, we are
> communicating with
> >>>>> ISCSI, but one allows the system to create/delete volumes (Dynamic)
> on the
> >>>>> device while the other requires the volume to be volume to be managed
> >>>>> outside of the CloudStack context. To ensure that we are in sync on
> >>>>> terminology, volume, in these definitions, refers to the physical
> >>>>> allocation on the device, correct? Minimally, we must be able to
> >>>>> communicate with a storage device to move bits from one place to
> another,
> >>>>> read bits, delete bits, etc. Optionally, a storage device may be
> able to
> >>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
> device
> >>>>> onto which we store a Xen SR, and we can have a managed SolidFire
> iSCSI
> >>>>> device on which CloudStack is capable of allocating LUNs and storing
> >>>>> volumes. Finally, while CloudStack may be able to manage a device,
> an
> >>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
> device is
> >>>>> shared by many services, and the operator has chosen to dedicate
> only a
> >>>>> portion of it to CloudStack). Does my reasoning make sense?
> >>>>>
> >>>>> Assuming my thoughts above are reasonable, it seems appropriate to
> >>>>> strip the management concerns from StoragePoolType, add the notion
> of a
> >>>>> storage device with an attached driver that indicates whether or not
> is
> >>>>> managed by CloudStack, and establish a abstraction representing a
> physical
> >>>>> allocation on a device separate that is associated with a volume.
> With
> >>>>> these notions in place, hypervisor drivers can declare which
> protocols they
> >>>>> support and when they encounter a device managed by CloudStack,
> utilize the
> >>>>> management operations exposed by the driver to automate allocation.
> If
> >>>>> these thoughts/concepts make sense, then we can sit down and drill
> down to
> >>>>> a more detailed design.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>
> >>>>>> Here is the difference between the current iSCSI type and the
> Dynamic
> >>>>> type:
> >>>>>>
> >>>>>> iSCSI type: The admin has to go in and create a Primary Storage
> based
> >>>>> on
> >>>>>> the iSCSI type. At this point in time, the iSCSI volume must exist
> on
> >>>>> the
> >>>>>> storage system (it is pre-allocated). Future CloudStack volumes are
> >>>>> created
> >>>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>>
> >>>>>> Dynamic type: The admin has to go in and create Primary Storage
> based
> >>>>> on a
> >>>>>> plug-in that will create and delete volumes on its storage system
> >>>>>> dynamically (as is enabled via the storage framework). When a user
> >>>>> wants to
> >>>>>> attach a CloudStack volume that was created, the framework tells the
> >>>>>> plug-in to create a new volume. After this is done, the attach logic
> >>>>> for
> >>>>>> the hypervisor in question is called. No hypervisor data structure
> >>>>> exists
> >>>>>> at this point because the volume was just created. The hypervisor
> data
> >>>>>> structure must be created.
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> These are new terms, so I should probably have defined them up
> front
> >>>>> for
> >>>>>>> you. :)
> >>>>>>>
> >>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
> creates a
> >>>>>>> volume on a SAN), then a hypervisor data structure is created to
> >>>>> consume
> >>>>>>> the storage (ex. XenServer SR), then that hypervisor data structure
> >>>>> is
> >>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
> >>>>> hypervisor
> >>>>>>> data structure as needed. In these cases, the attach logic assumes
> >>>>> the
> >>>>>>> hypervisor data structure is already in place and simply attaches
> >>>>> the VDI
> >>>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>>
> >>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
> >>>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN,
> >>>>> but the
> >>>>>>> SAN itself). The hypervisor data structure must be created when an
> >>>>> attach
> >>>>>>> volume is performed because these types of volumes have not been
> >>>>> pre-hooked
> >>>>>>> up to such a hypervisor data structure by an admin. Once the attach
> >>>>> logic
> >>>>>>> creates, say, an SR on XenServer for this volume, it attaches the
> >>>>> one and
> >>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> The current implementation of the Dynamic type attach behavior
> >>>>> works in
> >>>>>>>> terms of Xen ISCSI which why I ask about the difference. Another
> >>>>> way to
> >>>>>>>> ask the question -- what is the definition of a Dynamic storage
> >>>>> pool type?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
> >>>>> when you
> >>>>>>>>> want to set up Primary Storage that is directly based on an iSCSI
> >>>>>>>> target.
> >>>>>>>>> This allows you to skip the step of going to the hypervisor and
> >>>>>>>> creating a
> >>>>>>>>> storage repository based on that iSCSI target as CloudStack does
> >>>>> that
> >>>>>>>> part
> >>>>>>>>> for you. I think this is only supported for XenServer. For all
> >>>>> other
> >>>>>>>>> hypervisors, you must first go to the hypervisor and perform this
> >>>>> step
> >>>>>>>>> manually.
> >>>>>>>>>
> >>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jburwell@basho.com
> >
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Mike,
> >>>>>>>>>>
> >>>>>>>>>> Reading through the code, what is the difference between the
> >>>>> ISCSI and
> >>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> This new type of storage is defined in the
> >>>>> Storage.StoragePoolType
> >>>>>>>> class
> >>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>
> >>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>
> >>>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>>
> >>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>
> >>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
> >>>>>>>>>>>
> >>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>
> >>>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>>
> >>>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>
> >>>>>>>>>>> CLVM(true),
> >>>>>>>>>>>
> >>>>>>>>>>> RBD(true),
> >>>>>>>>>>>
> >>>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>>
> >>>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>
> >>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
> >>>>>>>>>>> customers.
> >>>>>>>>>>>
> >>>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>
> >>>>>>>>>>> OCFS2(true),
> >>>>>>>>>>>
> >>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex.
> SolidFire)
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> boolean shared;
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>>
> >>>>>>>>>>> this.shared = shared;
> >>>>>>>>>>>
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>>
> >>>>>>>>>>> return shared;
> >>>>>>>>>>>
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> For example, let's say another storage company wants to
> >>>>> implement a
> >>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It would
> be
> >>>>>>>> dynamic,
> >>>>>>>>>>>> zone-wide storage, as well. They would need only implement a
> >>>>> storage
> >>>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>>> hypervisor-attach
> >>>>>>>>>> logic
> >>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific.
> >>>>> It is
> >>>>>>>> not.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
> >>>>>>>> storage
> >>>>>>>>>>>>> (and SolidFire is an implementation of this kind of storage).
> >>>>> This
> >>>>>>>>>> kind of
> >>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
> changes.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Edison created a new framework that supported the creation
> and
> >>>>>>>> deletion
> >>>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
> >>>>> Portland
> >>>>>>>>>> back
> >>>>>>>>>>>>> in April, we realized that it was not complete. We realized
> >>>>> there
> >>>>>>>> was
> >>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
> >>>>> attach
> >>>>>>>> logic
> >>>>>>>>>> was
> >>>>>>>>>>>>> changed to recognize this new type of storage and create the
> >>>>>>>>>> appropriate
> >>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> It is generally odd to me that any operation in the Storage
> >>>>> layer
> >>>>>>>>>> would
> >>>>>>>>>>>>>> understand or care about details. I expect to see the
> Storage
> >>>>>>>>>> services
> >>>>>>>>>>>>>> expose a set of operations that can be composed/driven by
> the
> >>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>> implementations to allocate space/create structures per
> their
> >>>>>>>> needs.
> >>>>>>>>>> If
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
> >>>>> massive
> >>>>>>>>>> n-to-n
> >>>>>>>>>>>>>> problem that will make the system increasingly difficult to
> >>>>>>>> maintain
> >>>>>>>>>> and
> >>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
> >>>>> code
> >>>>>>>> is
> >>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer,
> >>>>> the
> >>>>>>>> admin
> >>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>> to first create a volume on the storage system, then go
> into
> >>>>> the
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> to set up a data structure to make use of the volume (ex. a
> >>>>>>>> storage
> >>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
> >>>>> data
> >>>>>>>> disks
> >>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> With Edison's new storage framework, storage need no longer
> >>>>> be so
> >>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
> >>>>> storage
> >>>>>>>>>> system's
> >>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage
> Quality
> >>>>> of
> >>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
> >>>>> create
> >>>>>>>> and
> >>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The problem that the storage framework did not address is
> in
> >>>>>>>> creating
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
> >>>>> performing an
> >>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so. I've
> >>>>> got
> >>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and
> >>>>> can
> >>>>>>>>>> submit
> >>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
> >>>>> code).
> >>>>>>>> The
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a
> volume
> >>>>> as
> >>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>> when the time comes to attach the volume to a hypervisor,
> >>>>> the
> >>>>>>>> attach
> >>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
> >>>>>>>> zone-wide
> >>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> (where the volume has just been created) and create, say,
> a
> >>>>>>>> storage
> >>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
> >>>>> make
> >>>>>>>> use
> >>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I've been spending most of my time recently making the
> >>>>> attach
> >>>>>>>> logic
> >>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
> >>>>>>>> specific?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
> >>>>> that
> >>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get
> KVM
> >>>>> in.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
> >>>>> what
> >>>>>>>> we do
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
> >>>>> VMware in
> >>>>>>>> my
> >>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
> >>>>>>>> Offerings.
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
> >>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
> >>>>> post
> >>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue
> >>>>> as I
> >>>>>>>> don't
> >>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
> >>>>> need
> >>>>>>>> to
> >>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>>> service_offering
> >>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
> >>>>> course
> >>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of individual
> >>>>> volume,
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
> >>>>> later. Do
> >>>>>>>> you
> >>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>>> ustcweizhou@gmail.com
> >>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
> >>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
> >>>>> rather
> >>>>>>>> than
> >>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
> >>>>> time.
> >>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
> read/write
> >>>>> for
> >>>>>>>> an
> >>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
> >>>>>>>> volumes
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
> >>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
> >>>>> users
> >>>>>>>> would
> >>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
> >>>>> operator
> >>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
> >>>>> setting
> >>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
> >>>>> command
> >>>>>>>> line.
> >>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters
> >>>>> of a
> >>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
> >>>>> being
> >>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set
> of
> >>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
> resource?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
> >>>>> commit
> >>>>>>>> Mike's
> >>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
> >>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
> >>>>> Mike
> >>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think
> that
> >>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think
> >>>>> we need
> >>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have have
> >>>>>>>> solidfire
> >>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
> >>>>> captured
> >>>>>>>> in
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
> >>>>> expressing
> >>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
> >>>>> reasoning
> >>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> >>>>> configurations
> >>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
> >>>>> particular,
> >>>>>>>> a
> >>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
> >>>>> configured
> >>>>>>>> for
> >>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
> >>>>> Another
> >>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
> >>>>>>>> device at
> >>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
> >>>>>>>> introduces
> >>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the
> >>>>> max
> >>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
> >>>>> storage
> >>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
> >>>>> until all
> >>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
> >>>>> over to
> >>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
> >>>>> addressed
> >>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
> >>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
> >>>>> understand
> >>>>>>>> it
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
> >>>>> ensure
> >>>>>>>> we
> >>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
> >>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
> >>>>> Monday,
> >>>>>>>> I
> >>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
> again
> >>>>>>>> that we
> >>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
> >>>>> release
> >>>>>>>> a
> >>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
> >>>>> that's
> >>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
> >>>>> read
> >>>>>>>> and
> >>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
> >>>>> don't see
> >>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore,
> we
> >>>>>>>> would
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
> >>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
> >>>>> that it
> >>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the
> 4.2
> >>>>>>>> release
> >>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
> >>>>> Total
> >>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
> >>>>> But if
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to
> do
> >>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> association
> >>>>>>>> between
> >>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
> >>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
> >>>>> decided to
> >>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
> >>>>> network
> >>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
> >>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
> >>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
> >>>>> then
> >>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
> >>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I
> will
> >>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
> >>>>> goes
> >>>>>>>> into
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
> >>>>> only
> >>>>>>>> total
> >>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch
> into
> >>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
> >>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
> >>>>> device
> >>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
> >>>>> device
> >>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
> >>>>> disk
> >>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit
> >>>>> the
> >>>>>>>> disk
> >>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
> >>>>> between
> >>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
> >>>>> rate
> >>>>>>>> for
> >>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
> >>>>> both
> >>>>>>>> the
> >>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
> >>>>> global
> >>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then
> Read
> >>>>>>>> IOps,
> >>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would
> set
> >>>>> a
> >>>>>>>> lower
> >>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think
> >>>>> it
> >>>>>>>> would
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
> >>>>> support
> >>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
> >>>>> values
> >>>>>>>> for
> >>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as
> the
> >>>>>>>> total
> >>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering,
> >>>>> and
> >>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>> https://cwiki.
> >>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
> success?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
> >>>>>>>> included
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
> potential
> >>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack
> >>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
> >>>>> migrate,
> >>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>> https://cwiki.
> >>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>>>>
> >>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
See my responses in-line.
Thanks,
-John
On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> I'm trying to picture this:
>
> "Finally, while CloudStack may be able to manage a device, an operator may
> chose to leave it unmanaged by CloudStack (e.g. the device is shared by
> many services, and the operator has chosen to dedicate only a portion of it
> to CloudStack). Does my reasoning make sense?"
>
> I guess I'm not sure how creating a SAN volume via the plug-in (before an
> attach request to the hypervisor) would work unless the hypervisor consumes
> the SAN volume in the form of, say, an SR.
My thinking is that, independent of CloudStack, an operator allocates a chunk of a SAN to CloudStack, and exposes it through a LUN. They simply want to turn control of that LUN over to CloudStack, but not allow CloudStack to allocate anymore LUNs.
>
> As the attach logic stands prior to my changes, we would be passing in a
> SAN volume that does not have the necessary hypervisor support (like an SR)
> and the logic will fail.
>
> Are you thinking we should maybe have the storage framework itself detect
> that such a SAN volume needs support from the hypervisor side and have it
> call into the agent code specifically to create the SR before the attach
> logic runs in the agent code?
I think the hypervisor management plugin should have a rich enough interface to storage to determine available for volume storage. For Xen, this interface would allow the interrogation of the device to determine the SR is present. The storage device itself shouldn't know or care that it is being used for a Xen SR -- simply be able to answer questions about it is storing.
>
>
> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
>> wrote:
>
>> So, the flow is as follows:
>>
>> * The admin registers the SolidFire driver (which is a type of so-called
>> Dynamic storage). Once this is done, a new Primary Storage shows up in the
>> applicable zone.
>>
>> * The admin creates a Disk Offering that references the storage tag of the
>> newly created Primary Storage.
>>
>> * The end user creates a CloudStack volume. This leads to a new row in the
>> cloud.volumes table.
>>
>> * The end user attaches the CloudStack volume to a VM (attach disk). This
>> leads to the storage framework calling the plug-in to create a new volume
>> on its storage system (in my case, a SAN). The plug-in also updates the
>> cloud.volumes row with applicable data (like the IQN of the SAN volume).
>> This plug-in code is only invoked if the CloudStack volume is in the
>> 'Allocated' state. After the attach, the volume will be in the 'Ready'
>> state (even after a detach disk) and the plug-in code will not be called
>> again to create this SAN volume.
>>
>> * The hypervisor-attach logic is run and detects the CloudStack volume to
>> attach needs "assistance" in the form of a hypervisor data structure (ex.
>> an SR on XenServer).
>>
>>
>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> "To ensure that we are in sync on terminology, volume, in these
>>> definitions, refers to the physical allocation on the device, correct?"
>>>
>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>>
>>> To refer to the 'volume' the end user can make in CloudStack, I try to
>>> use 'CloudStack volume'.
>>>
>>>
>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> Hi John,
>>>>
>>>> What you say here may very well make sense, but I'm having a hard time
>>>> envisioning it.
>>>>
>>>> Perhaps we should draw Edison in on this conversation as he was the
>>>> initial person to suggest the approach I took.
>>>>
>>>> What do you think?
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jb...@basho.com>wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> It feels like we are combining two distinct concepts -- storage device
>>>>> management and storage protocols. In both cases, we are communicating with
>>>>> ISCSI, but one allows the system to create/delete volumes (Dynamic) on the
>>>>> device while the other requires the volume to be volume to be managed
>>>>> outside of the CloudStack context. To ensure that we are in sync on
>>>>> terminology, volume, in these definitions, refers to the physical
>>>>> allocation on the device, correct? Minimally, we must be able to
>>>>> communicate with a storage device to move bits from one place to another,
>>>>> read bits, delete bits, etc. Optionally, a storage device may be able to
>>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI device
>>>>> onto which we store a Xen SR, and we can have a managed SolidFire iSCSI
>>>>> device on which CloudStack is capable of allocating LUNs and storing
>>>>> volumes. Finally, while CloudStack may be able to manage a device, an
>>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the device is
>>>>> shared by many services, and the operator has chosen to dedicate only a
>>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>>
>>>>> Assuming my thoughts above are reasonable, it seems appropriate to
>>>>> strip the management concerns from StoragePoolType, add the notion of a
>>>>> storage device with an attached driver that indicates whether or not is
>>>>> managed by CloudStack, and establish a abstraction representing a physical
>>>>> allocation on a device separate that is associated with a volume. With
>>>>> these notions in place, hypervisor drivers can declare which protocols they
>>>>> support and when they encounter a device managed by CloudStack, utilize the
>>>>> management operations exposed by the driver to automate allocation. If
>>>>> these thoughts/concepts make sense, then we can sit down and drill down to
>>>>> a more detailed design.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> Here is the difference between the current iSCSI type and the Dynamic
>>>>> type:
>>>>>>
>>>>>> iSCSI type: The admin has to go in and create a Primary Storage based
>>>>> on
>>>>>> the iSCSI type. At this point in time, the iSCSI volume must exist on
>>>>> the
>>>>>> storage system (it is pre-allocated). Future CloudStack volumes are
>>>>> created
>>>>>> as VDIs on the SR that was created behind the scenes.
>>>>>>
>>>>>> Dynamic type: The admin has to go in and create Primary Storage based
>>>>> on a
>>>>>> plug-in that will create and delete volumes on its storage system
>>>>>> dynamically (as is enabled via the storage framework). When a user
>>>>> wants to
>>>>>> attach a CloudStack volume that was created, the framework tells the
>>>>>> plug-in to create a new volume. After this is done, the attach logic
>>>>> for
>>>>>> the hypervisor in question is called. No hypervisor data structure
>>>>> exists
>>>>>> at this point because the volume was just created. The hypervisor data
>>>>>> structure must be created.
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com
>>>>>>> wrote:
>>>>>>
>>>>>>> These are new terms, so I should probably have defined them up front
>>>>> for
>>>>>>> you. :)
>>>>>>>
>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin creates a
>>>>>>> volume on a SAN), then a hypervisor data structure is created to
>>>>> consume
>>>>>>> the storage (ex. XenServer SR), then that hypervisor data structure
>>>>> is
>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>>>>> hypervisor
>>>>>>> data structure as needed. In these cases, the attach logic assumes
>>>>> the
>>>>>>> hypervisor data structure is already in place and simply attaches
>>>>> the VDI
>>>>>>> on the hypervisor data structure to the VM in question.
>>>>>>>
>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>>>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN,
>>>>> but the
>>>>>>> SAN itself). The hypervisor data structure must be created when an
>>>>> attach
>>>>>>> volume is performed because these types of volumes have not been
>>>>> pre-hooked
>>>>>>> up to such a hypervisor data structure by an admin. Once the attach
>>>>> logic
>>>>>>> creates, say, an SR on XenServer for this volume, it attaches the
>>>>> one and
>>>>>>> only VDI within the SR to the VM in question.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> Mike,
>>>>>>>>
>>>>>>>> The current implementation of the Dynamic type attach behavior
>>>>> works in
>>>>>>>> terms of Xen ISCSI which why I ask about the difference. Another
>>>>> way to
>>>>>>>> ask the question -- what is the definition of a Dynamic storage
>>>>> pool type?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
>>>>> when you
>>>>>>>>> want to set up Primary Storage that is directly based on an iSCSI
>>>>>>>> target.
>>>>>>>>> This allows you to skip the step of going to the hypervisor and
>>>>>>>> creating a
>>>>>>>>> storage repository based on that iSCSI target as CloudStack does
>>>>> that
>>>>>>>> part
>>>>>>>>> for you. I think this is only supported for XenServer. For all
>>>>> other
>>>>>>>>> hypervisors, you must first go to the hypervisor and perform this
>>>>> step
>>>>>>>>> manually.
>>>>>>>>>
>>>>>>>>> I don't really know what RBD is.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mike,
>>>>>>>>>>
>>>>>>>>>> Reading through the code, what is the difference between the
>>>>> ISCSI and
>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> This new type of storage is defined in the
>>>>> Storage.StoragePoolType
>>>>>>>> class
>>>>>>>>>>> (called Dynamic):
>>>>>>>>>>>
>>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>>
>>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>>
>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>>
>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>>>>>>>>>
>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>>
>>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>>
>>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>>
>>>>>>>>>>> CLVM(true),
>>>>>>>>>>>
>>>>>>>>>>> RBD(true),
>>>>>>>>>>>
>>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>>
>>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>>
>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>>>>>>>>>>> customers.
>>>>>>>>>>>
>>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>>
>>>>>>>>>>> OCFS2(true),
>>>>>>>>>>>
>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> boolean shared;
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>>
>>>>>>>>>>> this.shared = shared;
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>>
>>>>>>>>>>> return shared;
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> For example, let's say another storage company wants to
>>>>> implement a
>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It would be
>>>>>>>> dynamic,
>>>>>>>>>>>> zone-wide storage, as well. They would need only implement a
>>>>> storage
>>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>>> hypervisor-attach
>>>>>>>>>> logic
>>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific.
>>>>> It is
>>>>>>>> not.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>>>>>>>> storage
>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of storage).
>>>>> This
>>>>>>>>>> kind of
>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Edison created a new framework that supported the creation and
>>>>>>>> deletion
>>>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
>>>>> Portland
>>>>>>>>>> back
>>>>>>>>>>>>> in April, we realized that it was not complete. We realized
>>>>> there
>>>>>>>> was
>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
>>>>> attach
>>>>>>>> logic
>>>>>>>>>> was
>>>>>>>>>>>>> changed to recognize this new type of storage and create the
>>>>>>>>>> appropriate
>>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>>> jburwell@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It is generally odd to me that any operation in the Storage
>>>>> layer
>>>>>>>>>> would
>>>>>>>>>>>>>> understand or care about details. I expect to see the Storage
>>>>>>>>>> services
>>>>>>>>>>>>>> expose a set of operations that can be composed/driven by the
>>>>>>>>>> Hypervisor
>>>>>>>>>>>>>> implementations to allocate space/create structures per their
>>>>>>>> needs.
>>>>>>>>>> If
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>>>>> massive
>>>>>>>>>> n-to-n
>>>>>>>>>>>>>> problem that will make the system increasingly difficult to
>>>>>>>> maintain
>>>>>>>>>> and
>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
>>>>> code
>>>>>>>> is
>>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer,
>>>>> the
>>>>>>>> admin
>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>> to first create a volume on the storage system, then go into
>>>>> the
>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>> to set up a data structure to make use of the volume (ex. a
>>>>>>>> storage
>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
>>>>> data
>>>>>>>> disks
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> With Edison's new storage framework, storage need no longer
>>>>> be so
>>>>>>>>>>>>>> static
>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
>>>>> storage
>>>>>>>>>> system's
>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage Quality
>>>>> of
>>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
>>>>> create
>>>>>>>> and
>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The problem that the storage framework did not address is in
>>>>>>>> creating
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>>>>> performing an
>>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so. I've
>>>>> got
>>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and
>>>>> can
>>>>>>>>>> submit
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>>>>> code).
>>>>>>>> The
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a volume
>>>>> as
>>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>> when the time comes to attach the volume to a hypervisor,
>>>>> the
>>>>>>>> attach
>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>>>>>>>> zone-wide
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>> (where the volume has just been created) and create, say, a
>>>>>>>> storage
>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
>>>>> make
>>>>>>>> use
>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I've been spending most of my time recently making the
>>>>> attach
>>>>>>>> logic
>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>>>>>>>> specific?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
>>>>> that
>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM
>>>>> in.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
>>>>> what
>>>>>>>> we do
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>>>> VMware in
>>>>>>>> my
>>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>>>>>>> Offerings.
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
>>>>> post
>>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue
>>>>> as I
>>>>>>>> don't
>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
>>>>> need
>>>>>>>> to
>>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>>> service_offering
>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
>>>>> course
>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>>>>> volume,
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>>>> later. Do
>>>>>>>> you
>>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>>> ustcweizhou@gmail.com
>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
>>>>> more.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>>>>> rather
>>>>>>>> than
>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>>>>> time.
>>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write
>>>>> for
>>>>>>>> an
>>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>>>>>>>> volumes
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
>>>>> users
>>>>>>>> would
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>>>>> operator
>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>>>>> setting
>>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>>>> command
>>>>>>>> line.
>>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters
>>>>> of a
>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
>>>>> being
>>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a resource?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>>>>> commit
>>>>>>>> Mike's
>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
>>>>> Mike
>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think that
>>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think
>>>>> we need
>>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have have
>>>>>>>> solidfire
>>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>>>> captured
>>>>>>>> in
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>>>> expressing
>>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>>>> reasoning
>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>>>> configurations
>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>>>> particular,
>>>>>>>> a
>>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>>>> configured
>>>>>>>> for
>>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>>>> Another
>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>>>>>>>> device at
>>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>>>>>>> introduces
>>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the
>>>>> max
>>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>>>>> storage
>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>>>>> until all
>>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>>>>> over to
>>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>>>> addressed
>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>>>> satisfied.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>>>>>>>> SolidFire
>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>>>> understand
>>>>>>>> it
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>>>>> ensure
>>>>>>>> we
>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>>>> SolidFire are
>>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>>>>> Monday,
>>>>>>>> I
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
>>>>>>>> that we
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>>>>> release
>>>>>>>> a
>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
>>>>> that's
>>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
>>>>> read
>>>>>>>> and
>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>>>>> don't see
>>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
>>>>>>>> would
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>>>>> that it
>>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
>>>>>>>> release
>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
>>>>> Total
>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
>>>>> But if
>>>>>>>> the
>>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no association
>>>>>>>> between
>>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>>>>> decided to
>>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>>>>> network
>>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>>>>>>>> completely
>>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
>>>>> then
>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
>>>>> goes
>>>>>>>> into
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
>>>>> only
>>>>>>>> total
>>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>>>>> device
>>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>>>>> device
>>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
>>>>> disk
>>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit
>>>>> the
>>>>>>>> disk
>>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>>>>> between
>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
>>>>> rate
>>>>>>>> for
>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
>>>>> both
>>>>>>>> the
>>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
>>>>> global
>>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
>>>>>>>> IOps,
>>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set
>>>>> a
>>>>>>>> lower
>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think
>>>>> it
>>>>>>>> would
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>>>>> support
>>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>>>>> values
>>>>>>>> for
>>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as the
>>>>>>>> total
>>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering,
>>>>> and
>>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>> https://cwiki.
>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>>>>>>>> included
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack
>>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>>>> migrate,
>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>>> https://cwiki.
>>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>>>>
>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I'm trying to picture this:
"Finally, while CloudStack may be able to manage a device, an operator may
chose to leave it unmanaged by CloudStack (e.g. the device is shared by
many services, and the operator has chosen to dedicate only a portion of it
to CloudStack). Does my reasoning make sense?"
I guess I'm not sure how creating a SAN volume via the plug-in (before an
attach request to the hypervisor) would work unless the hypervisor consumes
the SAN volume in the form of, say, an SR.
As the attach logic stands prior to my changes, we would be passing in a
SAN volume that does not have the necessary hypervisor support (like an SR)
and the logic will fail.
Are you thinking we should maybe have the storage framework itself detect
that such a SAN volume needs support from the hypervisor side and have it
call into the agent code specifically to create the SR before the attach
logic runs in the agent code?
On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> So, the flow is as follows:
>
> * The admin registers the SolidFire driver (which is a type of so-called
> Dynamic storage). Once this is done, a new Primary Storage shows up in the
> applicable zone.
>
> * The admin creates a Disk Offering that references the storage tag of the
> newly created Primary Storage.
>
> * The end user creates a CloudStack volume. This leads to a new row in the
> cloud.volumes table.
>
> * The end user attaches the CloudStack volume to a VM (attach disk). This
> leads to the storage framework calling the plug-in to create a new volume
> on its storage system (in my case, a SAN). The plug-in also updates the
> cloud.volumes row with applicable data (like the IQN of the SAN volume).
> This plug-in code is only invoked if the CloudStack volume is in the
> 'Allocated' state. After the attach, the volume will be in the 'Ready'
> state (even after a detach disk) and the plug-in code will not be called
> again to create this SAN volume.
>
> * The hypervisor-attach logic is run and detects the CloudStack volume to
> attach needs "assistance" in the form of a hypervisor data structure (ex.
> an SR on XenServer).
>
>
> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> "To ensure that we are in sync on terminology, volume, in these
>> definitions, refers to the physical allocation on the device, correct?"
>>
>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>
>> To refer to the 'volume' the end user can make in CloudStack, I try to
>> use 'CloudStack volume'.
>>
>>
>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Hi John,
>>>
>>> What you say here may very well make sense, but I'm having a hard time
>>> envisioning it.
>>>
>>> Perhaps we should draw Edison in on this conversation as he was the
>>> initial person to suggest the approach I took.
>>>
>>> What do you think?
>>>
>>> Thanks!
>>>
>>>
>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jb...@basho.com>wrote:
>>>
>>>> Mike,
>>>>
>>>> It feels like we are combining two distinct concepts -- storage device
>>>> management and storage protocols. In both cases, we are communicating with
>>>> ISCSI, but one allows the system to create/delete volumes (Dynamic) on the
>>>> device while the other requires the volume to be volume to be managed
>>>> outside of the CloudStack context. To ensure that we are in sync on
>>>> terminology, volume, in these definitions, refers to the physical
>>>> allocation on the device, correct? Minimally, we must be able to
>>>> communicate with a storage device to move bits from one place to another,
>>>> read bits, delete bits, etc. Optionally, a storage device may be able to
>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI device
>>>> onto which we store a Xen SR, and we can have a managed SolidFire iSCSI
>>>> device on which CloudStack is capable of allocating LUNs and storing
>>>> volumes. Finally, while CloudStack may be able to manage a device, an
>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the device is
>>>> shared by many services, and the operator has chosen to dedicate only a
>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>
>>>> Assuming my thoughts above are reasonable, it seems appropriate to
>>>> strip the management concerns from StoragePoolType, add the notion of a
>>>> storage device with an attached driver that indicates whether or not is
>>>> managed by CloudStack, and establish a abstraction representing a physical
>>>> allocation on a device separate that is associated with a volume. With
>>>> these notions in place, hypervisor drivers can declare which protocols they
>>>> support and when they encounter a device managed by CloudStack, utilize the
>>>> management operations exposed by the driver to automate allocation. If
>>>> these thoughts/concepts make sense, then we can sit down and drill down to
>>>> a more detailed design.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>> > Here is the difference between the current iSCSI type and the Dynamic
>>>> type:
>>>> >
>>>> > iSCSI type: The admin has to go in and create a Primary Storage based
>>>> on
>>>> > the iSCSI type. At this point in time, the iSCSI volume must exist on
>>>> the
>>>> > storage system (it is pre-allocated). Future CloudStack volumes are
>>>> created
>>>> > as VDIs on the SR that was created behind the scenes.
>>>> >
>>>> > Dynamic type: The admin has to go in and create Primary Storage based
>>>> on a
>>>> > plug-in that will create and delete volumes on its storage system
>>>> > dynamically (as is enabled via the storage framework). When a user
>>>> wants to
>>>> > attach a CloudStack volume that was created, the framework tells the
>>>> > plug-in to create a new volume. After this is done, the attach logic
>>>> for
>>>> > the hypervisor in question is called. No hypervisor data structure
>>>> exists
>>>> > at this point because the volume was just created. The hypervisor data
>>>> > structure must be created.
>>>> >
>>>> >
>>>> > On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com
>>>> >> wrote:
>>>> >
>>>> >> These are new terms, so I should probably have defined them up front
>>>> for
>>>> >> you. :)
>>>> >>
>>>> >> Static storage: Storage that is pre-allocated (ex. an admin creates a
>>>> >> volume on a SAN), then a hypervisor data structure is created to
>>>> consume
>>>> >> the storage (ex. XenServer SR), then that hypervisor data structure
>>>> is
>>>> >> consumed by CloudStack. Disks (VDI) are later placed on this
>>>> hypervisor
>>>> >> data structure as needed. In these cases, the attach logic assumes
>>>> the
>>>> >> hypervisor data structure is already in place and simply attaches
>>>> the VDI
>>>> >> on the hypervisor data structure to the VM in question.
>>>> >>
>>>> >> Dynamic storage: Storage that is not pre-allocated. Instead of
>>>> >> pre-existent storage, this could be a SAN (not a volume on a SAN,
>>>> but the
>>>> >> SAN itself). The hypervisor data structure must be created when an
>>>> attach
>>>> >> volume is performed because these types of volumes have not been
>>>> pre-hooked
>>>> >> up to such a hypervisor data structure by an admin. Once the attach
>>>> logic
>>>> >> creates, say, an SR on XenServer for this volume, it attaches the
>>>> one and
>>>> >> only VDI within the SR to the VM in question.
>>>> >>
>>>> >>
>>>> >> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>> >>
>>>> >>> Mike,
>>>> >>>
>>>> >>> The current implementation of the Dynamic type attach behavior
>>>> works in
>>>> >>> terms of Xen ISCSI which why I ask about the difference. Another
>>>> way to
>>>> >>> ask the question -- what is the definition of a Dynamic storage
>>>> pool type?
>>>> >>>
>>>> >>> Thanks,
>>>> >>> -John
>>>> >>>
>>>> >>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>> >>> wrote:
>>>> >>>
>>>> >>>> As far as I know, the iSCSI type is uniquely used by XenServer
>>>> when you
>>>> >>>> want to set up Primary Storage that is directly based on an iSCSI
>>>> >>> target.
>>>> >>>> This allows you to skip the step of going to the hypervisor and
>>>> >>> creating a
>>>> >>>> storage repository based on that iSCSI target as CloudStack does
>>>> that
>>>> >>> part
>>>> >>>> for you. I think this is only supported for XenServer. For all
>>>> other
>>>> >>>> hypervisors, you must first go to the hypervisor and perform this
>>>> step
>>>> >>>> manually.
>>>> >>>>
>>>> >>>> I don't really know what RBD is.
>>>> >>>>
>>>> >>>>
>>>> >>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
>>>> >>> wrote:
>>>> >>>>
>>>> >>>>> Mike,
>>>> >>>>>
>>>> >>>>> Reading through the code, what is the difference between the
>>>> ISCSI and
>>>> >>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>> >>>>>
>>>> >>>>> Thanks,
>>>> >>>>> -John
>>>> >>>>>
>>>> >>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>> >>> mike.tutkowski@solidfire.com>
>>>> >>>>> wrote:
>>>> >>>>>
>>>> >>>>>> This new type of storage is defined in the
>>>> Storage.StoragePoolType
>>>> >>> class
>>>> >>>>>> (called Dynamic):
>>>> >>>>>>
>>>> >>>>>> public static enum StoragePoolType {
>>>> >>>>>>
>>>> >>>>>> Filesystem(false), // local directory
>>>> >>>>>>
>>>> >>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>> >>>>>>
>>>> >>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>> >>>>>>
>>>> >>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>> >>>>>>
>>>> >>>>>> ISO(false), // for iso image
>>>> >>>>>>
>>>> >>>>>> LVM(false), // XenServer local LVM SR
>>>> >>>>>>
>>>> >>>>>> CLVM(true),
>>>> >>>>>>
>>>> >>>>>> RBD(true),
>>>> >>>>>>
>>>> >>>>>> SharedMountPoint(true),
>>>> >>>>>>
>>>> >>>>>> VMFS(true), // VMware VMFS storage
>>>> >>>>>>
>>>> >>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>>>> >>>>>> customers.
>>>> >>>>>>
>>>> >>>>>> EXT(false), // XenServer local EXT SR
>>>> >>>>>>
>>>> >>>>>> OCFS2(true),
>>>> >>>>>>
>>>> >>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> boolean shared;
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> StoragePoolType(boolean shared) {
>>>> >>>>>>
>>>> >>>>>> this.shared = shared;
>>>> >>>>>>
>>>> >>>>>> }
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> public boolean isShared() {
>>>> >>>>>>
>>>> >>>>>> return shared;
>>>> >>>>>>
>>>> >>>>>> }
>>>> >>>>>>
>>>> >>>>>> }
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>> >>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>> wrote:
>>>> >>>>>>
>>>> >>>>>>> For example, let's say another storage company wants to
>>>> implement a
>>>> >>>>>>> plug-in to leverage its Quality of Service feature. It would be
>>>> >>> dynamic,
>>>> >>>>>>> zone-wide storage, as well. They would need only implement a
>>>> storage
>>>> >>>>>>> plug-in as I've made the necessary changes to the
>>>> hypervisor-attach
>>>> >>>>> logic
>>>> >>>>>>> to support their plug-in.
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>> >>>>>>>
>>>> >>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific.
>>>> It is
>>>> >>> not.
>>>> >>>>>>>>
>>>> >>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>>>> >>> storage
>>>> >>>>>>>> (and SolidFire is an implementation of this kind of storage).
>>>> This
>>>> >>>>> kind of
>>>> >>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
>>>> >>>>>>>>
>>>> >>>>>>>> Edison created a new framework that supported the creation and
>>>> >>> deletion
>>>> >>>>>>>> of volumes dynamically. However, when I visited with him in
>>>> Portland
>>>> >>>>> back
>>>> >>>>>>>> in April, we realized that it was not complete. We realized
>>>> there
>>>> >>> was
>>>> >>>>>>>> nothing CloudStack could do with these volumes unless the
>>>> attach
>>>> >>> logic
>>>> >>>>> was
>>>> >>>>>>>> changed to recognize this new type of storage and create the
>>>> >>>>> appropriate
>>>> >>>>>>>> hypervisor data structure.
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>> jburwell@basho.com>
>>>> >>>>> wrote:
>>>> >>>>>>>>
>>>> >>>>>>>>> Mike,
>>>> >>>>>>>>>
>>>> >>>>>>>>> It is generally odd to me that any operation in the Storage
>>>> layer
>>>> >>>>> would
>>>> >>>>>>>>> understand or care about details. I expect to see the Storage
>>>> >>>>> services
>>>> >>>>>>>>> expose a set of operations that can be composed/driven by the
>>>> >>>>> Hypervisor
>>>> >>>>>>>>> implementations to allocate space/create structures per their
>>>> >>> needs.
>>>> >>>>> If
>>>> >>>>>>>>> we
>>>> >>>>>>>>> don't invert this dependency, we are going to end with a
>>>> massive
>>>> >>>>> n-to-n
>>>> >>>>>>>>> problem that will make the system increasingly difficult to
>>>> >>> maintain
>>>> >>>>> and
>>>> >>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
>>>> code
>>>> >>> is
>>>> >>>>>>>>> located in the CitrixResourceBase class?
>>>> >>>>>>>>>
>>>> >>>>>>>>> Thanks,
>>>> >>>>>>>>> -John
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>> >>>>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>>> wrote:
>>>> >>>>>>>>>
>>>> >>>>>>>>>> To delve into this in a bit more detail:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer,
>>>> the
>>>> >>> admin
>>>> >>>>>>>>> had
>>>> >>>>>>>>>> to first create a volume on the storage system, then go into
>>>> the
>>>> >>>>>>>>> hypervisor
>>>> >>>>>>>>>> to set up a data structure to make use of the volume (ex. a
>>>> >>> storage
>>>> >>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
>>>> data
>>>> >>> disks
>>>> >>>>>>>>> then
>>>> >>>>>>>>>> shared this storage system's volume.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> With Edison's new storage framework, storage need no longer
>>>> be so
>>>> >>>>>>>>> static
>>>> >>>>>>>>>> and you can easily create a 1:1 relationship between a
>>>> storage
>>>> >>>>> system's
>>>> >>>>>>>>>> volume and the VM's data disk (necessary for storage Quality
>>>> of
>>>> >>>>>>>>> Service).
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> You can now write a plug-in that is called to dynamically
>>>> create
>>>> >>> and
>>>> >>>>>>>>> delete
>>>> >>>>>>>>>> volumes as needed.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> The problem that the storage framework did not address is in
>>>> >>> creating
>>>> >>>>>>>>> and
>>>> >>>>>>>>>> deleting the hypervisor-specific data structure when
>>>> performing an
>>>> >>>>>>>>>> attach/detach.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> That being the case, I've been enhancing it to do so. I've
>>>> got
>>>> >>>>>>>>> XenServer
>>>> >>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and
>>>> can
>>>> >>>>> submit
>>>> >>>>>>>>> this
>>>> >>>>>>>>>> if we extend the 4.2 freeze date.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Does that help a bit? :)
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>> >>>>>>>>>> mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>> Hi John,
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>>>> code).
>>>> >>> The
>>>> >>>>>>>>>> storage
>>>> >>>>>>>>>>> framework calls into the plug-in to have it create a volume
>>>> as
>>>> >>>>>>>>> needed,
>>>> >>>>>>>>>> but
>>>> >>>>>>>>>>> when the time comes to attach the volume to a hypervisor,
>>>> the
>>>> >>> attach
>>>> >>>>>>>>>> logic
>>>> >>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>>>> >>> zone-wide
>>>> >>>>>>>>>> storage
>>>> >>>>>>>>>>> (where the volume has just been created) and create, say, a
>>>> >>> storage
>>>> >>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
>>>> make
>>>> >>> use
>>>> >>>>>>>>> of the
>>>> >>>>>>>>>>> volume that was just created.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> I've been spending most of my time recently making the
>>>> attach
>>>> >>> logic
>>>> >>>>>>>>> work
>>>> >>>>>>>>>>> in the agent code.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Does that clear it up?
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Thanks!
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>> >>> jburwell@basho.com>
>>>> >>>>>>>>>> wrote:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>>>> >>> specific?
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
>>>> that
>>>> >>>>>>>>>> CloudStack
>>>> >>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM
>>>> in.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
>>>> what
>>>> >>> we do
>>>> >>>>>>>>>> with
>>>> >>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>>> VMware in
>>>> >>> my
>>>> >>>>>>>>>>>> sandbox,
>>>> >>>>>>>>>>>>> as well.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>>> >>> Offerings.
>>>> >>>>>>>>> I
>>>> >>>>>>>>>>>> plan to
>>>> >>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>>> >>>>>>>>>> kelcey@bbits.ca
>>>> >>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
>>>> post
>>>> >>> 4.2?
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> ----- Original Message -----
>>>> >>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>> >>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>> >>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>> >>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue
>>>> as I
>>>> >>> don't
>>>> >>>>>>>>>>>> support
>>>> >>>>>>>>>>>>>> KVM in 4.2.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>> >>> ustcweizhou@gmail.com>
>>>> >>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> John,
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
>>>> need
>>>> >>> to
>>>> >>>>>>>>>>>> calculate
>>>> >>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>> service_offering
>>>> >>> and
>>>> >>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
>>>> course
>>>> >>>>>>>>> it is
>>>> >>>>>>>>>>>>>> better
>>>> >>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>>>> volume,
>>>> >>>>>>>>> and
>>>> >>>>>>>>>>>> network
>>>> >>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>>> later. Do
>>>> >>> you
>>>> >>>>>>>>>>>> think it
>>>> >>>>>>>>>>>>>>> is a part of I/O throttling?
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Wei,
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>> ustcweizhou@gmail.com
>>>> >>>>
>>>> >>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Hi John, Mike
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
>>>> more.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>>>> rather
>>>> >>> than
>>>> >>>>>>>>>> IOPS
>>>> >>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>>>> time.
>>>> >>>>>>>>>>>>>> disk_io_stat
>>>> >>>>>>>>>>>>>>>> can
>>>> >>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write
>>>> for
>>>> >>> an
>>>> >>>>>>>>>>>>>> individual
>>>> >>>>>>>>>>>>>>>>> virtual machine.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>>>> >>> volumes
>>>> >>>>>>>>> with
>>>> >>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>> >>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
>>>> users
>>>> >>> would
>>>> >>>>>>>>> like
>>>> >>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>> option to employ similar cost models. Could an
>>>> operator
>>>> >>>>>>>>> implement
>>>> >>>>>>>>>>>>>> such a
>>>> >>>>>>>>>>>>>>>> billing model in the current patch?
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>>>> setting
>>>> >>>>>>>>> IOPS/BPS
>>>> >>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>>> command
>>>> >>> line.
>>>> >>>>>>>>>>>>>> However,
>>>> >>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters
>>>> of a
>>>> >>>>>>>>> created
>>>> >>>>>>>>>>>>>>> offering
>>>> >>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
>>>> being
>>>> >>>>>>>>>> unclear.
>>>> >>>>>>>>>>>>>> Can
>>>> >>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>>>> >>>>>>>>> interfaces
>>>> >>>>>>>>>>>> that
>>>> >>>>>>>>>>>>>>>> describe the service guarantees provided by a resource?
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
>>>> commit
>>>> >>> Mike's
>>>> >>>>>>>>>> patch
>>>> >>>>>>>>>>>>>>>> after
>>>> >>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>>> >>> limitation in
>>>> >>>>>>>>>>>>>>> hypervisor
>>>> >>>>>>>>>>>>>>>>> type, I think.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
>>>> Mike
>>>> >>>>>>>>> regarding
>>>> >>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>> touch points between these two features. I think that
>>>> >>>>>>>>>>>>>> disk_io_throttling
>>>> >>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think
>>>> we need
>>>> >>>>>>>>> closer
>>>> >>>>>>>>>>>>>>>> coordination between the branches (possibly have have
>>>> >>> solidfire
>>>> >>>>>>>>>> track
>>>> >>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> - Wei
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> Mike,
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>>> captured
>>>> >>> in
>>>> >>>>>>>>> a
>>>> >>>>>>>>>>>>>> common
>>>> >>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>>> expressing
>>>> >>>>>>>>>> offerings).
>>>> >>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>>> reasoning
>>>> >>>>>>>>> about
>>>> >>>>>>>>>>>>>> IOP
>>>> >>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>> >>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
>>>> configurations
>>>> >>>>>>>>> between
>>>> >>>>>>>>>> a
>>>> >>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>>> particular,
>>>> >>> a
>>>> >>>>>>>>>>>> scenario
>>>> >>>>>>>>>>>>>>>> where
>>>> >>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>>> configured
>>>> >>> for
>>>> >>>>>>>>> a VM
>>>> >>>>>>>>>>>>>>> than a
>>>> >>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>>> Another
>>>> >>>>>>>>> scenario
>>>> >>>>>>>>>>>>>>> could a
>>>> >>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>>>> >>> device at
>>>> >>>>>>>>>>>>>> creation
>>>> >>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>>> >>> introduces
>>>> >>>>>>>>>> logical
>>>> >>>>>>>>>>>>>>>>>> inconsistency.
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>> >>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> Hi John,
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the
>>>> max
>>>> >>>>>>>>> number of
>>>> >>>>>>>>>>>>>> IOPS
>>>> >>>>>>>>>>>>>>>> from
>>>> >>>>>>>>>>>>>>>>>> the hypervisor side.
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>>>> storage
>>>> >>>>>>>>> system
>>>> >>>>>>>>>>>>>>> side.
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>> >>>>>>>>> jburwell@basho.com
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wei,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>>>> until all
>>>> >>>>>>>>>>>>>> functional
>>>> >>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>>>> over to
>>>> >>>>>>>>> test.
>>>> >>>>>>>>>>>>>>> Until
>>>> >>>>>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>>> addressed
>>>> >>> and
>>>> >>>>>>>>>>>>>>> re-reviewed
>>>> >>>>>>>>>>>>>>>>>> by
>>>> >>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
>>>> satisfied.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>>>> >>> SolidFire
>>>> >>>>>>>>>> patch
>>>> >>>>>>>>>>>> (
>>>> >>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>>> understand
>>>> >>> it
>>>> >>>>>>>>> that
>>>> >>>>>>>>>>>>>> work
>>>> >>>>>>>>>>>>>>> is
>>>> >>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>>>> ensure
>>>> >>> we
>>>> >>>>>>>>> don't
>>>> >>>>>>>>>>>>>>> have a
>>>> >>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
>>>> SolidFire are
>>>> >>>>>>>>>>>>>>> unnecessarily
>>>> >>>>>>>>>>>>>>>>>>> incompatible.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>> >>> ustcweizhou@gmail.com
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wido,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wido,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>>>> Monday,
>>>> >>> I
>>>> >>>>>>>>> will
>>>> >>>>>>>>>>>>>>> revert
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> them.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
>>>> >>> that we
>>>> >>>>>>>>>>>> should
>>>> >>>>>>>>>>>>>>>>>> change
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>>>> release
>>>> >>> a
>>>> >>>>>>>>>> version
>>>> >>>>>>>>>>>>>>> with
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> only total IOps.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
>>>> that's
>>>> >>>>>>>>> great!
>>>> >>>>>>>>>>>> We
>>>> >>>>>>>>>>>>>>>>>> should
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wido,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
>>>> read
>>>> >>> and
>>>> >>>>>>>>> write
>>>> >>>>>>>>>>>>>>> IOPS.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> I
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>>>> don't see
>>>> >>>>>>>>> how we
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> correctly
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
>>>> >>> would
>>>> >>>>>>>>> be
>>>> >>>>>>>>>>>> stuck
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> with a
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>> >>>>>>>>> compatibility.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>>>> that it
>>>> >>>>>>>>> will go
>>>> >>>>>>>>>>>> in
>>>> >>>>>>>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
>>>> >>> release
>>>> >>>>>>>>> so
>>>> >>>>>>>>>>>> that
>>>> >>>>>>>>>>>>>>> 4.2
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
>>>> Total
>>>> >>> IOps.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
>>>> But if
>>>> >>> the
>>>> >>>>>>>>>>>> window
>>>> >>>>>>>>>>>>>>>> moves
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
>>>> >>> that.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> I also completely agree that there is no association
>>>> >>> between
>>>> >>>>>>>>>>>>>> network
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> disk I/O.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> -John
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>> >>>>>>>>> wido@widodh.nl
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Hi Wido,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
>>>> decided to
>>>> >>>>>>>>> ignore
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>>>> network
>>>> >>>>>>>>>> throttling
>>>> >>>>>>>>>>>>>>> did
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> not
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>>>> well.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>>>> >>> completely
>>>> >>>>>>>>>>>>>>> different.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
>>>> then
>>>> >>>>>>>>> network
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> bandwith.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>>>> could be
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> implemented in
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>>>> >>>>>>>>> implement
>>>> >>>>>>>>>> it
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> after
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> that , if needed.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
>>>> goes
>>>> >>> into
>>>> >>>>>>>>> the
>>>> >>>>>>>>>> 4.2
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
>>>> only
>>>> >>> total
>>>> >>>>>>>>>> IOps
>>>> >>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> fix
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> -Wei
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Hi,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>>>> >>> master.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>>>> hours.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> The purpose is :
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>>>> device
>>>> >>>>>>>>> (local
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> storage or
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>>>> device
>>>> >>>>>>>>> (such as
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> iops), if
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
>>>> disk
>>>> >>>>>>>>>>>> performance
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit
>>>> the
>>>> >>> disk
>>>> >>>>>>>>> I/O
>>>> >>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> VMs.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>>>> between
>>>> >>>>>>>>> Read
>>>> >>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Write
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> IOps.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
>>>> rate
>>>> >>> for
>>>> >>>>>>>>> Read
>>>> >>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Write
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
>>>> both
>>>> >>> the
>>>> >>>>>>>>> Read
>>>> >>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Write
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
>>>> global
>>>> >>>>>>>>> value.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
>>>> >>> IOps,
>>>> >>>>>>>>> so it
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> seems
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set
>>>> a
>>>> >>> lower
>>>> >>>>>>>>>> value
>>>> >>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> write
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think
>>>> it
>>>> >>> would
>>>> >>>>>>>>> be
>>>> >>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> great
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>>>> support
>>>> >>>>>>>>> both,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> implementing
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>>>> values
>>>> >>> for
>>>> >>>>>>>>>> read
>>>> >>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as the
>>>> >>> total
>>>> >>>>>>>>>> limit.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wido
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> The feature includes:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering,
>>>> and
>>>> >>>>>>>>> global
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> configuration)
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>> >>>>>>>>> issues.apache.org/****
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>> >>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>> >>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <**
>>>> >>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>> >>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>> <
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>> >>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>> >>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>> >>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>> >>> https://cwiki.
>>>> >>>>>>>>> **
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>> >>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>
>>>> >>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Merge check list :-
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Yes
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> No
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>>>> >>> included
>>>> >>>>>>>>> in
>>>> >>>>>>>>>> the
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> new
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> feature?
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>> >>>>>>>>> regressions?
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack
>>>> UI.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>>> migrate,
>>>> >>>>>>>>> restore
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Attach, Detach
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> git diff
>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Best regards,
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Wei
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> [1]
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
>>>> <
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>> >>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>> >>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>> >>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>> >>> https://cwiki.
>>>> >>>>>>>>> **
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>> >>>>>>>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>
>>>> >>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> [3]
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>> >>>>>>>>> <
>>>> >>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>> >>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>> >>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>> >>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <**
>>>> >>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>> >>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>>> >>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>> >>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>> >>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> **<
>>>> >>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>> >>> <
>>>> >>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <**
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>> >>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> <h**ttps://
>>>> >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>> >>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
>>>> >
>>>> >>>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> --
>>>> >>>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> --
>>>> >>>>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>>>> o: 303.746.7302
>>>> >>>>>>>>>> Advancing the way the world uses the
>>>> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>>>> *â„¢*
>>>> >>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> --
>>>> >>>>>>>> *Mike Tutkowski*
>>>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>>> o: 303.746.7302
>>>> >>>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>>> *â„¢*
>>>> >>>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> --
>>>> >>>>>>> *Mike Tutkowski*
>>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>>> o: 303.746.7302
>>>> >>>>>>> Advancing the way the world uses the cloud<
>>>> >>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>>> *â„¢*
>>>> >>>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> *Mike Tutkowski*
>>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>>>> e: mike.tutkowski@solidfire.com
>>>> >>>>>> o: 303.746.7302
>>>> >>>>>> Advancing the way the world uses the
>>>> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>>>>> *â„¢*
>>>> >>>>>
>>>> >>>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> *Mike Tutkowski*
>>>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>>> e: mike.tutkowski@solidfire.com
>>>> >>>> o: 303.746.7302
>>>> >>>> Advancing the way the world uses the
>>>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> >>>> *â„¢*
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> *Mike Tutkowski*
>>>> >> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >> e: mike.tutkowski@solidfire.com
>>>> >> o: 303.746.7302
>>>> >> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>> >> *â„¢*
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > *Mike Tutkowski*
>>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>>> > e: mike.tutkowski@solidfire.com
>>>> > o: 303.746.7302
>>>> > Advancing the way the world uses the
>>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>>> > *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"Just to clarify my understanding, multiple CloudStack volumes can be
stored on a single SAN volume. If so, how does the Management Server
determine when to create a SAN volume?"
This might be where we're missing each other. :)
A main purpose for Edison's storage framework changes were to enable a
single CloudStack volume to reside by itself on a single SAN volume.
Quality of Service (Qos) features like SolidFire's (and those of other
storage vendors) require this 1:1 mapping. That way you can tune the SAN
volume's IOPS and guarantee that the one and only data disk using that SAN
volume (the CloudStack volume) has the proper QoS characteristics. If you
share a single SAN volume among more than one CloudStack volume, you can
suffer from the "Noisy Neighbor" problem, which is what QoS aimes to solve.
So, in short:
One SAN volume (meaning one LUN) is mapped to one and only CloudStack
volume.
On Tue, Jun 4, 2013 at 1:13 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> To me, the key is to recognize the difference between the two types of
> volumes. So, in Dynamic, the system is capable of allocation space on the
> device (in SolidFire's case, does this mean that a LUN gets created as
> well?), as well as, move bits around. As I look though it, it seems
> reasonable to split the management portion of the StoragePoolType from the
> communicate portion.
>
> On Jun 4, 2013, at 3:01 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > So, the flow is as follows:
> >
> > * The admin registers the SolidFire driver (which is a type of so-called
> > Dynamic storage). Once this is done, a new Primary Storage shows up in
> the
> > applicable zone.
> >
> > * The admin creates a Disk Offering that references the storage tag of
> the
> > newly created Primary Storage.
>
> Does this become a template for creating a SAN volume on the SolidFire
> device?
>
> > * The end user creates a CloudStack volume. This leads to a new row in
> the
> > cloud.volumes table.
> >
> > * The end user attaches the CloudStack volume to a VM (attach disk). This
> > leads to the storage framework calling the plug-in to create a new volume
> > on its storage system (in my case, a SAN). The plug-in also updates the
> > cloud.volumes row with applicable data (like the IQN of the SAN volume).
> > This plug-in code is only invoked if the CloudStack volume is in the
> > 'Allocated' state. After the attach, the volume will be in the 'Ready'
> > state (even after a detach disk) and the plug-in code will not be called
> > again to create this SAN volume.
>
> Just to clarify my understanding, multiple CloudStack volumes can be
> stored on a single SAN volume. If so, how does the Management Server
> determine when to create a SAN volume?
>
> >
> > * The hypervisor-attach logic is run and detects the CloudStack volume to
> > attach needs "assistance" in the form of a hypervisor data structure (ex.
> > an SR on XenServer).
> >
> >
> > On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> >> "To ensure that we are in sync on terminology, volume, in these
> >> definitions, refers to the physical allocation on the device, correct?"
> >>
> >> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>
> >> To refer to the 'volume' the end user can make in CloudStack, I try to
> use
> >> 'CloudStack volume'.
> >>
> >>
> >> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> Hi John,
> >>>
> >>> What you say here may very well make sense, but I'm having a hard time
> >>> envisioning it.
> >>>
> >>> Perhaps we should draw Edison in on this conversation as he was the
> >>> initial person to suggest the approach I took.
> >>>
> >>> What do you think?
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jb...@basho.com>
> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> It feels like we are combining two distinct concepts -- storage device
> >>>> management and storage protocols. In both cases, we are
> communicating with
> >>>> ISCSI, but one allows the system to create/delete volumes (Dynamic)
> on the
> >>>> device while the other requires the volume to be volume to be managed
> >>>> outside of the CloudStack context. To ensure that we are in sync on
> >>>> terminology, volume, in these definitions, refers to the physical
> >>>> allocation on the device, correct? Minimally, we must be able to
> >>>> communicate with a storage device to move bits from one place to
> another,
> >>>> read bits, delete bits, etc. Optionally, a storage device may be
> able to
> >>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI
> device
> >>>> onto which we store a Xen SR, and we can have a managed SolidFire
> iSCSI
> >>>> device on which CloudStack is capable of allocating LUNs and storing
> >>>> volumes. Finally, while CloudStack may be able to manage a device, an
> >>>> operator may chose to leave it unmanaged by CloudStack (e.g. the
> device is
> >>>> shared by many services, and the operator has chosen to dedicate only
> a
> >>>> portion of it to CloudStack). Does my reasoning make sense?
> >>>>
> >>>> Assuming my thoughts above are reasonable, it seems appropriate to
> strip
> >>>> the management concerns from StoragePoolType, add the notion of a
> storage
> >>>> device with an attached driver that indicates whether or not is
> managed by
> >>>> CloudStack, and establish a abstraction representing a physical
> allocation
> >>>> on a device separate that is associated with a volume. With these
> notions
> >>>> in place, hypervisor drivers can declare which protocols they support
> and
> >>>> when they encounter a device managed by CloudStack, utilize the
> management
> >>>> operations exposed by the driver to automate allocation. If these
> >>>> thoughts/concepts make sense, then we can sit down and drill down to
> a more
> >>>> detailed design.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> Here is the difference between the current iSCSI type and the Dynamic
> >>>> type:
> >>>>>
> >>>>> iSCSI type: The admin has to go in and create a Primary Storage based
> >>>> on
> >>>>> the iSCSI type. At this point in time, the iSCSI volume must exist on
> >>>> the
> >>>>> storage system (it is pre-allocated). Future CloudStack volumes are
> >>>> created
> >>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>
> >>>>> Dynamic type: The admin has to go in and create Primary Storage based
> >>>> on a
> >>>>> plug-in that will create and delete volumes on its storage system
> >>>>> dynamically (as is enabled via the storage framework). When a user
> >>>> wants to
> >>>>> attach a CloudStack volume that was created, the framework tells the
> >>>>> plug-in to create a new volume. After this is done, the attach logic
> >>>> for
> >>>>> the hypervisor in question is called. No hypervisor data structure
> >>>> exists
> >>>>> at this point because the volume was just created. The hypervisor
> data
> >>>>> structure must be created.
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com
> >>>>>> wrote:
> >>>>>
> >>>>>> These are new terms, so I should probably have defined them up front
> >>>> for
> >>>>>> you. :)
> >>>>>>
> >>>>>> Static storage: Storage that is pre-allocated (ex. an admin creates
> a
> >>>>>> volume on a SAN), then a hypervisor data structure is created to
> >>>> consume
> >>>>>> the storage (ex. XenServer SR), then that hypervisor data structure
> is
> >>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
> >>>> hypervisor
> >>>>>> data structure as needed. In these cases, the attach logic assumes
> the
> >>>>>> hypervisor data structure is already in place and simply attaches
> the
> >>>> VDI
> >>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>
> >>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
> >>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN,
> but
> >>>> the
> >>>>>> SAN itself). The hypervisor data structure must be created when an
> >>>> attach
> >>>>>> volume is performed because these types of volumes have not been
> >>>> pre-hooked
> >>>>>> up to such a hypervisor data structure by an admin. Once the attach
> >>>> logic
> >>>>>> creates, say, an SR on XenServer for this volume, it attaches the
> one
> >>>> and
> >>>>>> only VDI within the SR to the VM in question.
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> The current implementation of the Dynamic type attach behavior
> works
> >>>> in
> >>>>>>> terms of Xen ISCSI which why I ask about the difference. Another
> >>>> way to
> >>>>>>> ask the question -- what is the definition of a Dynamic storage
> pool
> >>>> type?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer
> when
> >>>> you
> >>>>>>>> want to set up Primary Storage that is directly based on an iSCSI
> >>>>>>> target.
> >>>>>>>> This allows you to skip the step of going to the hypervisor and
> >>>>>>> creating a
> >>>>>>>> storage repository based on that iSCSI target as CloudStack does
> >>>> that
> >>>>>>> part
> >>>>>>>> for you. I think this is only supported for XenServer. For all
> other
> >>>>>>>> hypervisors, you must first go to the hypervisor and perform this
> >>>> step
> >>>>>>>> manually.
> >>>>>>>>
> >>>>>>>> I don't really know what RBD is.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> Reading through the code, what is the difference between the
> ISCSI
> >>>> and
> >>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> This new type of storage is defined in the
> Storage.StoragePoolType
> >>>>>>> class
> >>>>>>>>>> (called Dynamic):
> >>>>>>>>>>
> >>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>
> >>>>>>>>>> Filesystem(false), // local directory
> >>>>>>>>>>
> >>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>
> >>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
> >>>>>>>>>>
> >>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>
> >>>>>>>>>> ISO(false), // for iso image
> >>>>>>>>>>
> >>>>>>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>>>>>
> >>>>>>>>>> CLVM(true),
> >>>>>>>>>>
> >>>>>>>>>> RBD(true),
> >>>>>>>>>>
> >>>>>>>>>> SharedMountPoint(true),
> >>>>>>>>>>
> >>>>>>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>>>>>
> >>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
> >>>>>>>>>> customers.
> >>>>>>>>>>
> >>>>>>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>>>>>
> >>>>>>>>>> OCFS2(true),
> >>>>>>>>>>
> >>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> boolean shared;
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> StoragePoolType(boolean shared) {
> >>>>>>>>>>
> >>>>>>>>>> this.shared = shared;
> >>>>>>>>>>
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> public boolean isShared() {
> >>>>>>>>>>
> >>>>>>>>>> return shared;
> >>>>>>>>>>
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> For example, let's say another storage company wants to
> >>>> implement a
> >>>>>>>>>>> plug-in to leverage its Quality of Service feature. It would be
> >>>>>>> dynamic,
> >>>>>>>>>>> zone-wide storage, as well. They would need only implement a
> >>>> storage
> >>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>> hypervisor-attach
> >>>>>>>>> logic
> >>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific.
> It
> >>>> is
> >>>>>>> not.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
> >>>>>>> storage
> >>>>>>>>>>>> (and SolidFire is an implementation of this kind of storage).
> >>>> This
> >>>>>>>>> kind of
> >>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Edison created a new framework that supported the creation and
> >>>>>>> deletion
> >>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
> >>>> Portland
> >>>>>>>>> back
> >>>>>>>>>>>> in April, we realized that it was not complete. We realized
> >>>> there
> >>>>>>> was
> >>>>>>>>>>>> nothing CloudStack could do with these volumes unless the
> attach
> >>>>>>> logic
> >>>>>>>>> was
> >>>>>>>>>>>> changed to recognize this new type of storage and create the
> >>>>>>>>> appropriate
> >>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>> jburwell@basho.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It is generally odd to me that any operation in the Storage
> >>>> layer
> >>>>>>>>> would
> >>>>>>>>>>>>> understand or care about details. I expect to see the
> Storage
> >>>>>>>>> services
> >>>>>>>>>>>>> expose a set of operations that can be composed/driven by the
> >>>>>>>>> Hypervisor
> >>>>>>>>>>>>> implementations to allocate space/create structures per their
> >>>>>>> needs.
> >>>>>>>>> If
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>> don't invert this dependency, we are going to end with a
> >>>> massive
> >>>>>>>>> n-to-n
> >>>>>>>>>>>>> problem that will make the system increasingly difficult to
> >>>>>>> maintain
> >>>>>>>>> and
> >>>>>>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
> >>>> code
> >>>>>>> is
> >>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer,
> >>>> the
> >>>>>>> admin
> >>>>>>>>>>>>> had
> >>>>>>>>>>>>>> to first create a volume on the storage system, then go into
> >>>> the
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>> to set up a data structure to make use of the volume (ex. a
> >>>>>>> storage
> >>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
> >>>> data
> >>>>>>> disks
> >>>>>>>>>>>>> then
> >>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> With Edison's new storage framework, storage need no longer
> >>>> be so
> >>>>>>>>>>>>> static
> >>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a
> storage
> >>>>>>>>> system's
> >>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage Quality
> >>>> of
> >>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
> >>>> create
> >>>>>>> and
> >>>>>>>>>>>>> delete
> >>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The problem that the storage framework did not address is in
> >>>>>>> creating
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
> >>>> performing an
> >>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> That being the case, I've been enhancing it to do so. I've
> got
> >>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and
> >>>> can
> >>>>>>>>> submit
> >>>>>>>>>>>>> this
> >>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
> >>>> code).
> >>>>>>> The
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>> framework calls into the plug-in to have it create a volume
> >>>> as
> >>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>> when the time comes to attach the volume to a hypervisor,
> the
> >>>>>>> attach
> >>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
> >>>>>>> zone-wide
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>> (where the volume has just been created) and create, say, a
> >>>>>>> storage
> >>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
> >>>> make
> >>>>>>> use
> >>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I've been spending most of my time recently making the
> attach
> >>>>>>> logic
> >>>>>>>>>>>>> work
> >>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
> >>>>>>> specific?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
> >>>> that
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM
> >>>> in.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
> >>>> what
> >>>>>>> we do
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
> >>>> VMware in
> >>>>>>> my
> >>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
> >>>>>>> Offerings.
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
> >>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
> >>>> post
> >>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue
> as
> >>>> I
> >>>>>>> don't
> >>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
> >>>> need
> >>>>>>> to
> >>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>> service_offering
> >>>>>>> and
> >>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
> >>>> course
> >>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of individual
> >>>> volume,
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
> >>>> later. Do
> >>>>>>> you
> >>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>> ustcweizhou@gmail.com
> >>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
> >>>> more.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
> >>>> rather
> >>>>>>> than
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
> >>>> time.
> >>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write
> >>>> for
> >>>>>>> an
> >>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
> >>>>>>> volumes
> >>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
> >>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our
> users
> >>>>>>> would
> >>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an
> operator
> >>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
> >>>> setting
> >>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
> >>>> command
> >>>>>>> line.
> >>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters
> of
> >>>> a
> >>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
> >>>> being
> >>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
> >>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
> resource?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to
> commit
> >>>>>>> Mike's
> >>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
> >>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to
> Mike
> >>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> touch points between these two features. I think that
> >>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think
> we
> >>>> need
> >>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have have
> >>>>>>> solidfire
> >>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
> >>>> captured
> >>>>>>> in
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
> >>>> expressing
> >>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
> >>>> reasoning
> >>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> configurations
> >>>>>>>>>>>>> between
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
> >>>> particular,
> >>>>>>> a
> >>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
> >>>> configured
> >>>>>>> for
> >>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
> >>>> Another
> >>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
> >>>>>>> device at
> >>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
> >>>>>>> introduces
> >>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the
> max
> >>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
> >>>> storage
> >>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
> >>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
> >>>> until all
> >>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
> >>>> over to
> >>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
> >>>> addressed
> >>>>>>> and
> >>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
> >>>>>>> SolidFire
> >>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
> >>>> understand
> >>>>>>> it
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
> >>>> ensure
> >>>>>>> we
> >>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
> SolidFire
> >>>> are
> >>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
> >>>> Monday,
> >>>>>>> I
> >>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention
> again
> >>>>>>> that we
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
> >>>> release
> >>>>>>> a
> >>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
> >>>> that's
> >>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
> >>>> read
> >>>>>>> and
> >>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
> >>>> don't see
> >>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore,
> we
> >>>>>>> would
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
> >>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
> >>>> that it
> >>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
> >>>>>>> release
> >>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of
> Total
> >>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
> >>>> But if
> >>>>>>> the
> >>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to
> do
> >>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> association
> >>>>>>> between
> >>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
> >>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I
> decided
> >>>> to
> >>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
> >>>> network
> >>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
> >>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
> >>>>>>> completely
> >>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
> >>>> then
> >>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
> >>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I
> will
> >>>>>>>>>>>>> implement
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it
> goes
> >>>>>>> into
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
> >>>> only
> >>>>>>> total
> >>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch
> into
> >>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
> >>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
> >>>> device
> >>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
> >>>> device
> >>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
> >>>> disk
> >>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit
> the
> >>>>>>> disk
> >>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
> >>>> between
> >>>>>>>>>>>>> Read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
> >>>> rate
> >>>>>>> for
> >>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
> >>>> both
> >>>>>>> the
> >>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
> >>>> global
> >>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then
> Read
> >>>>>>> IOps,
> >>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would
> set a
> >>>>>>> lower
> >>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think
> it
> >>>>>>> would
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
> >>>> support
> >>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
> >>>> values
> >>>>>>> for
> >>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as
> the
> >>>>>>> total
> >>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering,
> >>>> and
> >>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>> https://cwiki.
> >>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
> >>>>>>> included
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
> >>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack
> >>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
> >>>> migrate,
> >>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>> https://cwiki.
> >>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/CLOUDSTACK-1301
> >>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>
> >>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
To me, the key is to recognize the difference between the two types of volumes. So, in Dynamic, the system is capable of allocation space on the device (in SolidFire's case, does this mean that a LUN gets created as well?), as well as, move bits around. As I look though it, it seems reasonable to split the management portion of the StoragePoolType from the communicate portion.
On Jun 4, 2013, at 3:01 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> So, the flow is as follows:
>
> * The admin registers the SolidFire driver (which is a type of so-called
> Dynamic storage). Once this is done, a new Primary Storage shows up in the
> applicable zone.
>
> * The admin creates a Disk Offering that references the storage tag of the
> newly created Primary Storage.
Does this become a template for creating a SAN volume on the SolidFire device?
> * The end user creates a CloudStack volume. This leads to a new row in the
> cloud.volumes table.
>
> * The end user attaches the CloudStack volume to a VM (attach disk). This
> leads to the storage framework calling the plug-in to create a new volume
> on its storage system (in my case, a SAN). The plug-in also updates the
> cloud.volumes row with applicable data (like the IQN of the SAN volume).
> This plug-in code is only invoked if the CloudStack volume is in the
> 'Allocated' state. After the attach, the volume will be in the 'Ready'
> state (even after a detach disk) and the plug-in code will not be called
> again to create this SAN volume.
Just to clarify my understanding, multiple CloudStack volumes can be stored on a single SAN volume. If so, how does the Management Server determine when to create a SAN volume?
>
> * The hypervisor-attach logic is run and detects the CloudStack volume to
> attach needs "assistance" in the form of a hypervisor data structure (ex.
> an SR on XenServer).
>
>
> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> "To ensure that we are in sync on terminology, volume, in these
>> definitions, refers to the physical allocation on the device, correct?"
>>
>> Yes...when I say 'volume', I try to mean 'SAN volume'.
>>
>> To refer to the 'volume' the end user can make in CloudStack, I try to use
>> 'CloudStack volume'.
>>
>>
>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Hi John,
>>>
>>> What you say here may very well make sense, but I'm having a hard time
>>> envisioning it.
>>>
>>> Perhaps we should draw Edison in on this conversation as he was the
>>> initial person to suggest the approach I took.
>>>
>>> What do you think?
>>>
>>> Thanks!
>>>
>>>
>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jb...@basho.com> wrote:
>>>
>>>> Mike,
>>>>
>>>> It feels like we are combining two distinct concepts -- storage device
>>>> management and storage protocols. In both cases, we are communicating with
>>>> ISCSI, but one allows the system to create/delete volumes (Dynamic) on the
>>>> device while the other requires the volume to be volume to be managed
>>>> outside of the CloudStack context. To ensure that we are in sync on
>>>> terminology, volume, in these definitions, refers to the physical
>>>> allocation on the device, correct? Minimally, we must be able to
>>>> communicate with a storage device to move bits from one place to another,
>>>> read bits, delete bits, etc. Optionally, a storage device may be able to
>>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI device
>>>> onto which we store a Xen SR, and we can have a managed SolidFire iSCSI
>>>> device on which CloudStack is capable of allocating LUNs and storing
>>>> volumes. Finally, while CloudStack may be able to manage a device, an
>>>> operator may chose to leave it unmanaged by CloudStack (e.g. the device is
>>>> shared by many services, and the operator has chosen to dedicate only a
>>>> portion of it to CloudStack). Does my reasoning make sense?
>>>>
>>>> Assuming my thoughts above are reasonable, it seems appropriate to strip
>>>> the management concerns from StoragePoolType, add the notion of a storage
>>>> device with an attached driver that indicates whether or not is managed by
>>>> CloudStack, and establish a abstraction representing a physical allocation
>>>> on a device separate that is associated with a volume. With these notions
>>>> in place, hypervisor drivers can declare which protocols they support and
>>>> when they encounter a device managed by CloudStack, utilize the management
>>>> operations exposed by the driver to automate allocation. If these
>>>> thoughts/concepts make sense, then we can sit down and drill down to a more
>>>> detailed design.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <mi...@solidfire.com>
>>>> wrote:
>>>>
>>>>> Here is the difference between the current iSCSI type and the Dynamic
>>>> type:
>>>>>
>>>>> iSCSI type: The admin has to go in and create a Primary Storage based
>>>> on
>>>>> the iSCSI type. At this point in time, the iSCSI volume must exist on
>>>> the
>>>>> storage system (it is pre-allocated). Future CloudStack volumes are
>>>> created
>>>>> as VDIs on the SR that was created behind the scenes.
>>>>>
>>>>> Dynamic type: The admin has to go in and create Primary Storage based
>>>> on a
>>>>> plug-in that will create and delete volumes on its storage system
>>>>> dynamically (as is enabled via the storage framework). When a user
>>>> wants to
>>>>> attach a CloudStack volume that was created, the framework tells the
>>>>> plug-in to create a new volume. After this is done, the attach logic
>>>> for
>>>>> the hypervisor in question is called. No hypervisor data structure
>>>> exists
>>>>> at this point because the volume was just created. The hypervisor data
>>>>> structure must be created.
>>>>>
>>>>>
>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com
>>>>>> wrote:
>>>>>
>>>>>> These are new terms, so I should probably have defined them up front
>>>> for
>>>>>> you. :)
>>>>>>
>>>>>> Static storage: Storage that is pre-allocated (ex. an admin creates a
>>>>>> volume on a SAN), then a hypervisor data structure is created to
>>>> consume
>>>>>> the storage (ex. XenServer SR), then that hypervisor data structure is
>>>>>> consumed by CloudStack. Disks (VDI) are later placed on this
>>>> hypervisor
>>>>>> data structure as needed. In these cases, the attach logic assumes the
>>>>>> hypervisor data structure is already in place and simply attaches the
>>>> VDI
>>>>>> on the hypervisor data structure to the VM in question.
>>>>>>
>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead of
>>>>>> pre-existent storage, this could be a SAN (not a volume on a SAN, but
>>>> the
>>>>>> SAN itself). The hypervisor data structure must be created when an
>>>> attach
>>>>>> volume is performed because these types of volumes have not been
>>>> pre-hooked
>>>>>> up to such a hypervisor data structure by an admin. Once the attach
>>>> logic
>>>>>> creates, say, an SR on XenServer for this volume, it attaches the one
>>>> and
>>>>>> only VDI within the SR to the VM in question.
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> The current implementation of the Dynamic type attach behavior works
>>>> in
>>>>>>> terms of Xen ISCSI which why I ask about the difference. Another
>>>> way to
>>>>>>> ask the question -- what is the definition of a Dynamic storage pool
>>>> type?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> As far as I know, the iSCSI type is uniquely used by XenServer when
>>>> you
>>>>>>>> want to set up Primary Storage that is directly based on an iSCSI
>>>>>>> target.
>>>>>>>> This allows you to skip the step of going to the hypervisor and
>>>>>>> creating a
>>>>>>>> storage repository based on that iSCSI target as CloudStack does
>>>> that
>>>>>>> part
>>>>>>>> for you. I think this is only supported for XenServer. For all other
>>>>>>>> hypervisors, you must first go to the hypervisor and perform this
>>>> step
>>>>>>>> manually.
>>>>>>>>
>>>>>>>> I don't really know what RBD is.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> Reading through the code, what is the difference between the ISCSI
>>>> and
>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> This new type of storage is defined in the Storage.StoragePoolType
>>>>>>> class
>>>>>>>>>> (called Dynamic):
>>>>>>>>>>
>>>>>>>>>> public static enum StoragePoolType {
>>>>>>>>>>
>>>>>>>>>> Filesystem(false), // local directory
>>>>>>>>>>
>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>>>>>
>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>>>>>>>>
>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>>>>>
>>>>>>>>>> ISO(false), // for iso image
>>>>>>>>>>
>>>>>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>>>>>
>>>>>>>>>> CLVM(true),
>>>>>>>>>>
>>>>>>>>>> RBD(true),
>>>>>>>>>>
>>>>>>>>>> SharedMountPoint(true),
>>>>>>>>>>
>>>>>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>>>>>
>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>>>>>>>>>> customers.
>>>>>>>>>>
>>>>>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>>>>>
>>>>>>>>>> OCFS2(true),
>>>>>>>>>>
>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> boolean shared;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> StoragePoolType(boolean shared) {
>>>>>>>>>>
>>>>>>>>>> this.shared = shared;
>>>>>>>>>>
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> public boolean isShared() {
>>>>>>>>>>
>>>>>>>>>> return shared;
>>>>>>>>>>
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> For example, let's say another storage company wants to
>>>> implement a
>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It would be
>>>>>>> dynamic,
>>>>>>>>>>> zone-wide storage, as well. They would need only implement a
>>>> storage
>>>>>>>>>>> plug-in as I've made the necessary changes to the
>>>> hypervisor-attach
>>>>>>>>> logic
>>>>>>>>>>> to support their plug-in.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It
>>>> is
>>>>>>> not.
>>>>>>>>>>>>
>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>>>>>>> storage
>>>>>>>>>>>> (and SolidFire is an implementation of this kind of storage).
>>>> This
>>>>>>>>> kind of
>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
>>>>>>>>>>>>
>>>>>>>>>>>> Edison created a new framework that supported the creation and
>>>>>>> deletion
>>>>>>>>>>>> of volumes dynamically. However, when I visited with him in
>>>> Portland
>>>>>>>>> back
>>>>>>>>>>>> in April, we realized that it was not complete. We realized
>>>> there
>>>>>>> was
>>>>>>>>>>>> nothing CloudStack could do with these volumes unless the attach
>>>>>>> logic
>>>>>>>>> was
>>>>>>>>>>>> changed to recognize this new type of storage and create the
>>>>>>>>> appropriate
>>>>>>>>>>>> hypervisor data structure.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>>> jburwell@basho.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> It is generally odd to me that any operation in the Storage
>>>> layer
>>>>>>>>> would
>>>>>>>>>>>>> understand or care about details. I expect to see the Storage
>>>>>>>>> services
>>>>>>>>>>>>> expose a set of operations that can be composed/driven by the
>>>>>>>>> Hypervisor
>>>>>>>>>>>>> implementations to allocate space/create structures per their
>>>>>>> needs.
>>>>>>>>> If
>>>>>>>>>>>>> we
>>>>>>>>>>>>> don't invert this dependency, we are going to end with a
>>>> massive
>>>>>>>>> n-to-n
>>>>>>>>>>>>> problem that will make the system increasingly difficult to
>>>>>>> maintain
>>>>>>>>> and
>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
>>>> code
>>>>>>> is
>>>>>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer,
>>>> the
>>>>>>> admin
>>>>>>>>>>>>> had
>>>>>>>>>>>>>> to first create a volume on the storage system, then go into
>>>> the
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>> to set up a data structure to make use of the volume (ex. a
>>>>>>> storage
>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
>>>> data
>>>>>>> disks
>>>>>>>>>>>>> then
>>>>>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> With Edison's new storage framework, storage need no longer
>>>> be so
>>>>>>>>>>>>> static
>>>>>>>>>>>>>> and you can easily create a 1:1 relationship between a storage
>>>>>>>>> system's
>>>>>>>>>>>>>> volume and the VM's data disk (necessary for storage Quality
>>>> of
>>>>>>>>>>>>> Service).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You can now write a plug-in that is called to dynamically
>>>> create
>>>>>>> and
>>>>>>>>>>>>> delete
>>>>>>>>>>>>>> volumes as needed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The problem that the storage framework did not address is in
>>>>>>> creating
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> deleting the hypervisor-specific data structure when
>>>> performing an
>>>>>>>>>>>>>> attach/detach.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That being the case, I've been enhancing it to do so. I've got
>>>>>>>>>>>>> XenServer
>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and
>>>> can
>>>>>>>>> submit
>>>>>>>>>>>>> this
>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>>>> code).
>>>>>>> The
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>> framework calls into the plug-in to have it create a volume
>>>> as
>>>>>>>>>>>>> needed,
>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>> when the time comes to attach the volume to a hypervisor, the
>>>>>>> attach
>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>>>>>>> zone-wide
>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>> (where the volume has just been created) and create, say, a
>>>>>>> storage
>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
>>>> make
>>>>>>> use
>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've been spending most of my time recently making the attach
>>>>>>> logic
>>>>>>>>>>>>> work
>>>>>>>>>>>>>>> in the agent code.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>>>>>> jburwell@basho.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>>>>>>> specific?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
>>>> that
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM
>>>> in.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
>>>> what
>>>>>>> we do
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>>> VMware in
>>>>>>> my
>>>>>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>>>>>> Offerings.
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> plan to
>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>>>>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
>>>> post
>>>>>>> 4.2?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue as
>>>> I
>>>>>>> don't
>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>>>>>> ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
>>>> need
>>>>>>> to
>>>>>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
>>>> service_offering
>>>>>>> and
>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
>>>> course
>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>>>> volume,
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>>> later. Do
>>>>>>> you
>>>>>>>>>>>>>>>> think it
>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>>> ustcweizhou@gmail.com
>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
>>>> more.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>>>> rather
>>>>>>> than
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>>>> time.
>>>>>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write
>>>> for
>>>>>>> an
>>>>>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>>>>>>> volumes
>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users
>>>>>>> would
>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Could an operator
>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>>>> setting
>>>>>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>>> command
>>>>>>> line.
>>>>>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters of
>>>> a
>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
>>>> being
>>>>>>>>>>>>>> unclear.
>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>>>>>>>>>>>>> interfaces
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a resource?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
>>>>>>> Mike's
>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>>>>>> limitation in
>>>>>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> touch points between these two features. I think that
>>>>>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think we
>>>> need
>>>>>>>>>>>>> closer
>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly have have
>>>>>>> solidfire
>>>>>>>>>>>>>> track
>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>>> captured
>>>>>>> in
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>>> expressing
>>>>>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>>> reasoning
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>>>>>>>>>>>>> between
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>>> particular,
>>>>>>> a
>>>>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>>> configured
>>>>>>> for
>>>>>>>>>>>>> a VM
>>>>>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>>> Another
>>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>>>>>>> device at
>>>>>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>>>>>> introduces
>>>>>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>>>>>>>>>>>>> number of
>>>>>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>>>> storage
>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>>>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>>>> until all
>>>>>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>>>> over to
>>>>>>>>>>>>> test.
>>>>>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>>> addressed
>>>>>>> and
>>>>>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>>>>>>> SolidFire
>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>>> understand
>>>>>>> it
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>>>> ensure
>>>>>>> we
>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire
>>>> are
>>>>>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>>>>>> ustcweizhou@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>>>> Monday,
>>>>>>> I
>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
>>>>>>> that we
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>>>> release
>>>>>>> a
>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
>>>> that's
>>>>>>>>>>>>> great!
>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
>>>> read
>>>>>>> and
>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>>>> don't see
>>>>>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
>>>>>>> would
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>>>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>>>> that it
>>>>>>>>>>>>> will go
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
>>>>>>> release
>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of Total
>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
>>>> But if
>>>>>>> the
>>>>>>>>>>>>>>>> window
>>>>>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no association
>>>>>>> between
>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>>>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided
>>>> to
>>>>>>>>>>>>> ignore
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>>>> network
>>>>>>>>>>>>>> throttling
>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>>>>>>> completely
>>>>>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
>>>> then
>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>>>> could be
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>>>>>>>>>>>>> implement
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes
>>>>>>> into
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
>>>> only
>>>>>>> total
>>>>>>>>>>>>>> IOps
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>>>>>>> master.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>>>> device
>>>>>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>>>> device
>>>>>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
>>>> disk
>>>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
>>>>>>> disk
>>>>>>>>>>>>> I/O
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>>>> between
>>>>>>>>>>>>> Read
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
>>>> rate
>>>>>>> for
>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
>>>> both
>>>>>>> the
>>>>>>>>>>>>> Read
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
>>>> global
>>>>>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
>>>>>>> IOps,
>>>>>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a
>>>>>>> lower
>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it
>>>>>>> would
>>>>>>>>>>>>> be
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>>>> support
>>>>>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>>>> values
>>>>>>> for
>>>>>>>>>>>>>> read
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as the
>>>>>>> total
>>>>>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering,
>>>> and
>>>>>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>> https://cwiki.
>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>>>>>>> included
>>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>>>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack
>>>> UI.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>>> migrate,
>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> git diff
>>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>>>>>> https://cwiki.
>>>>>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301
>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>>>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>>>>>> cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
So, the flow is as follows:
* The admin registers the SolidFire driver (which is a type of so-called
Dynamic storage). Once this is done, a new Primary Storage shows up in the
applicable zone.
* The admin creates a Disk Offering that references the storage tag of the
newly created Primary Storage.
* The end user creates a CloudStack volume. This leads to a new row in the
cloud.volumes table.
* The end user attaches the CloudStack volume to a VM (attach disk). This
leads to the storage framework calling the plug-in to create a new volume
on its storage system (in my case, a SAN). The plug-in also updates the
cloud.volumes row with applicable data (like the IQN of the SAN volume).
This plug-in code is only invoked if the CloudStack volume is in the
'Allocated' state. After the attach, the volume will be in the 'Ready'
state (even after a detach disk) and the plug-in code will not be called
again to create this SAN volume.
* The hypervisor-attach logic is run and detects the CloudStack volume to
attach needs "assistance" in the form of a hypervisor data structure (ex.
an SR on XenServer).
On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> "To ensure that we are in sync on terminology, volume, in these
> definitions, refers to the physical allocation on the device, correct?"
>
> Yes...when I say 'volume', I try to mean 'SAN volume'.
>
> To refer to the 'volume' the end user can make in CloudStack, I try to use
> 'CloudStack volume'.
>
>
> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Hi John,
>>
>> What you say here may very well make sense, but I'm having a hard time
>> envisioning it.
>>
>> Perhaps we should draw Edison in on this conversation as he was the
>> initial person to suggest the approach I took.
>>
>> What do you think?
>>
>> Thanks!
>>
>>
>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> It feels like we are combining two distinct concepts -- storage device
>>> management and storage protocols. In both cases, we are communicating with
>>> ISCSI, but one allows the system to create/delete volumes (Dynamic) on the
>>> device while the other requires the volume to be volume to be managed
>>> outside of the CloudStack context. To ensure that we are in sync on
>>> terminology, volume, in these definitions, refers to the physical
>>> allocation on the device, correct? Minimally, we must be able to
>>> communicate with a storage device to move bits from one place to another,
>>> read bits, delete bits, etc. Optionally, a storage device may be able to
>>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI device
>>> onto which we store a Xen SR, and we can have a managed SolidFire iSCSI
>>> device on which CloudStack is capable of allocating LUNs and storing
>>> volumes. Finally, while CloudStack may be able to manage a device, an
>>> operator may chose to leave it unmanaged by CloudStack (e.g. the device is
>>> shared by many services, and the operator has chosen to dedicate only a
>>> portion of it to CloudStack). Does my reasoning make sense?
>>>
>>> Assuming my thoughts above are reasonable, it seems appropriate to strip
>>> the management concerns from StoragePoolType, add the notion of a storage
>>> device with an attached driver that indicates whether or not is managed by
>>> CloudStack, and establish a abstraction representing a physical allocation
>>> on a device separate that is associated with a volume. With these notions
>>> in place, hypervisor drivers can declare which protocols they support and
>>> when they encounter a device managed by CloudStack, utilize the management
>>> operations exposed by the driver to automate allocation. If these
>>> thoughts/concepts make sense, then we can sit down and drill down to a more
>>> detailed design.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>> > Here is the difference between the current iSCSI type and the Dynamic
>>> type:
>>> >
>>> > iSCSI type: The admin has to go in and create a Primary Storage based
>>> on
>>> > the iSCSI type. At this point in time, the iSCSI volume must exist on
>>> the
>>> > storage system (it is pre-allocated). Future CloudStack volumes are
>>> created
>>> > as VDIs on the SR that was created behind the scenes.
>>> >
>>> > Dynamic type: The admin has to go in and create Primary Storage based
>>> on a
>>> > plug-in that will create and delete volumes on its storage system
>>> > dynamically (as is enabled via the storage framework). When a user
>>> wants to
>>> > attach a CloudStack volume that was created, the framework tells the
>>> > plug-in to create a new volume. After this is done, the attach logic
>>> for
>>> > the hypervisor in question is called. No hypervisor data structure
>>> exists
>>> > at this point because the volume was just created. The hypervisor data
>>> > structure must be created.
>>> >
>>> >
>>> > On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com
>>> >> wrote:
>>> >
>>> >> These are new terms, so I should probably have defined them up front
>>> for
>>> >> you. :)
>>> >>
>>> >> Static storage: Storage that is pre-allocated (ex. an admin creates a
>>> >> volume on a SAN), then a hypervisor data structure is created to
>>> consume
>>> >> the storage (ex. XenServer SR), then that hypervisor data structure is
>>> >> consumed by CloudStack. Disks (VDI) are later placed on this
>>> hypervisor
>>> >> data structure as needed. In these cases, the attach logic assumes the
>>> >> hypervisor data structure is already in place and simply attaches the
>>> VDI
>>> >> on the hypervisor data structure to the VM in question.
>>> >>
>>> >> Dynamic storage: Storage that is not pre-allocated. Instead of
>>> >> pre-existent storage, this could be a SAN (not a volume on a SAN, but
>>> the
>>> >> SAN itself). The hypervisor data structure must be created when an
>>> attach
>>> >> volume is performed because these types of volumes have not been
>>> pre-hooked
>>> >> up to such a hypervisor data structure by an admin. Once the attach
>>> logic
>>> >> creates, say, an SR on XenServer for this volume, it attaches the one
>>> and
>>> >> only VDI within the SR to the VM in question.
>>> >>
>>> >>
>>> >> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>> >>
>>> >>> Mike,
>>> >>>
>>> >>> The current implementation of the Dynamic type attach behavior works
>>> in
>>> >>> terms of Xen ISCSI which why I ask about the difference. Another
>>> way to
>>> >>> ask the question -- what is the definition of a Dynamic storage pool
>>> type?
>>> >>>
>>> >>> Thanks,
>>> >>> -John
>>> >>>
>>> >>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>> >>> wrote:
>>> >>>
>>> >>>> As far as I know, the iSCSI type is uniquely used by XenServer when
>>> you
>>> >>>> want to set up Primary Storage that is directly based on an iSCSI
>>> >>> target.
>>> >>>> This allows you to skip the step of going to the hypervisor and
>>> >>> creating a
>>> >>>> storage repository based on that iSCSI target as CloudStack does
>>> that
>>> >>> part
>>> >>>> for you. I think this is only supported for XenServer. For all other
>>> >>>> hypervisors, you must first go to the hypervisor and perform this
>>> step
>>> >>>> manually.
>>> >>>>
>>> >>>> I don't really know what RBD is.
>>> >>>>
>>> >>>>
>>> >>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
>>> >>> wrote:
>>> >>>>
>>> >>>>> Mike,
>>> >>>>>
>>> >>>>> Reading through the code, what is the difference between the ISCSI
>>> and
>>> >>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>> >>>>>
>>> >>>>> Thanks,
>>> >>>>> -John
>>> >>>>>
>>> >>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>> >>> mike.tutkowski@solidfire.com>
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>>> This new type of storage is defined in the Storage.StoragePoolType
>>> >>> class
>>> >>>>>> (called Dynamic):
>>> >>>>>>
>>> >>>>>> public static enum StoragePoolType {
>>> >>>>>>
>>> >>>>>> Filesystem(false), // local directory
>>> >>>>>>
>>> >>>>>> NetworkFilesystem(true), // NFS or CIFS
>>> >>>>>>
>>> >>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>> >>>>>>
>>> >>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>> >>>>>>
>>> >>>>>> ISO(false), // for iso image
>>> >>>>>>
>>> >>>>>> LVM(false), // XenServer local LVM SR
>>> >>>>>>
>>> >>>>>> CLVM(true),
>>> >>>>>>
>>> >>>>>> RBD(true),
>>> >>>>>>
>>> >>>>>> SharedMountPoint(true),
>>> >>>>>>
>>> >>>>>> VMFS(true), // VMware VMFS storage
>>> >>>>>>
>>> >>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>>> >>>>>> customers.
>>> >>>>>>
>>> >>>>>> EXT(false), // XenServer local EXT SR
>>> >>>>>>
>>> >>>>>> OCFS2(true),
>>> >>>>>>
>>> >>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> boolean shared;
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> StoragePoolType(boolean shared) {
>>> >>>>>>
>>> >>>>>> this.shared = shared;
>>> >>>>>>
>>> >>>>>> }
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> public boolean isShared() {
>>> >>>>>>
>>> >>>>>> return shared;
>>> >>>>>>
>>> >>>>>> }
>>> >>>>>>
>>> >>>>>> }
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>> >>>>> mike.tutkowski@solidfire.com
>>> >>>>>>> wrote:
>>> >>>>>>
>>> >>>>>>> For example, let's say another storage company wants to
>>> implement a
>>> >>>>>>> plug-in to leverage its Quality of Service feature. It would be
>>> >>> dynamic,
>>> >>>>>>> zone-wide storage, as well. They would need only implement a
>>> storage
>>> >>>>>>> plug-in as I've made the necessary changes to the
>>> hypervisor-attach
>>> >>>>> logic
>>> >>>>>>> to support their plug-in.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>>> >>>>>>>
>>> >>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It
>>> is
>>> >>> not.
>>> >>>>>>>>
>>> >>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>>> >>> storage
>>> >>>>>>>> (and SolidFire is an implementation of this kind of storage).
>>> This
>>> >>>>> kind of
>>> >>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
>>> >>>>>>>>
>>> >>>>>>>> Edison created a new framework that supported the creation and
>>> >>> deletion
>>> >>>>>>>> of volumes dynamically. However, when I visited with him in
>>> Portland
>>> >>>>> back
>>> >>>>>>>> in April, we realized that it was not complete. We realized
>>> there
>>> >>> was
>>> >>>>>>>> nothing CloudStack could do with these volumes unless the attach
>>> >>> logic
>>> >>>>> was
>>> >>>>>>>> changed to recognize this new type of storage and create the
>>> >>>>> appropriate
>>> >>>>>>>> hypervisor data structure.
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
>>> jburwell@basho.com>
>>> >>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> Mike,
>>> >>>>>>>>>
>>> >>>>>>>>> It is generally odd to me that any operation in the Storage
>>> layer
>>> >>>>> would
>>> >>>>>>>>> understand or care about details. I expect to see the Storage
>>> >>>>> services
>>> >>>>>>>>> expose a set of operations that can be composed/driven by the
>>> >>>>> Hypervisor
>>> >>>>>>>>> implementations to allocate space/create structures per their
>>> >>> needs.
>>> >>>>> If
>>> >>>>>>>>> we
>>> >>>>>>>>> don't invert this dependency, we are going to end with a
>>> massive
>>> >>>>> n-to-n
>>> >>>>>>>>> problem that will make the system increasingly difficult to
>>> >>> maintain
>>> >>>>> and
>>> >>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
>>> code
>>> >>> is
>>> >>>>>>>>> located in the CitrixResourceBase class?
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks,
>>> >>>>>>>>> -John
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>> >>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>> To delve into this in a bit more detail:
>>> >>>>>>>>>>
>>> >>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer,
>>> the
>>> >>> admin
>>> >>>>>>>>> had
>>> >>>>>>>>>> to first create a volume on the storage system, then go into
>>> the
>>> >>>>>>>>> hypervisor
>>> >>>>>>>>>> to set up a data structure to make use of the volume (ex. a
>>> >>> storage
>>> >>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and
>>> data
>>> >>> disks
>>> >>>>>>>>> then
>>> >>>>>>>>>> shared this storage system's volume.
>>> >>>>>>>>>>
>>> >>>>>>>>>> With Edison's new storage framework, storage need no longer
>>> be so
>>> >>>>>>>>> static
>>> >>>>>>>>>> and you can easily create a 1:1 relationship between a storage
>>> >>>>> system's
>>> >>>>>>>>>> volume and the VM's data disk (necessary for storage Quality
>>> of
>>> >>>>>>>>> Service).
>>> >>>>>>>>>>
>>> >>>>>>>>>> You can now write a plug-in that is called to dynamically
>>> create
>>> >>> and
>>> >>>>>>>>> delete
>>> >>>>>>>>>> volumes as needed.
>>> >>>>>>>>>>
>>> >>>>>>>>>> The problem that the storage framework did not address is in
>>> >>> creating
>>> >>>>>>>>> and
>>> >>>>>>>>>> deleting the hypervisor-specific data structure when
>>> performing an
>>> >>>>>>>>>> attach/detach.
>>> >>>>>>>>>>
>>> >>>>>>>>>> That being the case, I've been enhancing it to do so. I've got
>>> >>>>>>>>> XenServer
>>> >>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and
>>> can
>>> >>>>> submit
>>> >>>>>>>>> this
>>> >>>>>>>>>> if we extend the 4.2 freeze date.
>>> >>>>>>>>>>
>>> >>>>>>>>>> Does that help a bit? :)
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>> >>>>>>>>>> mike.tutkowski@solidfire.com
>>> >>>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> Hi John,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> The issue is with the volume-attach logic (in the agent
>>> code).
>>> >>> The
>>> >>>>>>>>>> storage
>>> >>>>>>>>>>> framework calls into the plug-in to have it create a volume
>>> as
>>> >>>>>>>>> needed,
>>> >>>>>>>>>> but
>>> >>>>>>>>>>> when the time comes to attach the volume to a hypervisor, the
>>> >>> attach
>>> >>>>>>>>>> logic
>>> >>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>>> >>> zone-wide
>>> >>>>>>>>>> storage
>>> >>>>>>>>>>> (where the volume has just been created) and create, say, a
>>> >>> storage
>>> >>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to
>>> make
>>> >>> use
>>> >>>>>>>>> of the
>>> >>>>>>>>>>> volume that was just created.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I've been spending most of my time recently making the attach
>>> >>> logic
>>> >>>>>>>>> work
>>> >>>>>>>>>>> in the agent code.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Does that clear it up?
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks!
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>> >>> jburwell@basho.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>>> >>> specific?
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>> -John
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors
>>> that
>>> >>>>>>>>>> CloudStack
>>> >>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM
>>> in.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on
>>> what
>>> >>> we do
>>> >>>>>>>>>> with
>>> >>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for
>>> VMware in
>>> >>> my
>>> >>>>>>>>>>>> sandbox,
>>> >>>>>>>>>>>>> as well.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>> >>> Offerings.
>>> >>>>>>>>> I
>>> >>>>>>>>>>>> plan to
>>> >>>>>>>>>>>>> support Compute Offerings post 4.2.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>> >>>>>>>>>> kelcey@bbits.ca
>>> >>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle
>>> post
>>> >>> 4.2?
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> ----- Original Message -----
>>> >>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>> >>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>> >>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>> >>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue as
>>> I
>>> >>> don't
>>> >>>>>>>>>>>> support
>>> >>>>>>>>>>>>>> KVM in 4.2.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>> >>> ustcweizhou@gmail.com>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> John,
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
>>> need
>>> >>> to
>>> >>>>>>>>>>>> calculate
>>> >>>>>>>>>>>>>>> the billing by themselves. They can get the
>>> service_offering
>>> >>> and
>>> >>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
>>> course
>>> >>>>>>>>> it is
>>> >>>>>>>>>>>>>> better
>>> >>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>>> volume,
>>> >>>>>>>>> and
>>> >>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it
>>> later. Do
>>> >>> you
>>> >>>>>>>>>>>> think it
>>> >>>>>>>>>>>>>>> is a part of I/O throttling?
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Agree with what you said about the two features.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Wei,
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>>> ustcweizhou@gmail.com
>>> >>>>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Hi John, Mike
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
>>> more.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>>> rather
>>> >>> than
>>> >>>>>>>>>> IOPS
>>> >>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have
>>> time.
>>> >>>>>>>>>>>>>> disk_io_stat
>>> >>>>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write
>>> for
>>> >>> an
>>> >>>>>>>>>>>>>> individual
>>> >>>>>>>>>>>>>>>>> virtual machine.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>>> >>> volumes
>>> >>>>>>>>> with
>>> >>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>> >>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users
>>> >>> would
>>> >>>>>>>>> like
>>> >>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>> option to employ similar cost models. Could an operator
>>> >>>>>>>>> implement
>>> >>>>>>>>>>>>>> such a
>>> >>>>>>>>>>>>>>>> billing model in the current patch?
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>>> setting
>>> >>>>>>>>> IOPS/BPS
>>> >>>>>>>>>>>>>>>>> limitation for a running virtual machine through
>>> command
>>> >>> line.
>>> >>>>>>>>>>>>>> However,
>>> >>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters of
>>> a
>>> >>>>>>>>> created
>>> >>>>>>>>>>>>>>> offering
>>> >>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
>>> being
>>> >>>>>>>>>> unclear.
>>> >>>>>>>>>>>>>> Can
>>> >>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>>> >>>>>>>>> interfaces
>>> >>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>>> describe the service guarantees provided by a resource?
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
>>> >>> Mike's
>>> >>>>>>>>>> patch
>>> >>>>>>>>>>>>>>>> after
>>> >>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>> >>> limitation in
>>> >>>>>>>>>>>>>>> hypervisor
>>> >>>>>>>>>>>>>>>>> type, I think.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>>> >>>>>>>>> regarding
>>> >>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>> touch points between these two features. I think that
>>> >>>>>>>>>>>>>> disk_io_throttling
>>> >>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think we
>>> need
>>> >>>>>>>>> closer
>>> >>>>>>>>>>>>>>>> coordination between the branches (possibly have have
>>> >>> solidfire
>>> >>>>>>>>>> track
>>> >>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> - Wei
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Mike,
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>>> captured
>>> >>> in
>>> >>>>>>>>> a
>>> >>>>>>>>>>>>>> common
>>> >>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes,
>>> expressing
>>> >>>>>>>>>> offerings).
>>> >>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>>> reasoning
>>> >>>>>>>>> about
>>> >>>>>>>>>>>>>> IOP
>>> >>>>>>>>>>>>>>>>>> provisioning at runtime?
>>> >>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>>> >>>>>>>>> between
>>> >>>>>>>>>> a
>>> >>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>>> particular,
>>> >>> a
>>> >>>>>>>>>>>> scenario
>>> >>>>>>>>>>>>>>>> where
>>> >>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>>> configured
>>> >>> for
>>> >>>>>>>>> a VM
>>> >>>>>>>>>>>>>>> than a
>>> >>>>>>>>>>>>>>>>>> storage device has been configured to deliver.
>>> Another
>>> >>>>>>>>> scenario
>>> >>>>>>>>>>>>>>> could a
>>> >>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>>> >>> device at
>>> >>>>>>>>>>>>>> creation
>>> >>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>> >>> introduces
>>> >>>>>>>>>> logical
>>> >>>>>>>>>>>>>>>>>> inconsistency.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>> >>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Hi John,
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>>> >>>>>>>>> number of
>>> >>>>>>>>>>>>>> IOPS
>>> >>>>>>>>>>>>>>>> from
>>> >>>>>>>>>>>>>>>>>> the hypervisor side.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>>> storage
>>> >>>>>>>>> system
>>> >>>>>>>>>>>>>>> side.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> I hope that helps. :)
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>> >>>>>>>>> jburwell@basho.com
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wei,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged
>>> until all
>>> >>>>>>>>>>>>>> functional
>>> >>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>>> over to
>>> >>>>>>>>> test.
>>> >>>>>>>>>>>>>>> Until
>>> >>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>>> addressed
>>> >>> and
>>> >>>>>>>>>>>>>>> re-reviewed
>>> >>>>>>>>>>>>>>>>>> by
>>> >>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>>> >>> SolidFire
>>> >>>>>>>>>> patch
>>> >>>>>>>>>>>> (
>>> >>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>>> understand
>>> >>> it
>>> >>>>>>>>> that
>>> >>>>>>>>>>>>>> work
>>> >>>>>>>>>>>>>>> is
>>> >>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>>> ensure
>>> >>> we
>>> >>>>>>>>> don't
>>> >>>>>>>>>>>>>>> have a
>>> >>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire
>>> are
>>> >>>>>>>>>>>>>>> unnecessarily
>>> >>>>>>>>>>>>>>>>>>> incompatible.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>> >>> ustcweizhou@gmail.com
>>> >>>>>>>>>>
>>> >>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>>> Monday,
>>> >>> I
>>> >>>>>>>>> will
>>> >>>>>>>>>>>>>>> revert
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> them.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
>>> >>> that we
>>> >>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>> change
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>>> release
>>> >>> a
>>> >>>>>>>>>> version
>>> >>>>>>>>>>>>>>> with
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> only total IOps.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
>>> that's
>>> >>>>>>>>> great!
>>> >>>>>>>>>>>> We
>>> >>>>>>>>>>>>>>>>>> should
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wido,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support
>>> read
>>> >>> and
>>> >>>>>>>>> write
>>> >>>>>>>>>>>>>>> IOPS.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> I
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I
>>> don't see
>>> >>>>>>>>> how we
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> correctly
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
>>> >>> would
>>> >>>>>>>>> be
>>> >>>>>>>>>>>> stuck
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> with a
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>> >>>>>>>>> compatibility.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so
>>> that it
>>> >>>>>>>>> will go
>>> >>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
>>> >>> release
>>> >>>>>>>>> so
>>> >>>>>>>>>>>> that
>>> >>>>>>>>>>>>>>> 4.2
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of Total
>>> >>> IOps.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date.
>>> But if
>>> >>> the
>>> >>>>>>>>>>>> window
>>> >>>>>>>>>>>>>>>> moves
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
>>> >>> that.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> I also completely agree that there is no association
>>> >>> between
>>> >>>>>>>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> disk I/O.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> -John
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>> >>>>>>>>> wido@widodh.nl
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Hi Wido,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided
>>> to
>>> >>>>>>>>> ignore
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>>> network
>>> >>>>>>>>>> throttling
>>> >>>>>>>>>>>>>>> did
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> not
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>>> well.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>>> >>> completely
>>> >>>>>>>>>>>>>>> different.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
>>> then
>>> >>>>>>>>> network
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> bandwith.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>>> could be
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> implemented in
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>>> >>>>>>>>> implement
>>> >>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> after
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> that , if needed.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes
>>> >>> into
>>> >>>>>>>>> the
>>> >>>>>>>>>> 4.2
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with
>>> only
>>> >>> total
>>> >>>>>>>>>> IOps
>>> >>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> fix
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> -Wei
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Hi,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>>> >>> master.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>>> hours.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> The purpose is :
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>>> device
>>> >>>>>>>>> (local
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> storage or
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>>> device
>>> >>>>>>>>> (such as
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> iops), if
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
>>> disk
>>> >>>>>>>>>>>> performance
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
>>> >>> disk
>>> >>>>>>>>> I/O
>>> >>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> VMs.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>>> between
>>> >>>>>>>>> Read
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> IOps.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different
>>> rate
>>> >>> for
>>> >>>>>>>>> Read
>>> >>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
>>> both
>>> >>> the
>>> >>>>>>>>> Read
>>> >>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Write
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
>>> global
>>> >>>>>>>>> value.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
>>> >>> IOps,
>>> >>>>>>>>> so it
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> seems
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a
>>> >>> lower
>>> >>>>>>>>>> value
>>> >>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> write
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it
>>> >>> would
>>> >>>>>>>>> be
>>> >>>>>>>>>> of
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> great
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>>> support
>>> >>>>>>>>> both,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> implementing
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>>> values
>>> >>> for
>>> >>>>>>>>>> read
>>> >>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as the
>>> >>> total
>>> >>>>>>>>>> limit.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wido
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> The feature includes:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering,
>>> and
>>> >>>>>>>>> global
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> configuration)
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>> >>>>>>>>> issues.apache.org/****
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>>>
>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>> >
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>
>>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>
>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> >>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>
>>> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>> >>>>>>>>>>>> cwiki.apache.org/confluence/****
>>> >>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> >>> https://cwiki.
>>> >>>>>>>>> **
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> >>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>
>>> >>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Merge check list :-
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Yes
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> No
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>>> >>> included
>>> >>>>>>>>> in
>>> >>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> new
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> feature?
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Unit tests are added.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>> >>>>>>>>> regressions?
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack
>>> UI.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>>> migrate,
>>> >>>>>>>>> restore
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Attach, Detach
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> To review the code, you can try
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> git diff
>>> c30057635d04a2396f84c588127d7e******be42e503a7
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Best regards,
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Wei
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> [1]
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>
>>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>
>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> >>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>
>>> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>> >>>>>>>>>>>> cwiki.apache.org/confluence/****
>>> >>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> >>> https://cwiki.
>>> >>>>>>>>> **
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> >>>>>>>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>
>>> >>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> [3]
>>> >>>>>>>>>>>>>>>>>>>
>>> >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>> >>>>>>>>> <
>>> >>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>> >>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>>>
>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301
>>> >
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>>> >>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>> >>>>>>>>>>>>>>>>>>>
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>> >>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> **<
>>> >>>>>>>>>>>>>>>>>>>
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>> >>> <
>>> >>>>>>>>>>>>>>>>>>
>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <**
>>> >>>>>>>>>>>>>>>>>>>
>>> >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>> >>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> <h**ttps://
>>> >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>> >>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>>>>>> cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>>> *â„¢*
>>> >>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> --
>>> >>>>>>>>>> *Mike Tutkowski*
>>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>>>> o: 303.746.7302
>>> >>>>>>>>>> Advancing the way the world uses the
>>> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>>>> *â„¢*
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>> *Mike Tutkowski*
>>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>>> o: 303.746.7302
>>> >>>>>>>> Advancing the way the world uses the cloud<
>>> >>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>>> *â„¢*
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> *Mike Tutkowski*
>>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>>> o: 303.746.7302
>>> >>>>>>> Advancing the way the world uses the cloud<
>>> >>>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>>>>> *â„¢*
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> *Mike Tutkowski*
>>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>>>> e: mike.tutkowski@solidfire.com
>>> >>>>>> o: 303.746.7302
>>> >>>>>> Advancing the way the world uses the
>>> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>>>> *â„¢*
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> *Mike Tutkowski*
>>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>> e: mike.tutkowski@solidfire.com
>>> >>>> o: 303.746.7302
>>> >>>> Advancing the way the world uses the
>>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>> *â„¢*
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> *Mike Tutkowski*
>>> >> *Senior CloudStack Developer, SolidFire Inc.*
>>> >> e: mike.tutkowski@solidfire.com
>>> >> o: 303.746.7302
>>> >> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >> *â„¢*
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
"To ensure that we are in sync on terminology, volume, in these
definitions, refers to the physical allocation on the device, correct?"
Yes...when I say 'volume', I try to mean 'SAN volume'.
To refer to the 'volume' the end user can make in CloudStack, I try to use
'CloudStack volume'.
On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Hi John,
>
> What you say here may very well make sense, but I'm having a hard time
> envisioning it.
>
> Perhaps we should draw Edison in on this conversation as he was the
> initial person to suggest the approach I took.
>
> What do you think?
>
> Thanks!
>
>
> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> It feels like we are combining two distinct concepts -- storage device
>> management and storage protocols. In both cases, we are communicating with
>> ISCSI, but one allows the system to create/delete volumes (Dynamic) on the
>> device while the other requires the volume to be volume to be managed
>> outside of the CloudStack context. To ensure that we are in sync on
>> terminology, volume, in these definitions, refers to the physical
>> allocation on the device, correct? Minimally, we must be able to
>> communicate with a storage device to move bits from one place to another,
>> read bits, delete bits, etc. Optionally, a storage device may be able to
>> managed by CloudStack. Therefore, we can have a unmanaged iSCSI device
>> onto which we store a Xen SR, and we can have a managed SolidFire iSCSI
>> device on which CloudStack is capable of allocating LUNs and storing
>> volumes. Finally, while CloudStack may be able to manage a device, an
>> operator may chose to leave it unmanaged by CloudStack (e.g. the device is
>> shared by many services, and the operator has chosen to dedicate only a
>> portion of it to CloudStack). Does my reasoning make sense?
>>
>> Assuming my thoughts above are reasonable, it seems appropriate to strip
>> the management concerns from StoragePoolType, add the notion of a storage
>> device with an attached driver that indicates whether or not is managed by
>> CloudStack, and establish a abstraction representing a physical allocation
>> on a device separate that is associated with a volume. With these notions
>> in place, hypervisor drivers can declare which protocols they support and
>> when they encounter a device managed by CloudStack, utilize the management
>> operations exposed by the driver to automate allocation. If these
>> thoughts/concepts make sense, then we can sit down and drill down to a more
>> detailed design.
>>
>> Thanks,
>> -John
>>
>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > Here is the difference between the current iSCSI type and the Dynamic
>> type:
>> >
>> > iSCSI type: The admin has to go in and create a Primary Storage based on
>> > the iSCSI type. At this point in time, the iSCSI volume must exist on
>> the
>> > storage system (it is pre-allocated). Future CloudStack volumes are
>> created
>> > as VDIs on the SR that was created behind the scenes.
>> >
>> > Dynamic type: The admin has to go in and create Primary Storage based
>> on a
>> > plug-in that will create and delete volumes on its storage system
>> > dynamically (as is enabled via the storage framework). When a user
>> wants to
>> > attach a CloudStack volume that was created, the framework tells the
>> > plug-in to create a new volume. After this is done, the attach logic for
>> > the hypervisor in question is called. No hypervisor data structure
>> exists
>> > at this point because the volume was just created. The hypervisor data
>> > structure must be created.
>> >
>> >
>> > On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>> >> wrote:
>> >
>> >> These are new terms, so I should probably have defined them up front
>> for
>> >> you. :)
>> >>
>> >> Static storage: Storage that is pre-allocated (ex. an admin creates a
>> >> volume on a SAN), then a hypervisor data structure is created to
>> consume
>> >> the storage (ex. XenServer SR), then that hypervisor data structure is
>> >> consumed by CloudStack. Disks (VDI) are later placed on this hypervisor
>> >> data structure as needed. In these cases, the attach logic assumes the
>> >> hypervisor data structure is already in place and simply attaches the
>> VDI
>> >> on the hypervisor data structure to the VM in question.
>> >>
>> >> Dynamic storage: Storage that is not pre-allocated. Instead of
>> >> pre-existent storage, this could be a SAN (not a volume on a SAN, but
>> the
>> >> SAN itself). The hypervisor data structure must be created when an
>> attach
>> >> volume is performed because these types of volumes have not been
>> pre-hooked
>> >> up to such a hypervisor data structure by an admin. Once the attach
>> logic
>> >> creates, say, an SR on XenServer for this volume, it attaches the one
>> and
>> >> only VDI within the SR to the VM in question.
>> >>
>> >>
>> >> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >>
>> >>> Mike,
>> >>>
>> >>> The current implementation of the Dynamic type attach behavior works
>> in
>> >>> terms of Xen ISCSI which why I ask about the difference. Another way
>> to
>> >>> ask the question -- what is the definition of a Dynamic storage pool
>> type?
>> >>>
>> >>> Thanks,
>> >>> -John
>> >>>
>> >>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >>> wrote:
>> >>>
>> >>>> As far as I know, the iSCSI type is uniquely used by XenServer when
>> you
>> >>>> want to set up Primary Storage that is directly based on an iSCSI
>> >>> target.
>> >>>> This allows you to skip the step of going to the hypervisor and
>> >>> creating a
>> >>>> storage repository based on that iSCSI target as CloudStack does that
>> >>> part
>> >>>> for you. I think this is only supported for XenServer. For all other
>> >>>> hypervisors, you must first go to the hypervisor and perform this
>> step
>> >>>> manually.
>> >>>>
>> >>>> I don't really know what RBD is.
>> >>>>
>> >>>>
>> >>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
>> >>> wrote:
>> >>>>
>> >>>>> Mike,
>> >>>>>
>> >>>>> Reading through the code, what is the difference between the ISCSI
>> and
>> >>>>> Dynamic types? Why isn't RBD considered Dynamic?
>> >>>>>
>> >>>>> Thanks,
>> >>>>> -John
>> >>>>>
>> >>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> >>> mike.tutkowski@solidfire.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> This new type of storage is defined in the Storage.StoragePoolType
>> >>> class
>> >>>>>> (called Dynamic):
>> >>>>>>
>> >>>>>> public static enum StoragePoolType {
>> >>>>>>
>> >>>>>> Filesystem(false), // local directory
>> >>>>>>
>> >>>>>> NetworkFilesystem(true), // NFS or CIFS
>> >>>>>>
>> >>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>> >>>>>>
>> >>>>>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>>>>
>> >>>>>> ISO(false), // for iso image
>> >>>>>>
>> >>>>>> LVM(false), // XenServer local LVM SR
>> >>>>>>
>> >>>>>> CLVM(true),
>> >>>>>>
>> >>>>>> RBD(true),
>> >>>>>>
>> >>>>>> SharedMountPoint(true),
>> >>>>>>
>> >>>>>> VMFS(true), // VMware VMFS storage
>> >>>>>>
>> >>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>> >>>>>> customers.
>> >>>>>>
>> >>>>>> EXT(false), // XenServer local EXT SR
>> >>>>>>
>> >>>>>> OCFS2(true),
>> >>>>>>
>> >>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>> >>>>>>
>> >>>>>>
>> >>>>>> boolean shared;
>> >>>>>>
>> >>>>>>
>> >>>>>> StoragePoolType(boolean shared) {
>> >>>>>>
>> >>>>>> this.shared = shared;
>> >>>>>>
>> >>>>>> }
>> >>>>>>
>> >>>>>>
>> >>>>>> public boolean isShared() {
>> >>>>>>
>> >>>>>> return shared;
>> >>>>>>
>> >>>>>> }
>> >>>>>>
>> >>>>>> }
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com
>> >>>>>>> wrote:
>> >>>>>>
>> >>>>>>> For example, let's say another storage company wants to implement
>> a
>> >>>>>>> plug-in to leverage its Quality of Service feature. It would be
>> >>> dynamic,
>> >>>>>>> zone-wide storage, as well. They would need only implement a
>> storage
>> >>>>>>> plug-in as I've made the necessary changes to the
>> hypervisor-attach
>> >>>>> logic
>> >>>>>>> to support their plug-in.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>
>> >>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It
>> is
>> >>> not.
>> >>>>>>>>
>> >>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>> >>> storage
>> >>>>>>>> (and SolidFire is an implementation of this kind of storage).
>> This
>> >>>>> kind of
>> >>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
>> >>>>>>>>
>> >>>>>>>> Edison created a new framework that supported the creation and
>> >>> deletion
>> >>>>>>>> of volumes dynamically. However, when I visited with him in
>> Portland
>> >>>>> back
>> >>>>>>>> in April, we realized that it was not complete. We realized there
>> >>> was
>> >>>>>>>> nothing CloudStack could do with these volumes unless the attach
>> >>> logic
>> >>>>> was
>> >>>>>>>> changed to recognize this new type of storage and create the
>> >>>>> appropriate
>> >>>>>>>> hypervisor data structure.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jburwell@basho.com
>> >
>> >>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Mike,
>> >>>>>>>>>
>> >>>>>>>>> It is generally odd to me that any operation in the Storage
>> layer
>> >>>>> would
>> >>>>>>>>> understand or care about details. I expect to see the Storage
>> >>>>> services
>> >>>>>>>>> expose a set of operations that can be composed/driven by the
>> >>>>> Hypervisor
>> >>>>>>>>> implementations to allocate space/create structures per their
>> >>> needs.
>> >>>>> If
>> >>>>>>>>> we
>> >>>>>>>>> don't invert this dependency, we are going to end with a massive
>> >>>>> n-to-n
>> >>>>>>>>> problem that will make the system increasingly difficult to
>> >>> maintain
>> >>>>> and
>> >>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire
>> code
>> >>> is
>> >>>>>>>>> located in the CitrixResourceBase class?
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> -John
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> To delve into this in a bit more detail:
>> >>>>>>>>>>
>> >>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer, the
>> >>> admin
>> >>>>>>>>> had
>> >>>>>>>>>> to first create a volume on the storage system, then go into
>> the
>> >>>>>>>>> hypervisor
>> >>>>>>>>>> to set up a data structure to make use of the volume (ex. a
>> >>> storage
>> >>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and data
>> >>> disks
>> >>>>>>>>> then
>> >>>>>>>>>> shared this storage system's volume.
>> >>>>>>>>>>
>> >>>>>>>>>> With Edison's new storage framework, storage need no longer be
>> so
>> >>>>>>>>> static
>> >>>>>>>>>> and you can easily create a 1:1 relationship between a storage
>> >>>>> system's
>> >>>>>>>>>> volume and the VM's data disk (necessary for storage Quality of
>> >>>>>>>>> Service).
>> >>>>>>>>>>
>> >>>>>>>>>> You can now write a plug-in that is called to dynamically
>> create
>> >>> and
>> >>>>>>>>> delete
>> >>>>>>>>>> volumes as needed.
>> >>>>>>>>>>
>> >>>>>>>>>> The problem that the storage framework did not address is in
>> >>> creating
>> >>>>>>>>> and
>> >>>>>>>>>> deleting the hypervisor-specific data structure when
>> performing an
>> >>>>>>>>>> attach/detach.
>> >>>>>>>>>>
>> >>>>>>>>>> That being the case, I've been enhancing it to do so. I've got
>> >>>>>>>>> XenServer
>> >>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
>> >>>>> submit
>> >>>>>>>>> this
>> >>>>>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>>>>
>> >>>>>>>>>> Does that help a bit? :)
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi John,
>> >>>>>>>>>>>
>> >>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>> >>>>>>>>>>>
>> >>>>>>>>>>> The issue is with the volume-attach logic (in the agent code).
>> >>> The
>> >>>>>>>>>> storage
>> >>>>>>>>>>> framework calls into the plug-in to have it create a volume as
>> >>>>>>>>> needed,
>> >>>>>>>>>> but
>> >>>>>>>>>>> when the time comes to attach the volume to a hypervisor, the
>> >>> attach
>> >>>>>>>>>> logic
>> >>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>> >>> zone-wide
>> >>>>>>>>>> storage
>> >>>>>>>>>>> (where the volume has just been created) and create, say, a
>> >>> storage
>> >>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to make
>> >>> use
>> >>>>>>>>> of the
>> >>>>>>>>>>> volume that was just created.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I've been spending most of my time recently making the attach
>> >>> logic
>> >>>>>>>>> work
>> >>>>>>>>>>> in the agent code.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Does that clear it up?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks!
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>> >>> jburwell@basho.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>> >>> specific?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>> -John
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors that
>> >>>>>>>>>> CloudStack
>> >>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM
>> in.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on what
>> >>> we do
>> >>>>>>>>>> with
>> >>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for VMware
>> in
>> >>> my
>> >>>>>>>>>>>> sandbox,
>> >>>>>>>>>>>>> as well.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>> >>> Offerings.
>> >>>>>>>>> I
>> >>>>>>>>>>>> plan to
>> >>>>>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>> >>>>>>>>>> kelcey@bbits.ca
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle post
>> >>> 4.2?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>> >>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue as I
>> >>> don't
>> >>>>>>>>>>>> support
>> >>>>>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>> >>> ustcweizhou@gmail.com>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> John,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> For the billing, as no one works on billing now, users
>> need
>> >>> to
>> >>>>>>>>>>>> calculate
>> >>>>>>>>>>>>>>> the billing by themselves. They can get the
>> service_offering
>> >>> and
>> >>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
>> course
>> >>>>>>>>> it is
>> >>>>>>>>>>>>>> better
>> >>>>>>>>>>>>>>> to tell user the exact limitation value of individual
>> volume,
>> >>>>>>>>> and
>> >>>>>>>>>>>> network
>> >>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it later.
>> Do
>> >>> you
>> >>>>>>>>>>>> think it
>> >>>>>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Agree with what you said about the two features.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
>> ustcweizhou@gmail.com
>> >>>>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
>> more.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics
>> rather
>> >>> than
>> >>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
>> >>>>>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write
>> for
>> >>> an
>> >>>>>>>>>>>>>> individual
>> >>>>>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>> >>> volumes
>> >>>>>>>>> with
>> >>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>> >>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users
>> >>> would
>> >>>>>>>>> like
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> option to employ similar cost models. Could an operator
>> >>>>>>>>> implement
>> >>>>>>>>>>>>>> such a
>> >>>>>>>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports
>> setting
>> >>>>>>>>> IOPS/BPS
>> >>>>>>>>>>>>>>>>> limitation for a running virtual machine through command
>> >>> line.
>> >>>>>>>>>>>>>> However,
>> >>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters of a
>> >>>>>>>>> created
>> >>>>>>>>>>>>>>> offering
>> >>>>>>>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
>> being
>> >>>>>>>>>> unclear.
>> >>>>>>>>>>>>>> Can
>> >>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>> >>>>>>>>> interfaces
>> >>>>>>>>>>>> that
>> >>>>>>>>>>>>>>>> describe the service guarantees provided by a resource?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
>> >>> Mike's
>> >>>>>>>>>> patch
>> >>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>> >>> limitation in
>> >>>>>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>> >>>>>>>>> regarding
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> touch points between these two features. I think that
>> >>>>>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think we
>> need
>> >>>>>>>>> closer
>> >>>>>>>>>>>>>>>> coordination between the branches (possibly have have
>> >>> solidfire
>> >>>>>>>>>> track
>> >>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be
>> captured
>> >>> in
>> >>>>>>>>> a
>> >>>>>>>>>>>>>> common
>> >>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
>> >>>>>>>>>> offerings).
>> >>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
>> reasoning
>> >>>>>>>>> about
>> >>>>>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>> >>>>>>>>> between
>> >>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
>> particular,
>> >>> a
>> >>>>>>>>>>>> scenario
>> >>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS
>> configured
>> >>> for
>> >>>>>>>>> a VM
>> >>>>>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>>>>> storage device has been configured to deliver. Another
>> >>>>>>>>> scenario
>> >>>>>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>> >>> device at
>> >>>>>>>>>>>>>> creation
>> >>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>> >>> introduces
>> >>>>>>>>>> logical
>> >>>>>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>> >>>>>>>>> number of
>> >>>>>>>>>>>>>> IOPS
>> >>>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
>> storage
>> >>>>>>>>> system
>> >>>>>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>> >>>>>>>>> jburwell@basho.com
>> >>>>>>>>>>>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged until
>> all
>> >>>>>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn
>> over to
>> >>>>>>>>> test.
>> >>>>>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is
>> addressed
>> >>> and
>> >>>>>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>> >>> SolidFire
>> >>>>>>>>>> patch
>> >>>>>>>>>>>> (
>> >>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I
>> understand
>> >>> it
>> >>>>>>>>> that
>> >>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to
>> ensure
>> >>> we
>> >>>>>>>>> don't
>> >>>>>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire
>> are
>> >>>>>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> >>> ustcweizhou@gmail.com
>> >>>>>>>>>>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
>> Monday,
>> >>> I
>> >>>>>>>>> will
>> >>>>>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
>> >>> that we
>> >>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
>> release
>> >>> a
>> >>>>>>>>>> version
>> >>>>>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
>> that's
>> >>>>>>>>> great!
>> >>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read
>> >>> and
>> >>>>>>>>> write
>> >>>>>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I don't
>> see
>> >>>>>>>>> how we
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
>> >>> would
>> >>>>>>>>> be
>> >>>>>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>> >>>>>>>>> compatibility.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so that
>> it
>> >>>>>>>>> will go
>> >>>>>>>>>>>> in
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
>> >>> release
>> >>>>>>>>> so
>> >>>>>>>>>>>> that
>> >>>>>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of Total
>> >>> IOps.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But
>> if
>> >>> the
>> >>>>>>>>>>>> window
>> >>>>>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
>> >>> that.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I also completely agree that there is no association
>> >>> between
>> >>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>> >>>>>>>>> wido@widodh.nl
>> >>>>>>>>>>>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided
>> to
>> >>>>>>>>> ignore
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> difference of read and write mainly because the
>> network
>> >>>>>>>>>> throttling
>> >>>>>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as
>> well.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>> >>> completely
>> >>>>>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations
>> then
>> >>>>>>>>> network
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It
>> could be
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>> >>>>>>>>> implement
>> >>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes
>> >>> into
>> >>>>>>>>> the
>> >>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with only
>> >>> total
>> >>>>>>>>>> IOps
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>> >>> master.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48
>> hours.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage
>> device
>> >>>>>>>>> (local
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of
>> device
>> >>>>>>>>> (such as
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the
>> disk
>> >>>>>>>>>>>> performance
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
>> >>> disk
>> >>>>>>>>> I/O
>> >>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
>> between
>> >>>>>>>>> Read
>> >>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate
>> >>> for
>> >>>>>>>>> Read
>> >>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect
>> both
>> >>> the
>> >>>>>>>>> Read
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
>> global
>> >>>>>>>>> value.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
>> >>> IOps,
>> >>>>>>>>> so it
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a
>> >>> lower
>> >>>>>>>>>> value
>> >>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it
>> >>> would
>> >>>>>>>>> be
>> >>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
>> support
>> >>>>>>>>> both,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
>> values
>> >>> for
>> >>>>>>>>>> read
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as the
>> >>> total
>> >>>>>>>>>> limit.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>> >>>>>>>>> global
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>
>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> >
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>
>> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>> https://cwiki.
>> >>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>
>> >>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>> >>> included
>> >>>>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>> >>>>>>>>> regressions?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge.
>> migrate,
>> >>>>>>>>> restore
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> git diff
>> c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>
>> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****
>> >
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>
>> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> >>> https://cwiki.
>> >>>>>>>>> **
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>
>> >>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>>>>
>> >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>>>>> <
>> >>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>>>>>
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >>> <
>> >>>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>> >
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>>>>
>> >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> <h**ttps://
>> >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>> >
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>>>>> cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>> *â„¢*
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the
>> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the
>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>> *â„¢*
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >> *â„¢*
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
What you say here may very well make sense, but I'm having a hard time
envisioning it.
Perhaps we should draw Edison in on this conversation as he was the initial
person to suggest the approach I took.
What do you think?
Thanks!
On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> It feels like we are combining two distinct concepts -- storage device
> management and storage protocols. In both cases, we are communicating with
> ISCSI, but one allows the system to create/delete volumes (Dynamic) on the
> device while the other requires the volume to be volume to be managed
> outside of the CloudStack context. To ensure that we are in sync on
> terminology, volume, in these definitions, refers to the physical
> allocation on the device, correct? Minimally, we must be able to
> communicate with a storage device to move bits from one place to another,
> read bits, delete bits, etc. Optionally, a storage device may be able to
> managed by CloudStack. Therefore, we can have a unmanaged iSCSI device
> onto which we store a Xen SR, and we can have a managed SolidFire iSCSI
> device on which CloudStack is capable of allocating LUNs and storing
> volumes. Finally, while CloudStack may be able to manage a device, an
> operator may chose to leave it unmanaged by CloudStack (e.g. the device is
> shared by many services, and the operator has chosen to dedicate only a
> portion of it to CloudStack). Does my reasoning make sense?
>
> Assuming my thoughts above are reasonable, it seems appropriate to strip
> the management concerns from StoragePoolType, add the notion of a storage
> device with an attached driver that indicates whether or not is managed by
> CloudStack, and establish a abstraction representing a physical allocation
> on a device separate that is associated with a volume. With these notions
> in place, hypervisor drivers can declare which protocols they support and
> when they encounter a device managed by CloudStack, utilize the management
> operations exposed by the driver to automate allocation. If these
> thoughts/concepts make sense, then we can sit down and drill down to a more
> detailed design.
>
> Thanks,
> -John
>
> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Here is the difference between the current iSCSI type and the Dynamic
> type:
> >
> > iSCSI type: The admin has to go in and create a Primary Storage based on
> > the iSCSI type. At this point in time, the iSCSI volume must exist on the
> > storage system (it is pre-allocated). Future CloudStack volumes are
> created
> > as VDIs on the SR that was created behind the scenes.
> >
> > Dynamic type: The admin has to go in and create Primary Storage based on
> a
> > plug-in that will create and delete volumes on its storage system
> > dynamically (as is enabled via the storage framework). When a user wants
> to
> > attach a CloudStack volume that was created, the framework tells the
> > plug-in to create a new volume. After this is done, the attach logic for
> > the hypervisor in question is called. No hypervisor data structure exists
> > at this point because the volume was just created. The hypervisor data
> > structure must be created.
> >
> >
> > On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >> wrote:
> >
> >> These are new terms, so I should probably have defined them up front for
> >> you. :)
> >>
> >> Static storage: Storage that is pre-allocated (ex. an admin creates a
> >> volume on a SAN), then a hypervisor data structure is created to consume
> >> the storage (ex. XenServer SR), then that hypervisor data structure is
> >> consumed by CloudStack. Disks (VDI) are later placed on this hypervisor
> >> data structure as needed. In these cases, the attach logic assumes the
> >> hypervisor data structure is already in place and simply attaches the
> VDI
> >> on the hypervisor data structure to the VM in question.
> >>
> >> Dynamic storage: Storage that is not pre-allocated. Instead of
> >> pre-existent storage, this could be a SAN (not a volume on a SAN, but
> the
> >> SAN itself). The hypervisor data structure must be created when an
> attach
> >> volume is performed because these types of volumes have not been
> pre-hooked
> >> up to such a hypervisor data structure by an admin. Once the attach
> logic
> >> creates, say, an SR on XenServer for this volume, it attaches the one
> and
> >> only VDI within the SR to the VM in question.
> >>
> >>
> >> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Mike,
> >>>
> >>> The current implementation of the Dynamic type attach behavior works in
> >>> terms of Xen ISCSI which why I ask about the difference. Another way
> to
> >>> ask the question -- what is the definition of a Dynamic storage pool
> type?
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >>> wrote:
> >>>
> >>>> As far as I know, the iSCSI type is uniquely used by XenServer when
> you
> >>>> want to set up Primary Storage that is directly based on an iSCSI
> >>> target.
> >>>> This allows you to skip the step of going to the hypervisor and
> >>> creating a
> >>>> storage repository based on that iSCSI target as CloudStack does that
> >>> part
> >>>> for you. I think this is only supported for XenServer. For all other
> >>>> hypervisors, you must first go to the hypervisor and perform this step
> >>>> manually.
> >>>>
> >>>> I don't really know what RBD is.
> >>>>
> >>>>
> >>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
> >>> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> Reading through the code, what is the difference between the ISCSI
> and
> >>>>> Dynamic types? Why isn't RBD considered Dynamic?
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>>> This new type of storage is defined in the Storage.StoragePoolType
> >>> class
> >>>>>> (called Dynamic):
> >>>>>>
> >>>>>> public static enum StoragePoolType {
> >>>>>>
> >>>>>> Filesystem(false), // local directory
> >>>>>>
> >>>>>> NetworkFilesystem(true), // NFS or CIFS
> >>>>>>
> >>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
> >>>>>>
> >>>>>> Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>
> >>>>>> ISO(false), // for iso image
> >>>>>>
> >>>>>> LVM(false), // XenServer local LVM SR
> >>>>>>
> >>>>>> CLVM(true),
> >>>>>>
> >>>>>> RBD(true),
> >>>>>>
> >>>>>> SharedMountPoint(true),
> >>>>>>
> >>>>>> VMFS(true), // VMware VMFS storage
> >>>>>>
> >>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
> >>>>>> customers.
> >>>>>>
> >>>>>> EXT(false), // XenServer local EXT SR
> >>>>>>
> >>>>>> OCFS2(true),
> >>>>>>
> >>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
> >>>>>>
> >>>>>>
> >>>>>> boolean shared;
> >>>>>>
> >>>>>>
> >>>>>> StoragePoolType(boolean shared) {
> >>>>>>
> >>>>>> this.shared = shared;
> >>>>>>
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>> public boolean isShared() {
> >>>>>>
> >>>>>> return shared;
> >>>>>>
> >>>>>> }
> >>>>>>
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> For example, let's say another storage company wants to implement a
> >>>>>>> plug-in to leverage its Quality of Service feature. It would be
> >>> dynamic,
> >>>>>>> zone-wide storage, as well. They would need only implement a
> storage
> >>>>>>> plug-in as I've made the necessary changes to the hypervisor-attach
> >>>>> logic
> >>>>>>> to support their plug-in.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>
> >>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It is
> >>> not.
> >>>>>>>>
> >>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
> >>> storage
> >>>>>>>> (and SolidFire is an implementation of this kind of storage). This
> >>>>> kind of
> >>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
> >>>>>>>>
> >>>>>>>> Edison created a new framework that supported the creation and
> >>> deletion
> >>>>>>>> of volumes dynamically. However, when I visited with him in
> Portland
> >>>>> back
> >>>>>>>> in April, we realized that it was not complete. We realized there
> >>> was
> >>>>>>>> nothing CloudStack could do with these volumes unless the attach
> >>> logic
> >>>>> was
> >>>>>>>> changed to recognize this new type of storage and create the
> >>>>> appropriate
> >>>>>>>> hypervisor data structure.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> It is generally odd to me that any operation in the Storage layer
> >>>>> would
> >>>>>>>>> understand or care about details. I expect to see the Storage
> >>>>> services
> >>>>>>>>> expose a set of operations that can be composed/driven by the
> >>>>> Hypervisor
> >>>>>>>>> implementations to allocate space/create structures per their
> >>> needs.
> >>>>> If
> >>>>>>>>> we
> >>>>>>>>> don't invert this dependency, we are going to end with a massive
> >>>>> n-to-n
> >>>>>>>>> problem that will make the system increasingly difficult to
> >>> maintain
> >>>>> and
> >>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire code
> >>> is
> >>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>
> >>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer, the
> >>> admin
> >>>>>>>>> had
> >>>>>>>>>> to first create a volume on the storage system, then go into the
> >>>>>>>>> hypervisor
> >>>>>>>>>> to set up a data structure to make use of the volume (ex. a
> >>> storage
> >>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and data
> >>> disks
> >>>>>>>>> then
> >>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>
> >>>>>>>>>> With Edison's new storage framework, storage need no longer be
> so
> >>>>>>>>> static
> >>>>>>>>>> and you can easily create a 1:1 relationship between a storage
> >>>>> system's
> >>>>>>>>>> volume and the VM's data disk (necessary for storage Quality of
> >>>>>>>>> Service).
> >>>>>>>>>>
> >>>>>>>>>> You can now write a plug-in that is called to dynamically create
> >>> and
> >>>>>>>>> delete
> >>>>>>>>>> volumes as needed.
> >>>>>>>>>>
> >>>>>>>>>> The problem that the storage framework did not address is in
> >>> creating
> >>>>>>>>> and
> >>>>>>>>>> deleting the hypervisor-specific data structure when performing
> an
> >>>>>>>>>> attach/detach.
> >>>>>>>>>>
> >>>>>>>>>> That being the case, I've been enhancing it to do so. I've got
> >>>>>>>>> XenServer
> >>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
> >>>>> submit
> >>>>>>>>> this
> >>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>
> >>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi John,
> >>>>>>>>>>>
> >>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
> >>>>>>>>>>>
> >>>>>>>>>>> The issue is with the volume-attach logic (in the agent code).
> >>> The
> >>>>>>>>>> storage
> >>>>>>>>>>> framework calls into the plug-in to have it create a volume as
> >>>>>>>>> needed,
> >>>>>>>>>> but
> >>>>>>>>>>> when the time comes to attach the volume to a hypervisor, the
> >>> attach
> >>>>>>>>>> logic
> >>>>>>>>>>> has to be smart enough to recognize it's being invoked on
> >>> zone-wide
> >>>>>>>>>> storage
> >>>>>>>>>>> (where the volume has just been created) and create, say, a
> >>> storage
> >>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to make
> >>> use
> >>>>>>>>> of the
> >>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>
> >>>>>>>>>>> I've been spending most of my time recently making the attach
> >>> logic
> >>>>>>>>> work
> >>>>>>>>>>> in the agent code.
> >>>>>>>>>>>
> >>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>> jburwell@basho.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Mike,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
> >>> specific?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> -John
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors that
> >>>>>>>>>> CloudStack
> >>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on what
> >>> we do
> >>>>>>>>>> with
> >>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for VMware
> in
> >>> my
> >>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
> >>> Offerings.
> >>>>>>>>> I
> >>>>>>>>>>>> plan to
> >>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
> >>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle post
> >>> 4.2?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue as I
> >>> don't
> >>>>>>>>>>>> support
> >>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>> ustcweizhou@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> For the billing, as no one works on billing now, users need
> >>> to
> >>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>> the billing by themselves. They can get the
> service_offering
> >>> and
> >>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of
> course
> >>>>>>>>> it is
> >>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>> to tell user the exact limitation value of individual
> volume,
> >>>>>>>>> and
> >>>>>>>>>>>> network
> >>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it later.
> Do
> >>> you
> >>>>>>>>>>>> think it
> >>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Agree with what you said about the two features.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> ustcweizhou@gmail.com
> >>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding
> more.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics rather
> >>> than
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
> >>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write for
> >>> an
> >>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
> >>> volumes
> >>>>>>>>> with
> >>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
> >>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users
> >>> would
> >>>>>>>>> like
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> option to employ similar cost models. Could an operator
> >>>>>>>>> implement
> >>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
> >>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>> limitation for a running virtual machine through command
> >>> line.
> >>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters of a
> >>>>>>>>> created
> >>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for
> being
> >>>>>>>>>> unclear.
> >>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
> >>>>>>>>> interfaces
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> describe the service guarantees provided by a resource?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
> >>> Mike's
> >>>>>>>>>> patch
> >>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
> >>> limitation in
> >>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
> >>>>>>>>> regarding
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> touch points between these two features. I think that
> >>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think we
> need
> >>>>>>>>> closer
> >>>>>>>>>>>>>>>> coordination between the branches (possibly have have
> >>> solidfire
> >>>>>>>>>> track
> >>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The things I want to understand are the following:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured
> >>> in
> >>>>>>>>> a
> >>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
> >>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for
> reasoning
> >>>>>>>>> about
> >>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
> >>>>>>>>> between
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In
> particular,
> >>> a
> >>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured
> >>> for
> >>>>>>>>> a VM
> >>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>> storage device has been configured to deliver. Another
> >>>>>>>>> scenario
> >>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
> >>> device at
> >>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>> time, but a later modification to storage device
> >>> introduces
> >>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
> >>>>>>>>> number of
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the
> storage
> >>>>>>>>> system
> >>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
> >>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged until
> all
> >>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn over
> to
> >>>>>>>>> test.
> >>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed
> >>> and
> >>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
> >>> SolidFire
> >>>>>>>>>> patch
> >>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand
> >>> it
> >>>>>>>>> that
> >>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure
> >>> we
> >>>>>>>>> don't
> >>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire
> are
> >>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>> ustcweizhou@gmail.com
> >>>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till
> Monday,
> >>> I
> >>>>>>>>> will
> >>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
> >>> that we
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never
> release
> >>> a
> >>>>>>>>>> version
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and
> that's
> >>>>>>>>> great!
> >>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read
> >>> and
> >>>>>>>>> write
> >>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I don't
> see
> >>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
> >>> would
> >>>>>>>>> be
> >>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
> >>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so that
> it
> >>>>>>>>> will go
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
> >>> release
> >>>>>>>>> so
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of Total
> >>> IOps.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But
> if
> >>> the
> >>>>>>>>>>>> window
> >>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
> >>> that.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I also completely agree that there is no association
> >>> between
> >>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
> >>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
> >>>>>>>>> ignore
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> difference of read and write mainly because the network
> >>>>>>>>>> throttling
> >>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as well.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
> >>> completely
> >>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
> >>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could
> be
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
> >>>>>>>>> implement
> >>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes
> >>> into
> >>>>>>>>> the
> >>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with only
> >>> total
> >>>>>>>>>> IOps
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
> >>> master.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
> >>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
> >>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
> >>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
> >>> disk
> >>>>>>>>> I/O
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference
> between
> >>>>>>>>> Read
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate
> >>> for
> >>>>>>>>> Read
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect both
> >>> the
> >>>>>>>>> Read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a
> global
> >>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
> >>> IOps,
> >>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a
> >>> lower
> >>>>>>>>>> value
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it
> >>> would
> >>>>>>>>> be
> >>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to
> support
> >>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different
> values
> >>> for
> >>>>>>>>>> read
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as the
> >>> total
> >>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
> >>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192
> >
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>
> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>> https://cwiki.
> >>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
> >>> included
> >>>>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
> >>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> >>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>
> >>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>> https://cwiki.
> >>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>
> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301
> >
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>> <
> >>>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>
> >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071
> >
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> <h**ttps://
> >>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
It feels like we are combining two distinct concepts -- storage device management and storage protocols. In both cases, we are communicating with ISCSI, but one allows the system to create/delete volumes (Dynamic) on the device while the other requires the volume to be volume to be managed outside of the CloudStack context. To ensure that we are in sync on terminology, volume, in these definitions, refers to the physical allocation on the device, correct? Minimally, we must be able to communicate with a storage device to move bits from one place to another, read bits, delete bits, etc. Optionally, a storage device may be able to managed by CloudStack. Therefore, we can have a unmanaged iSCSI device onto which we store a Xen SR, and we can have a managed SolidFire iSCSI device on which CloudStack is capable of allocating LUNs and storing volumes. Finally, while CloudStack may be able to manage a device, an operator may chose to leave it unmanaged by CloudStack (e.g. the device is shared by many services, and the operator has chosen to dedicate only a portion of it to CloudStack). Does my reasoning make sense?
Assuming my thoughts above are reasonable, it seems appropriate to strip the management concerns from StoragePoolType, add the notion of a storage device with an attached driver that indicates whether or not is managed by CloudStack, and establish a abstraction representing a physical allocation on a device separate that is associated with a volume. With these notions in place, hypervisor drivers can declare which protocols they support and when they encounter a device managed by CloudStack, utilize the management operations exposed by the driver to automate allocation. If these thoughts/concepts make sense, then we can sit down and drill down to a more detailed design.
Thanks,
-John
On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Here is the difference between the current iSCSI type and the Dynamic type:
>
> iSCSI type: The admin has to go in and create a Primary Storage based on
> the iSCSI type. At this point in time, the iSCSI volume must exist on the
> storage system (it is pre-allocated). Future CloudStack volumes are created
> as VDIs on the SR that was created behind the scenes.
>
> Dynamic type: The admin has to go in and create Primary Storage based on a
> plug-in that will create and delete volumes on its storage system
> dynamically (as is enabled via the storage framework). When a user wants to
> attach a CloudStack volume that was created, the framework tells the
> plug-in to create a new volume. After this is done, the attach logic for
> the hypervisor in question is called. No hypervisor data structure exists
> at this point because the volume was just created. The hypervisor data
> structure must be created.
>
>
> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
>> wrote:
>
>> These are new terms, so I should probably have defined them up front for
>> you. :)
>>
>> Static storage: Storage that is pre-allocated (ex. an admin creates a
>> volume on a SAN), then a hypervisor data structure is created to consume
>> the storage (ex. XenServer SR), then that hypervisor data structure is
>> consumed by CloudStack. Disks (VDI) are later placed on this hypervisor
>> data structure as needed. In these cases, the attach logic assumes the
>> hypervisor data structure is already in place and simply attaches the VDI
>> on the hypervisor data structure to the VM in question.
>>
>> Dynamic storage: Storage that is not pre-allocated. Instead of
>> pre-existent storage, this could be a SAN (not a volume on a SAN, but the
>> SAN itself). The hypervisor data structure must be created when an attach
>> volume is performed because these types of volumes have not been pre-hooked
>> up to such a hypervisor data structure by an admin. Once the attach logic
>> creates, say, an SR on XenServer for this volume, it attaches the one and
>> only VDI within the SR to the VM in question.
>>
>>
>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> The current implementation of the Dynamic type attach behavior works in
>>> terms of Xen ISCSI which why I ask about the difference. Another way to
>>> ask the question -- what is the definition of a Dynamic storage pool type?
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>>> As far as I know, the iSCSI type is uniquely used by XenServer when you
>>>> want to set up Primary Storage that is directly based on an iSCSI
>>> target.
>>>> This allows you to skip the step of going to the hypervisor and
>>> creating a
>>>> storage repository based on that iSCSI target as CloudStack does that
>>> part
>>>> for you. I think this is only supported for XenServer. For all other
>>>> hypervisors, you must first go to the hypervisor and perform this step
>>>> manually.
>>>>
>>>> I don't really know what RBD is.
>>>>
>>>>
>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
>>> wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> Reading through the code, what is the difference between the ISCSI and
>>>>> Dynamic types? Why isn't RBD considered Dynamic?
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>>
>>>>>> This new type of storage is defined in the Storage.StoragePoolType
>>> class
>>>>>> (called Dynamic):
>>>>>>
>>>>>> public static enum StoragePoolType {
>>>>>>
>>>>>> Filesystem(false), // local directory
>>>>>>
>>>>>> NetworkFilesystem(true), // NFS or CIFS
>>>>>>
>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>>>>
>>>>>> Iscsi(true), // for e.g., ZFS Comstar
>>>>>>
>>>>>> ISO(false), // for iso image
>>>>>>
>>>>>> LVM(false), // XenServer local LVM SR
>>>>>>
>>>>>> CLVM(true),
>>>>>>
>>>>>> RBD(true),
>>>>>>
>>>>>> SharedMountPoint(true),
>>>>>>
>>>>>> VMFS(true), // VMware VMFS storage
>>>>>>
>>>>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>>>>>> customers.
>>>>>>
>>>>>> EXT(false), // XenServer local EXT SR
>>>>>>
>>>>>> OCFS2(true),
>>>>>>
>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>>>>>>
>>>>>>
>>>>>> boolean shared;
>>>>>>
>>>>>>
>>>>>> StoragePoolType(boolean shared) {
>>>>>>
>>>>>> this.shared = shared;
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> public boolean isShared() {
>>>>>>
>>>>>> return shared;
>>>>>>
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com
>>>>>>> wrote:
>>>>>>
>>>>>>> For example, let's say another storage company wants to implement a
>>>>>>> plug-in to leverage its Quality of Service feature. It would be
>>> dynamic,
>>>>>>> zone-wide storage, as well. They would need only implement a storage
>>>>>>> plug-in as I've made the necessary changes to the hypervisor-attach
>>>>> logic
>>>>>>> to support their plug-in.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It is
>>> not.
>>>>>>>>
>>>>>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>>> storage
>>>>>>>> (and SolidFire is an implementation of this kind of storage). This
>>>>> kind of
>>>>>>>> storage is new to 4.2 with Edison's storage framework changes.
>>>>>>>>
>>>>>>>> Edison created a new framework that supported the creation and
>>> deletion
>>>>>>>> of volumes dynamically. However, when I visited with him in Portland
>>>>> back
>>>>>>>> in April, we realized that it was not complete. We realized there
>>> was
>>>>>>>> nothing CloudStack could do with these volumes unless the attach
>>> logic
>>>>> was
>>>>>>>> changed to recognize this new type of storage and create the
>>>>> appropriate
>>>>>>>> hypervisor data structure.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> It is generally odd to me that any operation in the Storage layer
>>>>> would
>>>>>>>>> understand or care about details. I expect to see the Storage
>>>>> services
>>>>>>>>> expose a set of operations that can be composed/driven by the
>>>>> Hypervisor
>>>>>>>>> implementations to allocate space/create structures per their
>>> needs.
>>>>> If
>>>>>>>>> we
>>>>>>>>> don't invert this dependency, we are going to end with a massive
>>>>> n-to-n
>>>>>>>>> problem that will make the system increasingly difficult to
>>> maintain
>>>>> and
>>>>>>>>> enhance. Am I understanding that the Xen specific SolidFire code
>>> is
>>>>>>>>> located in the CitrixResourceBase class?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> To delve into this in a bit more detail:
>>>>>>>>>>
>>>>>>>>>> Prior to 4.2 and aside from one setup method for XenServer, the
>>> admin
>>>>>>>>> had
>>>>>>>>>> to first create a volume on the storage system, then go into the
>>>>>>>>> hypervisor
>>>>>>>>>> to set up a data structure to make use of the volume (ex. a
>>> storage
>>>>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and data
>>> disks
>>>>>>>>> then
>>>>>>>>>> shared this storage system's volume.
>>>>>>>>>>
>>>>>>>>>> With Edison's new storage framework, storage need no longer be so
>>>>>>>>> static
>>>>>>>>>> and you can easily create a 1:1 relationship between a storage
>>>>> system's
>>>>>>>>>> volume and the VM's data disk (necessary for storage Quality of
>>>>>>>>> Service).
>>>>>>>>>>
>>>>>>>>>> You can now write a plug-in that is called to dynamically create
>>> and
>>>>>>>>> delete
>>>>>>>>>> volumes as needed.
>>>>>>>>>>
>>>>>>>>>> The problem that the storage framework did not address is in
>>> creating
>>>>>>>>> and
>>>>>>>>>> deleting the hypervisor-specific data structure when performing an
>>>>>>>>>> attach/detach.
>>>>>>>>>>
>>>>>>>>>> That being the case, I've been enhancing it to do so. I've got
>>>>>>>>> XenServer
>>>>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
>>>>> submit
>>>>>>>>> this
>>>>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>>>>
>>>>>>>>>> Does that help a bit? :)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi John,
>>>>>>>>>>>
>>>>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>>>>>>>>>
>>>>>>>>>>> The issue is with the volume-attach logic (in the agent code).
>>> The
>>>>>>>>>> storage
>>>>>>>>>>> framework calls into the plug-in to have it create a volume as
>>>>>>>>> needed,
>>>>>>>>>> but
>>>>>>>>>>> when the time comes to attach the volume to a hypervisor, the
>>> attach
>>>>>>>>>> logic
>>>>>>>>>>> has to be smart enough to recognize it's being invoked on
>>> zone-wide
>>>>>>>>>> storage
>>>>>>>>>>> (where the volume has just been created) and create, say, a
>>> storage
>>>>>>>>>>> repository (for XenServer) or a datastore (for VMware) to make
>>> use
>>>>>>>>> of the
>>>>>>>>>>> volume that was just created.
>>>>>>>>>>>
>>>>>>>>>>> I've been spending most of my time recently making the attach
>>> logic
>>>>>>>>> work
>>>>>>>>>>> in the agent code.
>>>>>>>>>>>
>>>>>>>>>>> Does that clear it up?
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>>> jburwell@basho.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mike,
>>>>>>>>>>>>
>>>>>>>>>>>> Can you explain why the the storage driver is hypervisor
>>> specific?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -John
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, ultimately I would like to support all hypervisors that
>>>>>>>>>> CloudStack
>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Right now this plug-in supports XenServer. Depending on what
>>> we do
>>>>>>>>>> with
>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working for VMware in
>>> my
>>>>>>>>>>>> sandbox,
>>>>>>>>>>>>> as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>>> Offerings.
>>>>>>>>> I
>>>>>>>>>>>> plan to
>>>>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>>>>>>>>> kelcey@bbits.ca
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle post
>>> 4.2?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If his feature is for KVM only, then it is a non issue as I
>>> don't
>>>>>>>>>>>> support
>>>>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>>> ustcweizhou@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> For the billing, as no one works on billing now, users need
>>> to
>>>>>>>>>>>> calculate
>>>>>>>>>>>>>>> the billing by themselves. They can get the service_offering
>>> and
>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of course
>>>>>>>>> it is
>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>> to tell user the exact limitation value of individual volume,
>>>>>>>>> and
>>>>>>>>>>>> network
>>>>>>>>>>>>>>> rate limitation for nics as well. I can work on it later. Do
>>> you
>>>>>>>>>>>> think it
>>>>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <ustcweizhou@gmail.com
>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding more.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics rather
>>> than
>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
>>>>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write for
>>> an
>>>>>>>>>>>>>> individual
>>>>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>>> volumes
>>>>>>>>> with
>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users
>>> would
>>>>>>>>> like
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> option to employ similar cost models. Could an operator
>>>>>>>>> implement
>>>>>>>>>>>>>> such a
>>>>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
>>>>>>>>> IOPS/BPS
>>>>>>>>>>>>>>>>> limitation for a running virtual machine through command
>>> line.
>>>>>>>>>>>>>> However,
>>>>>>>>>>>>>>>>> CloudStack does not support changing the parameters of a
>>>>>>>>> created
>>>>>>>>>>>>>>> offering
>>>>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I meant at the Java interface level. I apologize for being
>>>>>>>>>> unclear.
>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>>>>>>>>> interfaces
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> describe the service guarantees provided by a resource?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
>>> Mike's
>>>>>>>>>> patch
>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>>> limitation in
>>>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>>>>>>>>> regarding
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> touch points between these two features. I think that
>>>>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but I think we need
>>>>>>>>> closer
>>>>>>>>>>>>>>>> coordination between the branches (possibly have have
>>> solidfire
>>>>>>>>>> track
>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The things I want to understand are the following:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured
>>> in
>>>>>>>>> a
>>>>>>>>>>>>>> common
>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
>>>>>>>>>> offerings).
>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model for reasoning
>>>>>>>>> about
>>>>>>>>>>>>>> IOP
>>>>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>>>>>>>>> between
>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In particular,
>>> a
>>>>>>>>>>>> scenario
>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured
>>> for
>>>>>>>>> a VM
>>>>>>>>>>>>>>> than a
>>>>>>>>>>>>>>>>>> storage device has been configured to deliver. Another
>>>>>>>>> scenario
>>>>>>>>>>>>>>> could a
>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>>> device at
>>>>>>>>>>>>>> creation
>>>>>>>>>>>>>>>>>> time, but a later modification to storage device
>>> introduces
>>>>>>>>>> logical
>>>>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>>>>>>>>> number of
>>>>>>>>>>>>>> IOPS
>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the storage
>>>>>>>>> system
>>>>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>>>>>>> jburwell@basho.com
>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> My opinion is that no features should be merged until all
>>>>>>>>>>>>>> functional
>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn over to
>>>>>>>>> test.
>>>>>>>>>>>>>>> Until
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed
>>> and
>>>>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>>> SolidFire
>>>>>>>>>> patch
>>>>>>>>>>>> (
>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand
>>> it
>>>>>>>>> that
>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure
>>> we
>>>>>>>>> don't
>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>>>>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>>> ustcweizhou@gmail.com
>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If anyone object thems for technical reason till Monday,
>>> I
>>>>>>>>> will
>>>>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
>>> that we
>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never release
>>> a
>>>>>>>>>> version
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and that's
>>>>>>>>> great!
>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read
>>> and
>>>>>>>>> write
>>>>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later because I don't see
>>>>>>>>> how we
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
>>> would
>>>>>>>>> be
>>>>>>>>>>>> stuck
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>>>>>>> compatibility.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so that it
>>>>>>>>> will go
>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
>>> release
>>>>>>>>> so
>>>>>>>>>>>> that
>>>>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> will be released with Read and Write instead of Total
>>> IOps.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But if
>>> the
>>>>>>>>>>>> window
>>>>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
>>> that.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I also completely agree that there is no association
>>> between
>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>>>>>>> wido@widodh.nl
>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
>>>>>>>>> ignore
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> difference of read and write mainly because the network
>>>>>>>>>> throttling
>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> care the difference of sent and received bytes as well.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>>> completely
>>>>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
>>>>>>>>> network
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could be
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>>>>>>>>> implement
>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes
>>> into
>>>>>>>>> the
>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> window we have to make sure we don't release with only
>>> total
>>>>>>>>>> IOps
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>>> master.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
>>>>>>>>> (local
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>>>>>>>>> (such as
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>>>>>>>> performance
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
>>> disk
>>>>>>>>> I/O
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no difference between
>>>>>>>>> Read
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate
>>> for
>>>>>>>>> Read
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect both
>>> the
>>>>>>>>> Read
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a global
>>>>>>>>> value.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
>>> IOps,
>>>>>>>>> so it
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a
>>> lower
>>>>>>>>>> value
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it
>>> would
>>>>>>>>> be
>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in place to support
>>>>>>>>> both,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different values
>>> for
>>>>>>>>>> read
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> write you can always sum both up and set that as the
>>> total
>>>>>>>>>> limit.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>>>>>>>>> global
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>
>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> https://cwiki.
>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>>> included
>>>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>>>>>>> regressions?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>
>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>>> https://cwiki.
>>>>>>>>> **
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>>>>> <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <ht**tps://
>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>>> <
>>>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>>>>>
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <h**ttps://
>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Here is the difference between the current iSCSI type and the Dynamic type:
iSCSI type: The admin has to go in and create a Primary Storage based on
the iSCSI type. At this point in time, the iSCSI volume must exist on the
storage system (it is pre-allocated). Future CloudStack volumes are created
as VDIs on the SR that was created behind the scenes.
Dynamic type: The admin has to go in and create Primary Storage based on a
plug-in that will create and delete volumes on its storage system
dynamically (as is enabled via the storage framework). When a user wants to
attach a CloudStack volume that was created, the framework tells the
plug-in to create a new volume. After this is done, the attach logic for
the hypervisor in question is called. No hypervisor data structure exists
at this point because the volume was just created. The hypervisor data
structure must be created.
On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> These are new terms, so I should probably have defined them up front for
> you. :)
>
> Static storage: Storage that is pre-allocated (ex. an admin creates a
> volume on a SAN), then a hypervisor data structure is created to consume
> the storage (ex. XenServer SR), then that hypervisor data structure is
> consumed by CloudStack. Disks (VDI) are later placed on this hypervisor
> data structure as needed. In these cases, the attach logic assumes the
> hypervisor data structure is already in place and simply attaches the VDI
> on the hypervisor data structure to the VM in question.
>
> Dynamic storage: Storage that is not pre-allocated. Instead of
> pre-existent storage, this could be a SAN (not a volume on a SAN, but the
> SAN itself). The hypervisor data structure must be created when an attach
> volume is performed because these types of volumes have not been pre-hooked
> up to such a hypervisor data structure by an admin. Once the attach logic
> creates, say, an SR on XenServer for this volume, it attaches the one and
> only VDI within the SR to the VM in question.
>
>
> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> The current implementation of the Dynamic type attach behavior works in
>> terms of Xen ISCSI which why I ask about the difference. Another way to
>> ask the question -- what is the definition of a Dynamic storage pool type?
>>
>> Thanks,
>> -John
>>
>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > As far as I know, the iSCSI type is uniquely used by XenServer when you
>> > want to set up Primary Storage that is directly based on an iSCSI
>> target.
>> > This allows you to skip the step of going to the hypervisor and
>> creating a
>> > storage repository based on that iSCSI target as CloudStack does that
>> part
>> > for you. I think this is only supported for XenServer. For all other
>> > hypervisors, you must first go to the hypervisor and perform this step
>> > manually.
>> >
>> > I don't really know what RBD is.
>> >
>> >
>> > On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >
>> >> Mike,
>> >>
>> >> Reading through the code, what is the difference between the ISCSI and
>> >> Dynamic types? Why isn't RBD considered Dynamic?
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >> wrote:
>> >>
>> >>> This new type of storage is defined in the Storage.StoragePoolType
>> class
>> >>> (called Dynamic):
>> >>>
>> >>> public static enum StoragePoolType {
>> >>>
>> >>> Filesystem(false), // local directory
>> >>>
>> >>> NetworkFilesystem(true), // NFS or CIFS
>> >>>
>> >>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>> >>>
>> >>> Iscsi(true), // for e.g., ZFS Comstar
>> >>>
>> >>> ISO(false), // for iso image
>> >>>
>> >>> LVM(false), // XenServer local LVM SR
>> >>>
>> >>> CLVM(true),
>> >>>
>> >>> RBD(true),
>> >>>
>> >>> SharedMountPoint(true),
>> >>>
>> >>> VMFS(true), // VMware VMFS storage
>> >>>
>> >>> PreSetup(true), // for XenServer, Storage Pool is set up by
>> >>> customers.
>> >>>
>> >>> EXT(false), // XenServer local EXT SR
>> >>>
>> >>> OCFS2(true),
>> >>>
>> >>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>> >>>
>> >>>
>> >>> boolean shared;
>> >>>
>> >>>
>> >>> StoragePoolType(boolean shared) {
>> >>>
>> >>> this.shared = shared;
>> >>>
>> >>> }
>> >>>
>> >>>
>> >>> public boolean isShared() {
>> >>>
>> >>> return shared;
>> >>>
>> >>> }
>> >>>
>> >>> }
>> >>>
>> >>>
>> >>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com
>> >>>> wrote:
>> >>>
>> >>>> For example, let's say another storage company wants to implement a
>> >>>> plug-in to leverage its Quality of Service feature. It would be
>> dynamic,
>> >>>> zone-wide storage, as well. They would need only implement a storage
>> >>>> plug-in as I've made the necessary changes to the hypervisor-attach
>> >> logic
>> >>>> to support their plug-in.
>> >>>>
>> >>>>
>> >>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>
>> >>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It is
>> not.
>> >>>>>
>> >>>>> The XenServer attach logic is now aware of dynamic, zone-wide
>> storage
>> >>>>> (and SolidFire is an implementation of this kind of storage). This
>> >> kind of
>> >>>>> storage is new to 4.2 with Edison's storage framework changes.
>> >>>>>
>> >>>>> Edison created a new framework that supported the creation and
>> deletion
>> >>>>> of volumes dynamically. However, when I visited with him in Portland
>> >> back
>> >>>>> in April, we realized that it was not complete. We realized there
>> was
>> >>>>> nothing CloudStack could do with these volumes unless the attach
>> logic
>> >> was
>> >>>>> changed to recognize this new type of storage and create the
>> >> appropriate
>> >>>>> hypervisor data structure.
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com>
>> >> wrote:
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> It is generally odd to me that any operation in the Storage layer
>> >> would
>> >>>>>> understand or care about details. I expect to see the Storage
>> >> services
>> >>>>>> expose a set of operations that can be composed/driven by the
>> >> Hypervisor
>> >>>>>> implementations to allocate space/create structures per their
>> needs.
>> >> If
>> >>>>>> we
>> >>>>>> don't invert this dependency, we are going to end with a massive
>> >> n-to-n
>> >>>>>> problem that will make the system increasingly difficult to
>> maintain
>> >> and
>> >>>>>> enhance. Am I understanding that the Xen specific SolidFire code
>> is
>> >>>>>> located in the CitrixResourceBase class?
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>>>> mike.tutkowski@solidfire.com
>> >>>>>>> wrote:
>> >>>>>>
>> >>>>>>> To delve into this in a bit more detail:
>> >>>>>>>
>> >>>>>>> Prior to 4.2 and aside from one setup method for XenServer, the
>> admin
>> >>>>>> had
>> >>>>>>> to first create a volume on the storage system, then go into the
>> >>>>>> hypervisor
>> >>>>>>> to set up a data structure to make use of the volume (ex. a
>> storage
>> >>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and data
>> disks
>> >>>>>> then
>> >>>>>>> shared this storage system's volume.
>> >>>>>>>
>> >>>>>>> With Edison's new storage framework, storage need no longer be so
>> >>>>>> static
>> >>>>>>> and you can easily create a 1:1 relationship between a storage
>> >> system's
>> >>>>>>> volume and the VM's data disk (necessary for storage Quality of
>> >>>>>> Service).
>> >>>>>>>
>> >>>>>>> You can now write a plug-in that is called to dynamically create
>> and
>> >>>>>> delete
>> >>>>>>> volumes as needed.
>> >>>>>>>
>> >>>>>>> The problem that the storage framework did not address is in
>> creating
>> >>>>>> and
>> >>>>>>> deleting the hypervisor-specific data structure when performing an
>> >>>>>>> attach/detach.
>> >>>>>>>
>> >>>>>>> That being the case, I've been enhancing it to do so. I've got
>> >>>>>> XenServer
>> >>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
>> >> submit
>> >>>>>> this
>> >>>>>>> if we extend the 4.2 freeze date.
>> >>>>>>>
>> >>>>>>> Does that help a bit? :)
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com
>> >>>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Hi John,
>> >>>>>>>>
>> >>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>> >>>>>>>>
>> >>>>>>>> The issue is with the volume-attach logic (in the agent code).
>> The
>> >>>>>>> storage
>> >>>>>>>> framework calls into the plug-in to have it create a volume as
>> >>>>>> needed,
>> >>>>>>> but
>> >>>>>>>> when the time comes to attach the volume to a hypervisor, the
>> attach
>> >>>>>>> logic
>> >>>>>>>> has to be smart enough to recognize it's being invoked on
>> zone-wide
>> >>>>>>> storage
>> >>>>>>>> (where the volume has just been created) and create, say, a
>> storage
>> >>>>>>>> repository (for XenServer) or a datastore (for VMware) to make
>> use
>> >>>>>> of the
>> >>>>>>>> volume that was just created.
>> >>>>>>>>
>> >>>>>>>> I've been spending most of my time recently making the attach
>> logic
>> >>>>>> work
>> >>>>>>>> in the agent code.
>> >>>>>>>>
>> >>>>>>>> Does that clear it up?
>> >>>>>>>>
>> >>>>>>>> Thanks!
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
>> jburwell@basho.com>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Mike,
>> >>>>>>>>>
>> >>>>>>>>> Can you explain why the the storage driver is hypervisor
>> specific?
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> -John
>> >>>>>>>>>
>> >>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Yes, ultimately I would like to support all hypervisors that
>> >>>>>>> CloudStack
>> >>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
>> >>>>>>>>>>
>> >>>>>>>>>> Right now this plug-in supports XenServer. Depending on what
>> we do
>> >>>>>>> with
>> >>>>>>>>>> regards to 4.2 feature freeze, I have it working for VMware in
>> my
>> >>>>>>>>> sandbox,
>> >>>>>>>>>> as well.
>> >>>>>>>>>>
>> >>>>>>>>>> Also, just to be clear, this is all in regards to Disk
>> Offerings.
>> >>>>>> I
>> >>>>>>>>> plan to
>> >>>>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>> >>>>>>> kelcey@bbits.ca
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle post
>> 4.2?
>> >>>>>>>>>>>
>> >>>>>>>>>>> ----- Original Message -----
>> >>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>> >>>>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >>>>>>>>>>>
>> >>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>> >>>>>>>>>>>
>> >>>>>>>>>>> If his feature is for KVM only, then it is a non issue as I
>> don't
>> >>>>>>>>> support
>> >>>>>>>>>>> KVM in 4.2.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
>> ustcweizhou@gmail.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> John,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> For the billing, as no one works on billing now, users need
>> to
>> >>>>>>>>> calculate
>> >>>>>>>>>>>> the billing by themselves. They can get the service_offering
>> and
>> >>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of course
>> >>>>>> it is
>> >>>>>>>>>>> better
>> >>>>>>>>>>>> to tell user the exact limitation value of individual volume,
>> >>>>>> and
>> >>>>>>>>> network
>> >>>>>>>>>>>> rate limitation for nics as well. I can work on it later. Do
>> you
>> >>>>>>>>> think it
>> >>>>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Sorry my misunstand the second the question.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Agree with what you said about the two features.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <ustcweizhou@gmail.com
>> >
>> >>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding more.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (1) I think billing should depend on IO statistics rather
>> than
>> >>>>>>> IOPS
>> >>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
>> >>>>>>>>>>> disk_io_stat
>> >>>>>>>>>>>>> can
>> >>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write for
>> an
>> >>>>>>>>>>> individual
>> >>>>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Going by the AWS model, customers are billed more for
>> volumes
>> >>>>>> with
>> >>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>> >>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users
>> would
>> >>>>>> like
>> >>>>>>>>> the
>> >>>>>>>>>>>>> option to employ similar cost models. Could an operator
>> >>>>>> implement
>> >>>>>>>>>>> such a
>> >>>>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
>> >>>>>> IOPS/BPS
>> >>>>>>>>>>>>>> limitation for a running virtual machine through command
>> line.
>> >>>>>>>>>>> However,
>> >>>>>>>>>>>>>> CloudStack does not support changing the parameters of a
>> >>>>>> created
>> >>>>>>>>>>>> offering
>> >>>>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I meant at the Java interface level. I apologize for being
>> >>>>>>> unclear.
>> >>>>>>>>>>> Can
>> >>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>> >>>>>> interfaces
>> >>>>>>>>> that
>> >>>>>>>>>>>>> describe the service guarantees provided by a resource?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
>> Mike's
>> >>>>>>> patch
>> >>>>>>>>>>>>> after
>> >>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the
>> limitation in
>> >>>>>>>>>>>> hypervisor
>> >>>>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>> >>>>>> regarding
>> >>>>>>>>> the
>> >>>>>>>>>>>>> touch points between these two features. I think that
>> >>>>>>>>>>> disk_io_throttling
>> >>>>>>>>>>>>> will need to be merged before SolidFire, but I think we need
>> >>>>>> closer
>> >>>>>>>>>>>>> coordination between the branches (possibly have have
>> solidfire
>> >>>>>>> track
>> >>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> The things I want to understand are the following:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured
>> in
>> >>>>>> a
>> >>>>>>>>>>> common
>> >>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
>> >>>>>>> offerings).
>> >>>>>>>>>>>>>>> 2) Should there be a common interface model for reasoning
>> >>>>>> about
>> >>>>>>>>>>> IOP
>> >>>>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>> >>>>>> between
>> >>>>>>> a
>> >>>>>>>>>>>>>>> hypervisor and storage device reconciled? In particular,
>> a
>> >>>>>>>>> scenario
>> >>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured
>> for
>> >>>>>> a VM
>> >>>>>>>>>>>> than a
>> >>>>>>>>>>>>>>> storage device has been configured to deliver. Another
>> >>>>>> scenario
>> >>>>>>>>>>>> could a
>> >>>>>>>>>>>>>>> consistent configuration between a VM and a storage
>> device at
>> >>>>>>>>>>> creation
>> >>>>>>>>>>>>>>> time, but a later modification to storage device
>> introduces
>> >>>>>>> logical
>> >>>>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>> >>>>>> number of
>> >>>>>>>>>>> IOPS
>> >>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the storage
>> >>>>>> system
>> >>>>>>>>>>>> side.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>> >>>>>> jburwell@basho.com
>> >>>>>>>>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> My opinion is that no features should be merged until all
>> >>>>>>>>>>> functional
>> >>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn over to
>> >>>>>> test.
>> >>>>>>>>>>>> Until
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed
>> and
>> >>>>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>>>> by
>> >>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
>> SolidFire
>> >>>>>>> patch
>> >>>>>>>>> (
>> >>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand
>> it
>> >>>>>> that
>> >>>>>>>>>>> work
>> >>>>>>>>>>>> is
>> >>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure
>> we
>> >>>>>> don't
>> >>>>>>>>>>>> have a
>> >>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>> >>>>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> ustcweizhou@gmail.com
>> >>>>>>>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> If anyone object thems for technical reason till Monday,
>> I
>> >>>>>> will
>> >>>>>>>>>>>> revert
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again
>> that we
>> >>>>>>>>> should
>> >>>>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never release
>> a
>> >>>>>>> version
>> >>>>>>>>>>>> with
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and that's
>> >>>>>> great!
>> >>>>>>>>> We
>> >>>>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read
>> and
>> >>>>>> write
>> >>>>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> don't see how it could be fixed later because I don't see
>> >>>>>> how we
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we
>> would
>> >>>>>> be
>> >>>>>>>>> stuck
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>> >>>>>> compatibility.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> What Wei meant was merging it into master now so that it
>> >>>>>> will go
>> >>>>>>>>> in
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
>> release
>> >>>>>> so
>> >>>>>>>>> that
>> >>>>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> will be released with Read and Write instead of Total
>> IOps.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But if
>> the
>> >>>>>>>>> window
>> >>>>>>>>>>>>> moves
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do
>> that.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I also completely agree that there is no association
>> between
>> >>>>>>>>>>> network
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>> >>>>>> wido@widodh.nl
>> >>>>>>>>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
>> >>>>>> ignore
>> >>>>>>> the
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> difference of read and write mainly because the network
>> >>>>>>> throttling
>> >>>>>>>>>>>> did
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> care the difference of sent and received bytes as well.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>> completely
>> >>>>>>>>>>>> different.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
>> >>>>>> network
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could be
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>> >>>>>> implement
>> >>>>>>> it
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes
>> into
>> >>>>>> the
>> >>>>>>> 4.2
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> window we have to make sure we don't release with only
>> total
>> >>>>>>> IOps
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>> master.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
>> >>>>>> (local
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>> >>>>>> (such as
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>> >>>>>>>>> performance
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> other VMs running on the same storage device.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the
>> disk
>> >>>>>> I/O
>> >>>>>>> of
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Looking at the code I see you make no difference between
>> >>>>>> Read
>> >>>>>>> and
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate
>> for
>> >>>>>> Read
>> >>>>>>>>> and
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> It's also strange, in the polling side you collect both
>> the
>> >>>>>> Read
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a global
>> >>>>>> value.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read
>> IOps,
>> >>>>>> so it
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a
>> lower
>> >>>>>>> value
>> >>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it
>> would
>> >>>>>> be
>> >>>>>>> of
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> value to at least have the mechanism in place to support
>> >>>>>> both,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different values
>> for
>> >>>>>>> read
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> write you can always sum both up and set that as the
>> total
>> >>>>>>> limit.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>> >>>>>> global
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>>>> issues.apache.org/****
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>>>>>>>>>>>
>> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>
>> >>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>> <
>> >>>>>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> https://cwiki.
>> >>>>>> **
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
>> included
>> >>>>>> in
>> >>>>>>> the
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> * What testing has been done to check for potential
>> >>>>>> regressions?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>> >>>>>> restore
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>>>>>>>>>>>
>> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>
>> >>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>>>> <
>> >>>>>>>>>>>>>>>
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
>> https://cwiki.
>> >>>>>> **
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>>>> <
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>>>
>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>>>> <
>> >>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> >
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <ht**tps://
>> >>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> >
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
>> <
>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>>>>
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> <h**ttps://
>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>> *â„¢*
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the cloud<
>> >>>>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *â„¢*
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the cloud<
>> >> http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the cloud<
>> >> http://solidfire.com/solution/overview/?video=play>
>> >>>> *â„¢*
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the
>> >>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>
>> >>
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
These are new terms, so I should probably have defined them up front for
you. :)
Static storage: Storage that is pre-allocated (ex. an admin creates a
volume on a SAN), then a hypervisor data structure is created to consume
the storage (ex. XenServer SR), then that hypervisor data structure is
consumed by CloudStack. Disks (VDI) are later placed on this hypervisor
data structure as needed. In these cases, the attach logic assumes the
hypervisor data structure is already in place and simply attaches the VDI
on the hypervisor data structure to the VM in question.
Dynamic storage: Storage that is not pre-allocated. Instead of pre-existent
storage, this could be a SAN (not a volume on a SAN, but the SAN itself).
The hypervisor data structure must be created when an attach volume is
performed because these types of volumes have not been pre-hooked up to
such a hypervisor data structure by an admin. Once the attach logic
creates, say, an SR on XenServer for this volume, it attaches the one and
only VDI within the SR to the VM in question.
On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> The current implementation of the Dynamic type attach behavior works in
> terms of Xen ISCSI which why I ask about the difference. Another way to
> ask the question -- what is the definition of a Dynamic storage pool type?
>
> Thanks,
> -John
>
> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > As far as I know, the iSCSI type is uniquely used by XenServer when you
> > want to set up Primary Storage that is directly based on an iSCSI target.
> > This allows you to skip the step of going to the hypervisor and creating
> a
> > storage repository based on that iSCSI target as CloudStack does that
> part
> > for you. I think this is only supported for XenServer. For all other
> > hypervisors, you must first go to the hypervisor and perform this step
> > manually.
> >
> > I don't really know what RBD is.
> >
> >
> > On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com> wrote:
> >
> >> Mike,
> >>
> >> Reading through the code, what is the difference between the ISCSI and
> >> Dynamic types? Why isn't RBD considered Dynamic?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> This new type of storage is defined in the Storage.StoragePoolType
> class
> >>> (called Dynamic):
> >>>
> >>> public static enum StoragePoolType {
> >>>
> >>> Filesystem(false), // local directory
> >>>
> >>> NetworkFilesystem(true), // NFS or CIFS
> >>>
> >>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
> >>>
> >>> Iscsi(true), // for e.g., ZFS Comstar
> >>>
> >>> ISO(false), // for iso image
> >>>
> >>> LVM(false), // XenServer local LVM SR
> >>>
> >>> CLVM(true),
> >>>
> >>> RBD(true),
> >>>
> >>> SharedMountPoint(true),
> >>>
> >>> VMFS(true), // VMware VMFS storage
> >>>
> >>> PreSetup(true), // for XenServer, Storage Pool is set up by
> >>> customers.
> >>>
> >>> EXT(false), // XenServer local EXT SR
> >>>
> >>> OCFS2(true),
> >>>
> >>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
> >>>
> >>>
> >>> boolean shared;
> >>>
> >>>
> >>> StoragePoolType(boolean shared) {
> >>>
> >>> this.shared = shared;
> >>>
> >>> }
> >>>
> >>>
> >>> public boolean isShared() {
> >>>
> >>> return shared;
> >>>
> >>> }
> >>>
> >>> }
> >>>
> >>>
> >>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com
> >>>> wrote:
> >>>
> >>>> For example, let's say another storage company wants to implement a
> >>>> plug-in to leverage its Quality of Service feature. It would be
> dynamic,
> >>>> zone-wide storage, as well. They would need only implement a storage
> >>>> plug-in as I've made the necessary changes to the hypervisor-attach
> >> logic
> >>>> to support their plug-in.
> >>>>
> >>>>
> >>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com> wrote:
> >>>>
> >>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It is
> not.
> >>>>>
> >>>>> The XenServer attach logic is now aware of dynamic, zone-wide storage
> >>>>> (and SolidFire is an implementation of this kind of storage). This
> >> kind of
> >>>>> storage is new to 4.2 with Edison's storage framework changes.
> >>>>>
> >>>>> Edison created a new framework that supported the creation and
> deletion
> >>>>> of volumes dynamically. However, when I visited with him in Portland
> >> back
> >>>>> in April, we realized that it was not complete. We realized there was
> >>>>> nothing CloudStack could do with these volumes unless the attach
> logic
> >> was
> >>>>> changed to recognize this new type of storage and create the
> >> appropriate
> >>>>> hypervisor data structure.
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com>
> >> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> It is generally odd to me that any operation in the Storage layer
> >> would
> >>>>>> understand or care about details. I expect to see the Storage
> >> services
> >>>>>> expose a set of operations that can be composed/driven by the
> >> Hypervisor
> >>>>>> implementations to allocate space/create structures per their needs.
> >> If
> >>>>>> we
> >>>>>> don't invert this dependency, we are going to end with a massive
> >> n-to-n
> >>>>>> problem that will make the system increasingly difficult to maintain
> >> and
> >>>>>> enhance. Am I understanding that the Xen specific SolidFire code is
> >>>>>> located in the CitrixResourceBase class?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> To delve into this in a bit more detail:
> >>>>>>>
> >>>>>>> Prior to 4.2 and aside from one setup method for XenServer, the
> admin
> >>>>>> had
> >>>>>>> to first create a volume on the storage system, then go into the
> >>>>>> hypervisor
> >>>>>>> to set up a data structure to make use of the volume (ex. a storage
> >>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and data
> disks
> >>>>>> then
> >>>>>>> shared this storage system's volume.
> >>>>>>>
> >>>>>>> With Edison's new storage framework, storage need no longer be so
> >>>>>> static
> >>>>>>> and you can easily create a 1:1 relationship between a storage
> >> system's
> >>>>>>> volume and the VM's data disk (necessary for storage Quality of
> >>>>>> Service).
> >>>>>>>
> >>>>>>> You can now write a plug-in that is called to dynamically create
> and
> >>>>>> delete
> >>>>>>> volumes as needed.
> >>>>>>>
> >>>>>>> The problem that the storage framework did not address is in
> creating
> >>>>>> and
> >>>>>>> deleting the hypervisor-specific data structure when performing an
> >>>>>>> attach/detach.
> >>>>>>>
> >>>>>>> That being the case, I've been enhancing it to do so. I've got
> >>>>>> XenServer
> >>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
> >> submit
> >>>>>> this
> >>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>
> >>>>>>> Does that help a bit? :)
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi John,
> >>>>>>>>
> >>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
> >>>>>>>>
> >>>>>>>> The issue is with the volume-attach logic (in the agent code). The
> >>>>>>> storage
> >>>>>>>> framework calls into the plug-in to have it create a volume as
> >>>>>> needed,
> >>>>>>> but
> >>>>>>>> when the time comes to attach the volume to a hypervisor, the
> attach
> >>>>>>> logic
> >>>>>>>> has to be smart enough to recognize it's being invoked on
> zone-wide
> >>>>>>> storage
> >>>>>>>> (where the volume has just been created) and create, say, a
> storage
> >>>>>>>> repository (for XenServer) or a datastore (for VMware) to make use
> >>>>>> of the
> >>>>>>>> volume that was just created.
> >>>>>>>>
> >>>>>>>> I've been spending most of my time recently making the attach
> logic
> >>>>>> work
> >>>>>>>> in the agent code.
> >>>>>>>>
> >>>>>>>> Does that clear it up?
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jburwell@basho.com
> >
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Mike,
> >>>>>>>>>
> >>>>>>>>> Can you explain why the the storage driver is hypervisor
> specific?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Yes, ultimately I would like to support all hypervisors that
> >>>>>>> CloudStack
> >>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
> >>>>>>>>>>
> >>>>>>>>>> Right now this plug-in supports XenServer. Depending on what we
> do
> >>>>>>> with
> >>>>>>>>>> regards to 4.2 feature freeze, I have it working for VMware in
> my
> >>>>>>>>> sandbox,
> >>>>>>>>>> as well.
> >>>>>>>>>>
> >>>>>>>>>> Also, just to be clear, this is all in regards to Disk
> Offerings.
> >>>>>> I
> >>>>>>>>> plan to
> >>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
> >>>>>>> kelcey@bbits.ca
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle post
> 4.2?
> >>>>>>>>>>>
> >>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> I agree on merging Wei's feature first, then mine.
> >>>>>>>>>>>
> >>>>>>>>>>> If his feature is for KVM only, then it is a non issue as I
> don't
> >>>>>>>>> support
> >>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> ustcweizhou@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> John,
> >>>>>>>>>>>>
> >>>>>>>>>>>> For the billing, as no one works on billing now, users need to
> >>>>>>>>> calculate
> >>>>>>>>>>>> the billing by themselves. They can get the service_offering
> and
> >>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of course
> >>>>>> it is
> >>>>>>>>>>> better
> >>>>>>>>>>>> to tell user the exact limitation value of individual volume,
> >>>>>> and
> >>>>>>>>> network
> >>>>>>>>>>>> rate limitation for nics as well. I can work on it later. Do
> you
> >>>>>>>>> think it
> >>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Agree with what you said about the two features.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Wei
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding more.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (1) I think billing should depend on IO statistics rather
> than
> >>>>>>> IOPS
> >>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
> >>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write for an
> >>>>>>>>>>> individual
> >>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Going by the AWS model, customers are billed more for volumes
> >>>>>> with
> >>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
> >>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users
> would
> >>>>>> like
> >>>>>>>>> the
> >>>>>>>>>>>>> option to employ similar cost models. Could an operator
> >>>>>> implement
> >>>>>>>>>>> such a
> >>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
> >>>>>> IOPS/BPS
> >>>>>>>>>>>>>> limitation for a running virtual machine through command
> line.
> >>>>>>>>>>> However,
> >>>>>>>>>>>>>> CloudStack does not support changing the parameters of a
> >>>>>> created
> >>>>>>>>>>>> offering
> >>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I meant at the Java interface level. I apologize for being
> >>>>>>> unclear.
> >>>>>>>>>>> Can
> >>>>>>>>>>>>> we more generalize allocation algorithms with a set of
> >>>>>> interfaces
> >>>>>>>>> that
> >>>>>>>>>>>>> describe the service guarantees provided by a resource?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
> Mike's
> >>>>>>> patch
> >>>>>>>>>>>>> after
> >>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the limitation
> in
> >>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
> >>>>>> regarding
> >>>>>>>>> the
> >>>>>>>>>>>>> touch points between these two features. I think that
> >>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>> will need to be merged before SolidFire, but I think we need
> >>>>>> closer
> >>>>>>>>>>>>> coordination between the branches (possibly have have
> solidfire
> >>>>>>> track
> >>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The things I want to understand are the following:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured in
> >>>>>> a
> >>>>>>>>>>> common
> >>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
> >>>>>>> offerings).
> >>>>>>>>>>>>>>> 2) Should there be a common interface model for reasoning
> >>>>>> about
> >>>>>>>>>>> IOP
> >>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
> >>>>>> between
> >>>>>>> a
> >>>>>>>>>>>>>>> hypervisor and storage device reconciled? In particular, a
> >>>>>>>>> scenario
> >>>>>>>>>>>>> where
> >>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured
> for
> >>>>>> a VM
> >>>>>>>>>>>> than a
> >>>>>>>>>>>>>>> storage device has been configured to deliver. Another
> >>>>>> scenario
> >>>>>>>>>>>> could a
> >>>>>>>>>>>>>>> consistent configuration between a VM and a storage device
> at
> >>>>>>>>>>> creation
> >>>>>>>>>>>>>>> time, but a later modification to storage device introduces
> >>>>>>> logical
> >>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
> >>>>>> number of
> >>>>>>>>>>> IOPS
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the storage
> >>>>>> system
> >>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
> >>>>>> jburwell@basho.com
> >>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> My opinion is that no features should be merged until all
> >>>>>>>>>>> functional
> >>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn over to
> >>>>>> test.
> >>>>>>>>>>>> Until
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed
> and
> >>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the
> SolidFire
> >>>>>>> patch
> >>>>>>>>> (
> >>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
> >>>>>> that
> >>>>>>>>>>> work
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure we
> >>>>>> don't
> >>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
> >>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> ustcweizhou@gmail.com
> >>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> If anyone object thems for technical reason till Monday, I
> >>>>>> will
> >>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again that
> we
> >>>>>>>>> should
> >>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never release a
> >>>>>>> version
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and that's
> >>>>>> great!
> >>>>>>>>> We
> >>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read and
> >>>>>> write
> >>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> don't see how it could be fixed later because I don't see
> >>>>>> how we
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we would
> >>>>>> be
> >>>>>>>>> stuck
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> total unless/until we decided to break backwards
> >>>>>> compatibility.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What Wei meant was merging it into master now so that it
> >>>>>> will go
> >>>>>>>>> in
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2
> release
> >>>>>> so
> >>>>>>>>> that
> >>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> will be released with Read and Write instead of Total
> IOps.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But if
> the
> >>>>>>>>> window
> >>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do that.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I also completely agree that there is no association
> between
> >>>>>>>>>>> network
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
> >>>>>> wido@widodh.nl
> >>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
> >>>>>> ignore
> >>>>>>> the
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> difference of read and write mainly because the network
> >>>>>>> throttling
> >>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> care the difference of sent and received bytes as well.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
> completely
> >>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
> >>>>>> network
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could be
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
> >>>>>> implement
> >>>>>>> it
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes into
> >>>>>> the
> >>>>>>> 4.2
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> window we have to make sure we don't release with only
> total
> >>>>>>> IOps
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
> master.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
> >>>>>> (local
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
> >>>>>> (such as
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
> >>>>>>>>> performance
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
> >>>>>> I/O
> >>>>>>> of
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Looking at the code I see you make no difference between
> >>>>>> Read
> >>>>>>> and
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate for
> >>>>>> Read
> >>>>>>>>> and
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> It's also strange, in the polling side you collect both
> the
> >>>>>> Read
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a global
> >>>>>> value.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read IOps,
> >>>>>> so it
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a
> lower
> >>>>>>> value
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it
> would
> >>>>>> be
> >>>>>>> of
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> value to at least have the mechanism in place to support
> >>>>>> both,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different values
> for
> >>>>>>> read
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> write you can always sum both up and set that as the total
> >>>>>>> limit.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
> >>>>>> global
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <ht**tps://
> >>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> <
> >>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>> <
> >>>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> https://cwiki.
> >>>>>> **
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> * What automated testing (unit and integration) is
> included
> >>>>>> in
> >>>>>>> the
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> * What testing has been done to check for potential
> >>>>>> regressions?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> >>>>>> restore
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> <
> >>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>
> >>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>> <
> >>>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> https://cwiki.
> >>>>>> **
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>> <
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>
> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>> <
> >>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <ht**tps://
> >>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <ht**tps://
> >>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <ht**tps://
> >>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> <
> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> <h**ttps://
> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *â„¢*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *â„¢*
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *â„¢*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
The current implementation of the Dynamic type attach behavior works in terms of Xen ISCSI which why I ask about the difference. Another way to ask the question -- what is the definition of a Dynamic storage pool type?
Thanks,
-John
On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> As far as I know, the iSCSI type is uniquely used by XenServer when you
> want to set up Primary Storage that is directly based on an iSCSI target.
> This allows you to skip the step of going to the hypervisor and creating a
> storage repository based on that iSCSI target as CloudStack does that part
> for you. I think this is only supported for XenServer. For all other
> hypervisors, you must first go to the hypervisor and perform this step
> manually.
>
> I don't really know what RBD is.
>
>
> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Reading through the code, what is the difference between the ISCSI and
>> Dynamic types? Why isn't RBD considered Dynamic?
>>
>> Thanks,
>> -John
>>
>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>>> This new type of storage is defined in the Storage.StoragePoolType class
>>> (called Dynamic):
>>>
>>> public static enum StoragePoolType {
>>>
>>> Filesystem(false), // local directory
>>>
>>> NetworkFilesystem(true), // NFS or CIFS
>>>
>>> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>>>
>>> Iscsi(true), // for e.g., ZFS Comstar
>>>
>>> ISO(false), // for iso image
>>>
>>> LVM(false), // XenServer local LVM SR
>>>
>>> CLVM(true),
>>>
>>> RBD(true),
>>>
>>> SharedMountPoint(true),
>>>
>>> VMFS(true), // VMware VMFS storage
>>>
>>> PreSetup(true), // for XenServer, Storage Pool is set up by
>>> customers.
>>>
>>> EXT(false), // XenServer local EXT SR
>>>
>>> OCFS2(true),
>>>
>>> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>>>
>>>
>>> boolean shared;
>>>
>>>
>>> StoragePoolType(boolean shared) {
>>>
>>> this.shared = shared;
>>>
>>> }
>>>
>>>
>>> public boolean isShared() {
>>>
>>> return shared;
>>>
>>> }
>>>
>>> }
>>>
>>>
>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>>>> wrote:
>>>
>>>> For example, let's say another storage company wants to implement a
>>>> plug-in to leverage its Quality of Service feature. It would be dynamic,
>>>> zone-wide storage, as well. They would need only implement a storage
>>>> plug-in as I've made the necessary changes to the hypervisor-attach
>> logic
>>>> to support their plug-in.
>>>>
>>>>
>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Oh, sorry to imply the XenServer code is SolidFire specific. It is not.
>>>>>
>>>>> The XenServer attach logic is now aware of dynamic, zone-wide storage
>>>>> (and SolidFire is an implementation of this kind of storage). This
>> kind of
>>>>> storage is new to 4.2 with Edison's storage framework changes.
>>>>>
>>>>> Edison created a new framework that supported the creation and deletion
>>>>> of volumes dynamically. However, when I visited with him in Portland
>> back
>>>>> in April, we realized that it was not complete. We realized there was
>>>>> nothing CloudStack could do with these volumes unless the attach logic
>> was
>>>>> changed to recognize this new type of storage and create the
>> appropriate
>>>>> hypervisor data structure.
>>>>>
>>>>>
>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> It is generally odd to me that any operation in the Storage layer
>> would
>>>>>> understand or care about details. I expect to see the Storage
>> services
>>>>>> expose a set of operations that can be composed/driven by the
>> Hypervisor
>>>>>> implementations to allocate space/create structures per their needs.
>> If
>>>>>> we
>>>>>> don't invert this dependency, we are going to end with a massive
>> n-to-n
>>>>>> problem that will make the system increasingly difficult to maintain
>> and
>>>>>> enhance. Am I understanding that the Xen specific SolidFire code is
>>>>>> located in the CitrixResourceBase class?
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com
>>>>>>> wrote:
>>>>>>
>>>>>>> To delve into this in a bit more detail:
>>>>>>>
>>>>>>> Prior to 4.2 and aside from one setup method for XenServer, the admin
>>>>>> had
>>>>>>> to first create a volume on the storage system, then go into the
>>>>>> hypervisor
>>>>>>> to set up a data structure to make use of the volume (ex. a storage
>>>>>>> repository on XenServer or a datastore on ESX(i)). VMs and data disks
>>>>>> then
>>>>>>> shared this storage system's volume.
>>>>>>>
>>>>>>> With Edison's new storage framework, storage need no longer be so
>>>>>> static
>>>>>>> and you can easily create a 1:1 relationship between a storage
>> system's
>>>>>>> volume and the VM's data disk (necessary for storage Quality of
>>>>>> Service).
>>>>>>>
>>>>>>> You can now write a plug-in that is called to dynamically create and
>>>>>> delete
>>>>>>> volumes as needed.
>>>>>>>
>>>>>>> The problem that the storage framework did not address is in creating
>>>>>> and
>>>>>>> deleting the hypervisor-specific data structure when performing an
>>>>>>> attach/detach.
>>>>>>>
>>>>>>> That being the case, I've been enhancing it to do so. I've got
>>>>>> XenServer
>>>>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
>> submit
>>>>>> this
>>>>>>> if we extend the 4.2 freeze date.
>>>>>>>
>>>>>>> Does that help a bit? :)
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com
>>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi John,
>>>>>>>>
>>>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>>>>>>
>>>>>>>> The issue is with the volume-attach logic (in the agent code). The
>>>>>>> storage
>>>>>>>> framework calls into the plug-in to have it create a volume as
>>>>>> needed,
>>>>>>> but
>>>>>>>> when the time comes to attach the volume to a hypervisor, the attach
>>>>>>> logic
>>>>>>>> has to be smart enough to recognize it's being invoked on zone-wide
>>>>>>> storage
>>>>>>>> (where the volume has just been created) and create, say, a storage
>>>>>>>> repository (for XenServer) or a datastore (for VMware) to make use
>>>>>> of the
>>>>>>>> volume that was just created.
>>>>>>>>
>>>>>>>> I've been spending most of my time recently making the attach logic
>>>>>> work
>>>>>>>> in the agent code.
>>>>>>>>
>>>>>>>> Does that clear it up?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Mike,
>>>>>>>>>
>>>>>>>>> Can you explain why the the storage driver is hypervisor specific?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Yes, ultimately I would like to support all hypervisors that
>>>>>>> CloudStack
>>>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
>>>>>>>>>>
>>>>>>>>>> Right now this plug-in supports XenServer. Depending on what we do
>>>>>>> with
>>>>>>>>>> regards to 4.2 feature freeze, I have it working for VMware in my
>>>>>>>>> sandbox,
>>>>>>>>>> as well.
>>>>>>>>>>
>>>>>>>>>> Also, just to be clear, this is all in regards to Disk Offerings.
>>>>>> I
>>>>>>>>> plan to
>>>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>>>>>> kelcey@bbits.ca
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Is there any plan on supporting KVM in the patch cycle post 4.2?
>>>>>>>>>>>
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>>>
>>>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>>>
>>>>>>>>>>> If his feature is for KVM only, then it is a non issue as I don't
>>>>>>>>> support
>>>>>>>>>>> KVM in 4.2.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> John,
>>>>>>>>>>>>
>>>>>>>>>>>> For the billing, as no one works on billing now, users need to
>>>>>>>>> calculate
>>>>>>>>>>>> the billing by themselves. They can get the service_offering and
>>>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of course
>>>>>> it is
>>>>>>>>>>> better
>>>>>>>>>>>> to tell user the exact limitation value of individual volume,
>>>>>> and
>>>>>>>>> network
>>>>>>>>>>>> rate limitation for nics as well. I can work on it later. Do you
>>>>>>>>> think it
>>>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>>>
>>>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>>>
>>>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>>>
>>>>>>>>>>>> -Wei
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>
>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding more.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (1) I think billing should depend on IO statistics rather than
>>>>>>> IOPS
>>>>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
>>>>>>>>>>> disk_io_stat
>>>>>>>>>>>>> can
>>>>>>>>>>>>>> get the IO statistics including bytes/iops read/write for an
>>>>>>>>>>> individual
>>>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Going by the AWS model, customers are billed more for volumes
>>>>>> with
>>>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users would
>>>>>> like
>>>>>>>>> the
>>>>>>>>>>>>> option to employ similar cost models. Could an operator
>>>>>> implement
>>>>>>>>>>> such a
>>>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
>>>>>> IOPS/BPS
>>>>>>>>>>>>>> limitation for a running virtual machine through command line.
>>>>>>>>>>> However,
>>>>>>>>>>>>>> CloudStack does not support changing the parameters of a
>>>>>> created
>>>>>>>>>>>> offering
>>>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>>>
>>>>>>>>>>>>> I meant at the Java interface level. I apologize for being
>>>>>>> unclear.
>>>>>>>>>>> Can
>>>>>>>>>>>>> we more generalize allocation algorithms with a set of
>>>>>> interfaces
>>>>>>>>> that
>>>>>>>>>>>>> describe the service guarantees provided by a resource?
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit Mike's
>>>>>>> patch
>>>>>>>>>>>>> after
>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the limitation in
>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>>> type, I think.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>>>>>> regarding
>>>>>>>>> the
>>>>>>>>>>>>> touch points between these two features. I think that
>>>>>>>>>>> disk_io_throttling
>>>>>>>>>>>>> will need to be merged before SolidFire, but I think we need
>>>>>> closer
>>>>>>>>>>>>> coordination between the branches (possibly have have solidfire
>>>>>>> track
>>>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Wei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The things I want to understand are the following:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured in
>>>>>> a
>>>>>>>>>>> common
>>>>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
>>>>>>> offerings).
>>>>>>>>>>>>>>> 2) Should there be a common interface model for reasoning
>>>>>> about
>>>>>>>>>>> IOP
>>>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>>>>>> between
>>>>>>> a
>>>>>>>>>>>>>>> hypervisor and storage device reconciled? In particular, a
>>>>>>>>> scenario
>>>>>>>>>>>>> where
>>>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured for
>>>>>> a VM
>>>>>>>>>>>> than a
>>>>>>>>>>>>>>> storage device has been configured to deliver. Another
>>>>>> scenario
>>>>>>>>>>>> could a
>>>>>>>>>>>>>>> consistent configuration between a VM and a storage device at
>>>>>>>>>>> creation
>>>>>>>>>>>>>>> time, but a later modification to storage device introduces
>>>>>>> logical
>>>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>>>>>> number of
>>>>>>>>>>> IOPS
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> My feature is focused on controlling IOPS from the storage
>>>>>> system
>>>>>>>>>>>> side.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>>>> jburwell@basho.com
>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My opinion is that no features should be merged until all
>>>>>>>>>>> functional
>>>>>>>>>>>>>>>> issues have been resolved and it is ready to turn over to
>>>>>> test.
>>>>>>>>>>>> Until
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed and
>>>>>>>>>>>> re-reviewed
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, how does this work intersect/compliment the SolidFire
>>>>>>> patch
>>>>>>>>> (
>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
>>>>>> that
>>>>>>>>>>> work
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure we
>>>>>> don't
>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>>>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <ustcweizhou@gmail.com
>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If anyone object thems for technical reason till Monday, I
>>>>>> will
>>>>>>>>>>>> revert
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> For the sake of clarity I just want to mention again that we
>>>>>>>>> should
>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never release a
>>>>>>> version
>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and that's
>>>>>> great!
>>>>>>>>> We
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read and
>>>>>> write
>>>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> don't see how it could be fixed later because I don't see
>>>>>> how we
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we would
>>>>>> be
>>>>>>>>> stuck
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>>>> compatibility.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What Wei meant was merging it into master now so that it
>>>>>> will go
>>>>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release
>>>>>> so
>>>>>>>>> that
>>>>>>>>>>>> 4.2
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> will be released with Read and Write instead of Total IOps.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But if the
>>>>>>>>> window
>>>>>>>>>>>>> moves
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I also completely agree that there is no association between
>>>>>>>>>>> network
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>>>> wido@widodh.nl
>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
>>>>>> ignore
>>>>>>> the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> difference of read and write mainly because the network
>>>>>>> throttling
>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> care the difference of sent and received bytes as well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O completely
>>>>>>>>>>>> different.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
>>>>>> network
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could be
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>>>>>> implement
>>>>>>> it
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes into
>>>>>> the
>>>>>>> 4.2
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> window we have to make sure we don't release with only total
>>>>>>> IOps
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Virtual machines are running on the same storage device
>>>>>> (local
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>>>>>> (such as
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>>>>> performance
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
>>>>>> I/O
>>>>>>> of
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Looking at the code I see you make no difference between
>>>>>> Read
>>>>>>> and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate for
>>>>>> Read
>>>>>>>>> and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It's also strange, in the polling side you collect both the
>>>>>> Read
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a global
>>>>>> value.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read IOps,
>>>>>> so it
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> like a valid use-case where that an admin would set a lower
>>>>>>> value
>>>>>>>>>>> for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Since this only supports KVM at this point I think it would
>>>>>> be
>>>>>>> of
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> value to at least have the mechanism in place to support
>>>>>> both,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting different values for
>>>>>>> read
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> write you can always sum both up and set that as the total
>>>>>>> limit.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>>>>>> global
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>>>> issues.apache.org/****
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <ht**tps://
>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>> <
>>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>> <
>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>>>>>> **
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is included
>>>>>> in
>>>>>>> the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>>>> regressions?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>>>>>> restore
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>>>> <
>>>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>>>> <
>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>>>>>> **
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>>>> <
>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <ht**tps://
>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <ht**tps://
>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <ht**tps://
>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *â„¢*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *â„¢*
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Alternatively, you can use the PreSetup type for XenServer. In this case,
you must go to XenServer and set up the storage repository (which can be
based on an iSCSI target). Then you must go into CloudStack and select the
PreSetup type for Primary Storage. This is like selecting the vmfs type for
VMware.
On Mon, Jun 3, 2013 at 3:10 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> As far as I know, the iSCSI type is uniquely used by XenServer when you
> want to set up Primary Storage that is directly based on an iSCSI target.
> This allows you to skip the step of going to the hypervisor and creating a
> storage repository based on that iSCSI target as CloudStack does that part
> for you. I think this is only supported for XenServer. For all other
> hypervisors, you must first go to the hypervisor and perform this step
> manually.
>
> I don't really know what RBD is.
>
>
> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Reading through the code, what is the difference between the ISCSI and
>> Dynamic types? Why isn't RBD considered Dynamic?
>>
>> Thanks,
>> -John
>>
>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > This new type of storage is defined in the Storage.StoragePoolType class
>> > (called Dynamic):
>> >
>> > public static enum StoragePoolType {
>> >
>> > Filesystem(false), // local directory
>> >
>> > NetworkFilesystem(true), // NFS or CIFS
>> >
>> > IscsiLUN(true), // shared LUN, with a clusterfs overlay
>> >
>> > Iscsi(true), // for e.g., ZFS Comstar
>> >
>> > ISO(false), // for iso image
>> >
>> > LVM(false), // XenServer local LVM SR
>> >
>> > CLVM(true),
>> >
>> > RBD(true),
>> >
>> > SharedMountPoint(true),
>> >
>> > VMFS(true), // VMware VMFS storage
>> >
>> > PreSetup(true), // for XenServer, Storage Pool is set up by
>> > customers.
>> >
>> > EXT(false), // XenServer local EXT SR
>> >
>> > OCFS2(true),
>> >
>> > Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>> >
>> >
>> > boolean shared;
>> >
>> >
>> > StoragePoolType(boolean shared) {
>> >
>> > this.shared = shared;
>> >
>> > }
>> >
>> >
>> > public boolean isShared() {
>> >
>> > return shared;
>> >
>> > }
>> >
>> > }
>> >
>> >
>> > On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>> >> wrote:
>> >
>> >> For example, let's say another storage company wants to implement a
>> >> plug-in to leverage its Quality of Service feature. It would be
>> dynamic,
>> >> zone-wide storage, as well. They would need only implement a storage
>> >> plug-in as I've made the necessary changes to the hypervisor-attach
>> logic
>> >> to support their plug-in.
>> >>
>> >>
>> >> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> >> mike.tutkowski@solidfire.com> wrote:
>> >>
>> >>> Oh, sorry to imply the XenServer code is SolidFire specific. It is
>> not.
>> >>>
>> >>> The XenServer attach logic is now aware of dynamic, zone-wide storage
>> >>> (and SolidFire is an implementation of this kind of storage). This
>> kind of
>> >>> storage is new to 4.2 with Edison's storage framework changes.
>> >>>
>> >>> Edison created a new framework that supported the creation and
>> deletion
>> >>> of volumes dynamically. However, when I visited with him in Portland
>> back
>> >>> in April, we realized that it was not complete. We realized there was
>> >>> nothing CloudStack could do with these volumes unless the attach
>> logic was
>> >>> changed to recognize this new type of storage and create the
>> appropriate
>> >>> hypervisor data structure.
>> >>>
>> >>>
>> >>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com>
>> wrote:
>> >>>
>> >>>> Mike,
>> >>>>
>> >>>> It is generally odd to me that any operation in the Storage layer
>> would
>> >>>> understand or care about details. I expect to see the Storage
>> services
>> >>>> expose a set of operations that can be composed/driven by the
>> Hypervisor
>> >>>> implementations to allocate space/create structures per their needs.
>> If
>> >>>> we
>> >>>> don't invert this dependency, we are going to end with a massive
>> n-to-n
>> >>>> problem that will make the system increasingly difficult to maintain
>> and
>> >>>> enhance. Am I understanding that the Xen specific SolidFire code is
>> >>>> located in the CitrixResourceBase class?
>> >>>>
>> >>>> Thanks,
>> >>>> -John
>> >>>>
>> >>>>
>> >>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com
>> >>>>> wrote:
>> >>>>
>> >>>>> To delve into this in a bit more detail:
>> >>>>>
>> >>>>> Prior to 4.2 and aside from one setup method for XenServer, the
>> admin
>> >>>> had
>> >>>>> to first create a volume on the storage system, then go into the
>> >>>> hypervisor
>> >>>>> to set up a data structure to make use of the volume (ex. a storage
>> >>>>> repository on XenServer or a datastore on ESX(i)). VMs and data
>> disks
>> >>>> then
>> >>>>> shared this storage system's volume.
>> >>>>>
>> >>>>> With Edison's new storage framework, storage need no longer be so
>> >>>> static
>> >>>>> and you can easily create a 1:1 relationship between a storage
>> system's
>> >>>>> volume and the VM's data disk (necessary for storage Quality of
>> >>>> Service).
>> >>>>>
>> >>>>> You can now write a plug-in that is called to dynamically create and
>> >>>> delete
>> >>>>> volumes as needed.
>> >>>>>
>> >>>>> The problem that the storage framework did not address is in
>> creating
>> >>>> and
>> >>>>> deleting the hypervisor-specific data structure when performing an
>> >>>>> attach/detach.
>> >>>>>
>> >>>>> That being the case, I've been enhancing it to do so. I've got
>> >>>> XenServer
>> >>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
>> submit
>> >>>> this
>> >>>>> if we extend the 4.2 freeze date.
>> >>>>>
>> >>>>> Does that help a bit? :)
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com
>> >>>>>> wrote:
>> >>>>>
>> >>>>>> Hi John,
>> >>>>>>
>> >>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>> >>>>>>
>> >>>>>> The issue is with the volume-attach logic (in the agent code). The
>> >>>>> storage
>> >>>>>> framework calls into the plug-in to have it create a volume as
>> >>>> needed,
>> >>>>> but
>> >>>>>> when the time comes to attach the volume to a hypervisor, the
>> attach
>> >>>>> logic
>> >>>>>> has to be smart enough to recognize it's being invoked on zone-wide
>> >>>>> storage
>> >>>>>> (where the volume has just been created) and create, say, a storage
>> >>>>>> repository (for XenServer) or a datastore (for VMware) to make use
>> >>>> of the
>> >>>>>> volume that was just created.
>> >>>>>>
>> >>>>>> I've been spending most of my time recently making the attach logic
>> >>>> work
>> >>>>>> in the agent code.
>> >>>>>>
>> >>>>>> Does that clear it up?
>> >>>>>>
>> >>>>>> Thanks!
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>> Mike,
>> >>>>>>>
>> >>>>>>> Can you explain why the the storage driver is hypervisor specific?
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> -John
>> >>>>>>>
>> >>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Yes, ultimately I would like to support all hypervisors that
>> >>>>> CloudStack
>> >>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
>> >>>>>>>>
>> >>>>>>>> Right now this plug-in supports XenServer. Depending on what we
>> do
>> >>>>> with
>> >>>>>>>> regards to 4.2 feature freeze, I have it working for VMware in my
>> >>>>>>> sandbox,
>> >>>>>>>> as well.
>> >>>>>>>>
>> >>>>>>>> Also, just to be clear, this is all in regards to Disk Offerings.
>> >>>> I
>> >>>>>>> plan to
>> >>>>>>>> support Compute Offerings post 4.2.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>> >>>>> kelcey@bbits.ca
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Is there any plan on supporting KVM in the patch cycle post 4.2?
>> >>>>>>>>>
>> >>>>>>>>> ----- Original Message -----
>> >>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>> >>>>>>>>> To: dev@cloudstack.apache.org
>> >>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>> >>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >>>>>>>>>
>> >>>>>>>>> I agree on merging Wei's feature first, then mine.
>> >>>>>>>>>
>> >>>>>>>>> If his feature is for KVM only, then it is a non issue as I
>> don't
>> >>>>>>> support
>> >>>>>>>>> KVM in 4.2.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <ustcweizhou@gmail.com
>> >
>> >>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> John,
>> >>>>>>>>>>
>> >>>>>>>>>> For the billing, as no one works on billing now, users need to
>> >>>>>>> calculate
>> >>>>>>>>>> the billing by themselves. They can get the service_offering
>> and
>> >>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of course
>> >>>> it is
>> >>>>>>>>> better
>> >>>>>>>>>> to tell user the exact limitation value of individual volume,
>> >>>> and
>> >>>>>>> network
>> >>>>>>>>>> rate limitation for nics as well. I can work on it later. Do
>> you
>> >>>>>>> think it
>> >>>>>>>>>> is a part of I/O throttling?
>> >>>>>>>>>>
>> >>>>>>>>>> Sorry my misunstand the second the question.
>> >>>>>>>>>>
>> >>>>>>>>>> Agree with what you said about the two features.
>> >>>>>>>>>>
>> >>>>>>>>>> -Wei
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>
>> >>>>>>>>>>> Wei,
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
>> >>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hi John, Mike
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding more.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> (1) I think billing should depend on IO statistics rather
>> than
>> >>>>> IOPS
>> >>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
>> >>>>>>>>> disk_io_stat
>> >>>>>>>>>>> can
>> >>>>>>>>>>>> get the IO statistics including bytes/iops read/write for an
>> >>>>>>>>> individual
>> >>>>>>>>>>>> virtual machine.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Going by the AWS model, customers are billed more for volumes
>> >>>> with
>> >>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>> >>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users would
>> >>>> like
>> >>>>>>> the
>> >>>>>>>>>>> option to employ similar cost models. Could an operator
>> >>>> implement
>> >>>>>>>>> such a
>> >>>>>>>>>>> billing model in the current patch?
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
>> >>>> IOPS/BPS
>> >>>>>>>>>>>> limitation for a running virtual machine through command
>> line.
>> >>>>>>>>> However,
>> >>>>>>>>>>>> CloudStack does not support changing the parameters of a
>> >>>> created
>> >>>>>>>>>> offering
>> >>>>>>>>>>>> (computer offering or disk offering).
>> >>>>>>>>>>>
>> >>>>>>>>>>> I meant at the Java interface level. I apologize for being
>> >>>>> unclear.
>> >>>>>>>>> Can
>> >>>>>>>>>>> we more generalize allocation algorithms with a set of
>> >>>> interfaces
>> >>>>>>> that
>> >>>>>>>>>>> describe the service guarantees provided by a resource?
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit
>> Mike's
>> >>>>> patch
>> >>>>>>>>>>> after
>> >>>>>>>>>>>> disk_io_throttling as Mike needs to consider the limitation
>> in
>> >>>>>>>>>> hypervisor
>> >>>>>>>>>>>> type, I think.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>> >>>> regarding
>> >>>>>>> the
>> >>>>>>>>>>> touch points between these two features. I think that
>> >>>>>>>>> disk_io_throttling
>> >>>>>>>>>>> will need to be merged before SolidFire, but I think we need
>> >>>> closer
>> >>>>>>>>>>> coordination between the branches (possibly have have
>> solidfire
>> >>>>> track
>> >>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>> >>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> - Wei
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Mike,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The things I want to understand are the following:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured in
>> >>>> a
>> >>>>>>>>> common
>> >>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
>> >>>>> offerings).
>> >>>>>>>>>>>>> 2) Should there be a common interface model for reasoning
>> >>>> about
>> >>>>>>>>> IOP
>> >>>>>>>>>>>>> provisioning at runtime?
>> >>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>> >>>> between
>> >>>>> a
>> >>>>>>>>>>>>> hypervisor and storage device reconciled? In particular, a
>> >>>>>>> scenario
>> >>>>>>>>>>> where
>> >>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured for
>> >>>> a VM
>> >>>>>>>>>> than a
>> >>>>>>>>>>>>> storage device has been configured to deliver. Another
>> >>>> scenario
>> >>>>>>>>>> could a
>> >>>>>>>>>>>>> consistent configuration between a VM and a storage device
>> at
>> >>>>>>>>> creation
>> >>>>>>>>>>>>> time, but a later modification to storage device introduces
>> >>>>> logical
>> >>>>>>>>>>>>> inconsistency.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> >>>>>>>>>>> mike.tutkowski@solidfire.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Hi John,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>> >>>> number of
>> >>>>>>>>> IOPS
>> >>>>>>>>>>> from
>> >>>>>>>>>>>>> the hypervisor side.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> My feature is focused on controlling IOPS from the storage
>> >>>> system
>> >>>>>>>>>> side.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I hope that helps. :)
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>> >>>> jburwell@basho.com
>> >>>>>>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wei,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> My opinion is that no features should be merged until all
>> >>>>>>>>> functional
>> >>>>>>>>>>>>>> issues have been resolved and it is ready to turn over to
>> >>>> test.
>> >>>>>>>>>> Until
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed and
>> >>>>>>>>>> re-reviewed
>> >>>>>>>>>>>>> by
>> >>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Also, how does this work intersect/compliment the SolidFire
>> >>>>> patch
>> >>>>>>> (
>> >>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
>> >>>> that
>> >>>>>>>>> work
>> >>>>>>>>>> is
>> >>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure we
>> >>>> don't
>> >>>>>>>>>> have a
>> >>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>> >>>>>>>>>> unnecessarily
>> >>>>>>>>>>>>>> incompatible.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
>> ustcweizhou@gmail.com
>> >>>>>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Sure. I will change it next week.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> If anyone object thems for technical reason till Monday, I
>> >>>> will
>> >>>>>>>>>> revert
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> them.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> For the sake of clarity I just want to mention again that
>> we
>> >>>>>>> should
>> >>>>>>>>>>>>> change
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never release a
>> >>>>> version
>> >>>>>>>>>> with
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> only total IOps.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and that's
>> >>>> great!
>> >>>>>>> We
>> >>>>>>>>>>>>> should
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wido,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read and
>> >>>> write
>> >>>>>>>>>> IOPS.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> don't see how it could be fixed later because I don't see
>> >>>> how we
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> correctly
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we would
>> >>>> be
>> >>>>>>> stuck
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> with a
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> total unless/until we decided to break backwards
>> >>>> compatibility.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> What Wei meant was merging it into master now so that it
>> >>>> will go
>> >>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release
>> >>>> so
>> >>>>>>> that
>> >>>>>>>>>> 4.2
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> will be released with Read and Write instead of Total IOps.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But if
>> the
>> >>>>>>> window
>> >>>>>>>>>>> moves
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (see other threads) then it won't be necessary to do that.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I also completely agree that there is no association
>> between
>> >>>>>>>>> network
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> disk I/O.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> -John
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>> >>>> wido@widodh.nl
>> >>>>>>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi Wido,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks. Good question.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
>> >>>> ignore
>> >>>>> the
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> difference of read and write mainly because the network
>> >>>>> throttling
>> >>>>>>>>>> did
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> not
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> care the difference of sent and received bytes as well.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O
>> completely
>> >>>>>>>>>> different.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
>> >>>> network
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> bandwith.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could be
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> implemented in
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>> >>>> implement
>> >>>>> it
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> after
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> that , if needed.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes into
>> >>>> the
>> >>>>> 4.2
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> window we have to make sure we don't release with only
>> total
>> >>>>> IOps
>> >>>>>>>>> and
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> fix
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> -Wei
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi Wei,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into
>> master.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> The purpose is :
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Virtual machines are running on the same storage device
>> >>>> (local
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> storage or
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>> >>>> (such as
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> iops), if
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>> >>>>>>> performance
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> other VMs running on the same storage device.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
>> >>>> I/O
>> >>>>> of
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> VMs.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Looking at the code I see you make no difference between
>> >>>> Read
>> >>>>> and
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> IOps.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate for
>> >>>> Read
>> >>>>>>> and
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> It's also strange, in the polling side you collect both the
>> >>>> Read
>> >>>>>>>>> and
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Write
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> IOps, but on the throttling side you only go for a global
>> >>>> value.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Write IOps are usually much more expensive then Read IOps,
>> >>>> so it
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> seems
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> like a valid use-case where that an admin would set a lower
>> >>>>> value
>> >>>>>>>>> for
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> write
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> IOps vs Read IOps.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Since this only supports KVM at this point I think it would
>> >>>> be
>> >>>>> of
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> great
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> value to at least have the mechanism in place to support
>> >>>> both,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> implementing
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> this later would be a lot of work.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> If a hypervisor doesn't support setting different values
>> for
>> >>>>> read
>> >>>>>>>>> and
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> write you can always sum both up and set that as the total
>> >>>>> limit.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Can you explain why you implemented it this way?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wido
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> The feature includes:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>> >>>> global
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> configuration)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>> >>>> issues.apache.org/****
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <ht**tps://
>> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> FS (I will update later) :
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>
>> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>> <
>> >>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>> <
>> >>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki
>> .
>> >>>> **
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>> <
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Merge check list :-
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Yes
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> * Are there new dependencies introduced?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> No
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> * What automated testing (unit and integration) is included
>> >>>> in
>> >>>>> the
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> feature?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Unit tests are added.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> * What testing has been done to check for potential
>> >>>> regressions?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (2) VM operations, including
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>> >>>> restore
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (3) Volume operations, including
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Attach, Detach
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> To review the code, you can try
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Best regards,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Wei
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>
>> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <
>> >>>>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> >>>>> <
>> >>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> >
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>> >>>>>>> cwiki.apache.org/confluence/****
>> >>>>>>>>> <
>> >>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki
>> .
>> >>>> **
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >>>>>>>>> <
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> [3]
>> >>>>>>>>>>>>>>
>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>> >>>> <
>> >>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <ht**tps://
>> >>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <ht**tps://
>> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <ht**tps://
>> >>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> **<
>> >>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <**
>> >>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071
>> <
>> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>>>> *â„¢*
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *â„¢*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the
>> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *â„¢*
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Mike Tutkowski*
>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>> e: mike.tutkowski@solidfire.com
>> >>>>> o: 303.746.7302
>> >>>>> Advancing the way the world uses the
>> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>> *â„¢*
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> *Mike Tutkowski*
>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>> e: mike.tutkowski@solidfire.com
>> >>> o: 303.746.7302
>> >>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >>> *â„¢*
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>> >> *â„¢*
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
As far as I know, the iSCSI type is uniquely used by XenServer when you
want to set up Primary Storage that is directly based on an iSCSI target.
This allows you to skip the step of going to the hypervisor and creating a
storage repository based on that iSCSI target as CloudStack does that part
for you. I think this is only supported for XenServer. For all other
hypervisors, you must first go to the hypervisor and perform this step
manually.
I don't really know what RBD is.
On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Reading through the code, what is the difference between the ISCSI and
> Dynamic types? Why isn't RBD considered Dynamic?
>
> Thanks,
> -John
>
> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > This new type of storage is defined in the Storage.StoragePoolType class
> > (called Dynamic):
> >
> > public static enum StoragePoolType {
> >
> > Filesystem(false), // local directory
> >
> > NetworkFilesystem(true), // NFS or CIFS
> >
> > IscsiLUN(true), // shared LUN, with a clusterfs overlay
> >
> > Iscsi(true), // for e.g., ZFS Comstar
> >
> > ISO(false), // for iso image
> >
> > LVM(false), // XenServer local LVM SR
> >
> > CLVM(true),
> >
> > RBD(true),
> >
> > SharedMountPoint(true),
> >
> > VMFS(true), // VMware VMFS storage
> >
> > PreSetup(true), // for XenServer, Storage Pool is set up by
> > customers.
> >
> > EXT(false), // XenServer local EXT SR
> >
> > OCFS2(true),
> >
> > Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
> >
> >
> > boolean shared;
> >
> >
> > StoragePoolType(boolean shared) {
> >
> > this.shared = shared;
> >
> > }
> >
> >
> > public boolean isShared() {
> >
> > return shared;
> >
> > }
> >
> > }
> >
> >
> > On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> >> wrote:
> >
> >> For example, let's say another storage company wants to implement a
> >> plug-in to leverage its Quality of Service feature. It would be dynamic,
> >> zone-wide storage, as well. They would need only implement a storage
> >> plug-in as I've made the necessary changes to the hypervisor-attach
> logic
> >> to support their plug-in.
> >>
> >>
> >> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com> wrote:
> >>
> >>> Oh, sorry to imply the XenServer code is SolidFire specific. It is not.
> >>>
> >>> The XenServer attach logic is now aware of dynamic, zone-wide storage
> >>> (and SolidFire is an implementation of this kind of storage). This
> kind of
> >>> storage is new to 4.2 with Edison's storage framework changes.
> >>>
> >>> Edison created a new framework that supported the creation and deletion
> >>> of volumes dynamically. However, when I visited with him in Portland
> back
> >>> in April, we realized that it was not complete. We realized there was
> >>> nothing CloudStack could do with these volumes unless the attach logic
> was
> >>> changed to recognize this new type of storage and create the
> appropriate
> >>> hypervisor data structure.
> >>>
> >>>
> >>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com>
> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> It is generally odd to me that any operation in the Storage layer
> would
> >>>> understand or care about details. I expect to see the Storage
> services
> >>>> expose a set of operations that can be composed/driven by the
> Hypervisor
> >>>> implementations to allocate space/create structures per their needs.
> If
> >>>> we
> >>>> don't invert this dependency, we are going to end with a massive
> n-to-n
> >>>> problem that will make the system increasingly difficult to maintain
> and
> >>>> enhance. Am I understanding that the Xen specific SolidFire code is
> >>>> located in the CitrixResourceBase class?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>>
> >>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com
> >>>>> wrote:
> >>>>
> >>>>> To delve into this in a bit more detail:
> >>>>>
> >>>>> Prior to 4.2 and aside from one setup method for XenServer, the admin
> >>>> had
> >>>>> to first create a volume on the storage system, then go into the
> >>>> hypervisor
> >>>>> to set up a data structure to make use of the volume (ex. a storage
> >>>>> repository on XenServer or a datastore on ESX(i)). VMs and data disks
> >>>> then
> >>>>> shared this storage system's volume.
> >>>>>
> >>>>> With Edison's new storage framework, storage need no longer be so
> >>>> static
> >>>>> and you can easily create a 1:1 relationship between a storage
> system's
> >>>>> volume and the VM's data disk (necessary for storage Quality of
> >>>> Service).
> >>>>>
> >>>>> You can now write a plug-in that is called to dynamically create and
> >>>> delete
> >>>>> volumes as needed.
> >>>>>
> >>>>> The problem that the storage framework did not address is in creating
> >>>> and
> >>>>> deleting the hypervisor-specific data structure when performing an
> >>>>> attach/detach.
> >>>>>
> >>>>> That being the case, I've been enhancing it to do so. I've got
> >>>> XenServer
> >>>>> worked out and submitted. I've got ESX(i) in my sandbox and can
> submit
> >>>> this
> >>>>> if we extend the 4.2 freeze date.
> >>>>>
> >>>>> Does that help a bit? :)
> >>>>>
> >>>>>
> >>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com
> >>>>>> wrote:
> >>>>>
> >>>>>> Hi John,
> >>>>>>
> >>>>>> The storage plug-in - by itself - is hypervisor agnostic.
> >>>>>>
> >>>>>> The issue is with the volume-attach logic (in the agent code). The
> >>>>> storage
> >>>>>> framework calls into the plug-in to have it create a volume as
> >>>> needed,
> >>>>> but
> >>>>>> when the time comes to attach the volume to a hypervisor, the attach
> >>>>> logic
> >>>>>> has to be smart enough to recognize it's being invoked on zone-wide
> >>>>> storage
> >>>>>> (where the volume has just been created) and create, say, a storage
> >>>>>> repository (for XenServer) or a datastore (for VMware) to make use
> >>>> of the
> >>>>>> volume that was just created.
> >>>>>>
> >>>>>> I've been spending most of my time recently making the attach logic
> >>>> work
> >>>>>> in the agent code.
> >>>>>>
> >>>>>> Does that clear it up?
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Mike,
> >>>>>>>
> >>>>>>> Can you explain why the the storage driver is hypervisor specific?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Yes, ultimately I would like to support all hypervisors that
> >>>>> CloudStack
> >>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
> >>>>>>>>
> >>>>>>>> Right now this plug-in supports XenServer. Depending on what we do
> >>>>> with
> >>>>>>>> regards to 4.2 feature freeze, I have it working for VMware in my
> >>>>>>> sandbox,
> >>>>>>>> as well.
> >>>>>>>>
> >>>>>>>> Also, just to be clear, this is all in regards to Disk Offerings.
> >>>> I
> >>>>>>> plan to
> >>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
> >>>>> kelcey@bbits.ca
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Is there any plan on supporting KVM in the patch cycle post 4.2?
> >>>>>>>>>
> >>>>>>>>> ----- Original Message -----
> >>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>
> >>>>>>>>> I agree on merging Wei's feature first, then mine.
> >>>>>>>>>
> >>>>>>>>> If his feature is for KVM only, then it is a non issue as I don't
> >>>>>>> support
> >>>>>>>>> KVM in 4.2.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> John,
> >>>>>>>>>>
> >>>>>>>>>> For the billing, as no one works on billing now, users need to
> >>>>>>> calculate
> >>>>>>>>>> the billing by themselves. They can get the service_offering and
> >>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of course
> >>>> it is
> >>>>>>>>> better
> >>>>>>>>>> to tell user the exact limitation value of individual volume,
> >>>> and
> >>>>>>> network
> >>>>>>>>>> rate limitation for nics as well. I can work on it later. Do you
> >>>>>>> think it
> >>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>
> >>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>
> >>>>>>>>>> Agree with what you said about the two features.
> >>>>>>>>>>
> >>>>>>>>>> -Wei
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>
> >>>>>>>>>>> Wei,
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>
> >>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding more.
> >>>>>>>>>>>>
> >>>>>>>>>>>> (1) I think billing should depend on IO statistics rather than
> >>>>> IOPS
> >>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
> >>>>>>>>> disk_io_stat
> >>>>>>>>>>> can
> >>>>>>>>>>>> get the IO statistics including bytes/iops read/write for an
> >>>>>>>>> individual
> >>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>
> >>>>>>>>>>> Going by the AWS model, customers are billed more for volumes
> >>>> with
> >>>>>>>>>>> provisioned IOPS, as well as, for those operations (
> >>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users would
> >>>> like
> >>>>>>> the
> >>>>>>>>>>> option to employ similar cost models. Could an operator
> >>>> implement
> >>>>>>>>> such a
> >>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
> >>>> IOPS/BPS
> >>>>>>>>>>>> limitation for a running virtual machine through command line.
> >>>>>>>>> However,
> >>>>>>>>>>>> CloudStack does not support changing the parameters of a
> >>>> created
> >>>>>>>>>> offering
> >>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>
> >>>>>>>>>>> I meant at the Java interface level. I apologize for being
> >>>>> unclear.
> >>>>>>>>> Can
> >>>>>>>>>>> we more generalize allocation algorithms with a set of
> >>>> interfaces
> >>>>>>> that
> >>>>>>>>>>> describe the service guarantees provided by a resource?
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit Mike's
> >>>>> patch
> >>>>>>>>>>> after
> >>>>>>>>>>>> disk_io_throttling as Mike needs to consider the limitation in
> >>>>>>>>>> hypervisor
> >>>>>>>>>>>> type, I think.
> >>>>>>>>>>>
> >>>>>>>>>>> I will expand on my thoughts in a later response to Mike
> >>>> regarding
> >>>>>>> the
> >>>>>>>>>>> touch points between these two features. I think that
> >>>>>>>>> disk_io_throttling
> >>>>>>>>>>> will need to be merged before SolidFire, but I think we need
> >>>> closer
> >>>>>>>>>>> coordination between the branches (possibly have have solidfire
> >>>>> track
> >>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Wei
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The things I want to understand are the following:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured in
> >>>> a
> >>>>>>>>> common
> >>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
> >>>>> offerings).
> >>>>>>>>>>>>> 2) Should there be a common interface model for reasoning
> >>>> about
> >>>>>>>>> IOP
> >>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
> >>>> between
> >>>>> a
> >>>>>>>>>>>>> hypervisor and storage device reconciled? In particular, a
> >>>>>>> scenario
> >>>>>>>>>>> where
> >>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured for
> >>>> a VM
> >>>>>>>>>> than a
> >>>>>>>>>>>>> storage device has been configured to deliver. Another
> >>>> scenario
> >>>>>>>>>> could a
> >>>>>>>>>>>>> consistent configuration between a VM and a storage device at
> >>>>>>>>> creation
> >>>>>>>>>>>>> time, but a later modification to storage device introduces
> >>>>> logical
> >>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
> >>>> number of
> >>>>>>>>> IOPS
> >>>>>>>>>>> from
> >>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> My feature is focused on controlling IOPS from the storage
> >>>> system
> >>>>>>>>>> side.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
> >>>> jburwell@basho.com
> >>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> My opinion is that no features should be merged until all
> >>>>>>>>> functional
> >>>>>>>>>>>>>> issues have been resolved and it is ready to turn over to
> >>>> test.
> >>>>>>>>>> Until
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed and
> >>>>>>>>>> re-reviewed
> >>>>>>>>>>>>> by
> >>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Also, how does this work intersect/compliment the SolidFire
> >>>>> patch
> >>>>>>> (
> >>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
> >>>> that
> >>>>>>>>> work
> >>>>>>>>>> is
> >>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure we
> >>>> don't
> >>>>>>>>>> have a
> >>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
> >>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <ustcweizhou@gmail.com
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Exactly. I have pushed the features into master.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If anyone object thems for technical reason till Monday, I
> >>>> will
> >>>>>>>>>> revert
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> For the sake of clarity I just want to mention again that we
> >>>>>>> should
> >>>>>>>>>>>>> change
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never release a
> >>>>> version
> >>>>>>>>>> with
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and that's
> >>>> great!
> >>>>>>> We
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> however prevent that we create legacy from day #1.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read and
> >>>> write
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> don't see how it could be fixed later because I don't see
> >>>> how we
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we would
> >>>> be
> >>>>>>> stuck
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> total unless/until we decided to break backwards
> >>>> compatibility.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What Wei meant was merging it into master now so that it
> >>>> will go
> >>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release
> >>>> so
> >>>>>>> that
> >>>>>>>>>> 4.2
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> will be released with Read and Write instead of Total IOps.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But if the
> >>>>>>> window
> >>>>>>>>>>> moves
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (see other threads) then it won't be necessary to do that.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I also completely agree that there is no association between
> >>>>>>>>> network
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
> >>>> wido@widodh.nl
> >>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
> >>>> ignore
> >>>>> the
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> difference of read and write mainly because the network
> >>>>> throttling
> >>>>>>>>>> did
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> care the difference of sent and received bytes as well.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O completely
> >>>>>>>>>> different.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
> >>>> network
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could be
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
> >>>> implement
> >>>>> it
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes into
> >>>> the
> >>>>> 4.2
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> window we have to make sure we don't release with only total
> >>>>> IOps
> >>>>>>>>> and
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Virtual machines are running on the same storage device
> >>>> (local
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> share strage). Because of the rate limitation of device
> >>>> (such as
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
> >>>>>>> performance
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> other VMs running on the same storage device.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
> >>>> I/O
> >>>>> of
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Looking at the code I see you make no difference between
> >>>> Read
> >>>>> and
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate for
> >>>> Read
> >>>>>>> and
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> It's also strange, in the polling side you collect both the
> >>>> Read
> >>>>>>>>> and
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> IOps, but on the throttling side you only go for a global
> >>>> value.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Write IOps are usually much more expensive then Read IOps,
> >>>> so it
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> like a valid use-case where that an admin would set a lower
> >>>>> value
> >>>>>>>>> for
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Since this only supports KVM at this point I think it would
> >>>> be
> >>>>> of
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> value to at least have the mechanism in place to support
> >>>> both,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If a hypervisor doesn't support setting different values for
> >>>>> read
> >>>>>>>>> and
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> write you can always sum both up and set that as the total
> >>>>> limit.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Can you explain why you implemented it this way?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
> >>>> global
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>> issues.apache.org/****
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <ht**tps://
> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>
> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>> <
> >>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>> <
> >>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
> >>>> **
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>> <
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> * What automated testing (unit and integration) is included
> >>>> in
> >>>>> the
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> * What testing has been done to check for potential
> >>>> regressions?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> >>>> restore
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>
> >>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>
> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>> <
> >>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>> <
> >>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
> >>>> **
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>> <
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>> <
> >>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <ht**tps://
> >>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <ht**tps://
> >>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <ht**tps://
> >>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - VM Disk I/O Throttling)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *â„¢*
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *â„¢*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Mike Tutkowski*
> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>> o: 303.746.7302
> >>>>>>>> Advancing the way the world uses the
> >>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>> *â„¢*
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *â„¢*
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >>> *â„¢*
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Reading through the code, what is the difference between the ISCSI and Dynamic types? Why isn't RBD considered Dynamic?
Thanks,
-John
On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> This new type of storage is defined in the Storage.StoragePoolType class
> (called Dynamic):
>
> public static enum StoragePoolType {
>
> Filesystem(false), // local directory
>
> NetworkFilesystem(true), // NFS or CIFS
>
> IscsiLUN(true), // shared LUN, with a clusterfs overlay
>
> Iscsi(true), // for e.g., ZFS Comstar
>
> ISO(false), // for iso image
>
> LVM(false), // XenServer local LVM SR
>
> CLVM(true),
>
> RBD(true),
>
> SharedMountPoint(true),
>
> VMFS(true), // VMware VMFS storage
>
> PreSetup(true), // for XenServer, Storage Pool is set up by
> customers.
>
> EXT(false), // XenServer local EXT SR
>
> OCFS2(true),
>
> Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
>
>
> boolean shared;
>
>
> StoragePoolType(boolean shared) {
>
> this.shared = shared;
>
> }
>
>
> public boolean isShared() {
>
> return shared;
>
> }
>
> }
>
>
> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
>> wrote:
>
>> For example, let's say another storage company wants to implement a
>> plug-in to leverage its Quality of Service feature. It would be dynamic,
>> zone-wide storage, as well. They would need only implement a storage
>> plug-in as I've made the necessary changes to the hypervisor-attach logic
>> to support their plug-in.
>>
>>
>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> Oh, sorry to imply the XenServer code is SolidFire specific. It is not.
>>>
>>> The XenServer attach logic is now aware of dynamic, zone-wide storage
>>> (and SolidFire is an implementation of this kind of storage). This kind of
>>> storage is new to 4.2 with Edison's storage framework changes.
>>>
>>> Edison created a new framework that supported the creation and deletion
>>> of volumes dynamically. However, when I visited with him in Portland back
>>> in April, we realized that it was not complete. We realized there was
>>> nothing CloudStack could do with these volumes unless the attach logic was
>>> changed to recognize this new type of storage and create the appropriate
>>> hypervisor data structure.
>>>
>>>
>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com> wrote:
>>>
>>>> Mike,
>>>>
>>>> It is generally odd to me that any operation in the Storage layer would
>>>> understand or care about details. I expect to see the Storage services
>>>> expose a set of operations that can be composed/driven by the Hypervisor
>>>> implementations to allocate space/create structures per their needs. If
>>>> we
>>>> don't invert this dependency, we are going to end with a massive n-to-n
>>>> problem that will make the system increasingly difficult to maintain and
>>>> enhance. Am I understanding that the Xen specific SolidFire code is
>>>> located in the CitrixResourceBase class?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>>
>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com
>>>>> wrote:
>>>>
>>>>> To delve into this in a bit more detail:
>>>>>
>>>>> Prior to 4.2 and aside from one setup method for XenServer, the admin
>>>> had
>>>>> to first create a volume on the storage system, then go into the
>>>> hypervisor
>>>>> to set up a data structure to make use of the volume (ex. a storage
>>>>> repository on XenServer or a datastore on ESX(i)). VMs and data disks
>>>> then
>>>>> shared this storage system's volume.
>>>>>
>>>>> With Edison's new storage framework, storage need no longer be so
>>>> static
>>>>> and you can easily create a 1:1 relationship between a storage system's
>>>>> volume and the VM's data disk (necessary for storage Quality of
>>>> Service).
>>>>>
>>>>> You can now write a plug-in that is called to dynamically create and
>>>> delete
>>>>> volumes as needed.
>>>>>
>>>>> The problem that the storage framework did not address is in creating
>>>> and
>>>>> deleting the hypervisor-specific data structure when performing an
>>>>> attach/detach.
>>>>>
>>>>> That being the case, I've been enhancing it to do so. I've got
>>>> XenServer
>>>>> worked out and submitted. I've got ESX(i) in my sandbox and can submit
>>>> this
>>>>> if we extend the 4.2 freeze date.
>>>>>
>>>>> Does that help a bit? :)
>>>>>
>>>>>
>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com
>>>>>> wrote:
>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> The storage plug-in - by itself - is hypervisor agnostic.
>>>>>>
>>>>>> The issue is with the volume-attach logic (in the agent code). The
>>>>> storage
>>>>>> framework calls into the plug-in to have it create a volume as
>>>> needed,
>>>>> but
>>>>>> when the time comes to attach the volume to a hypervisor, the attach
>>>>> logic
>>>>>> has to be smart enough to recognize it's being invoked on zone-wide
>>>>> storage
>>>>>> (where the volume has just been created) and create, say, a storage
>>>>>> repository (for XenServer) or a datastore (for VMware) to make use
>>>> of the
>>>>>> volume that was just created.
>>>>>>
>>>>>> I've been spending most of my time recently making the attach logic
>>>> work
>>>>>> in the agent code.
>>>>>>
>>>>>> Does that clear it up?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
>>>>> wrote:
>>>>>>
>>>>>>> Mike,
>>>>>>>
>>>>>>> Can you explain why the the storage driver is hypervisor specific?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yes, ultimately I would like to support all hypervisors that
>>>>> CloudStack
>>>>>>>> supports. I think I'm just out of time for 4.2 to get KVM in.
>>>>>>>>
>>>>>>>> Right now this plug-in supports XenServer. Depending on what we do
>>>>> with
>>>>>>>> regards to 4.2 feature freeze, I have it working for VMware in my
>>>>>>> sandbox,
>>>>>>>> as well.
>>>>>>>>
>>>>>>>> Also, just to be clear, this is all in regards to Disk Offerings.
>>>> I
>>>>>>> plan to
>>>>>>>> support Compute Offerings post 4.2.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>>>> kelcey@bbits.ca
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Is there any plan on supporting KVM in the patch cycle post 4.2?
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Mike Tutkowski" <mi...@solidfire.com>
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>>>>>>>>
>>>>>>>>> I agree on merging Wei's feature first, then mine.
>>>>>>>>>
>>>>>>>>> If his feature is for KVM only, then it is a non issue as I don't
>>>>>>> support
>>>>>>>>> KVM in 4.2.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> John,
>>>>>>>>>>
>>>>>>>>>> For the billing, as no one works on billing now, users need to
>>>>>>> calculate
>>>>>>>>>> the billing by themselves. They can get the service_offering and
>>>>>>>>>> disk_offering of a VMs and volumes for calculation. Of course
>>>> it is
>>>>>>>>> better
>>>>>>>>>> to tell user the exact limitation value of individual volume,
>>>> and
>>>>>>> network
>>>>>>>>>> rate limitation for nics as well. I can work on it later. Do you
>>>>>>> think it
>>>>>>>>>> is a part of I/O throttling?
>>>>>>>>>>
>>>>>>>>>> Sorry my misunstand the second the question.
>>>>>>>>>>
>>>>>>>>>> Agree with what you said about the two features.
>>>>>>>>>>
>>>>>>>>>> -Wei
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>
>>>>>>>>>>> Wei,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi John, Mike
>>>>>>>>>>>>
>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying to adding more.
>>>>>>>>>>>>
>>>>>>>>>>>> (1) I think billing should depend on IO statistics rather than
>>>>> IOPS
>>>>>>>>>>>> limitation. Please review disk_io_stat if you have time.
>>>>>>>>> disk_io_stat
>>>>>>>>>>> can
>>>>>>>>>>>> get the IO statistics including bytes/iops read/write for an
>>>>>>>>> individual
>>>>>>>>>>>> virtual machine.
>>>>>>>>>>>
>>>>>>>>>>> Going by the AWS model, customers are billed more for volumes
>>>> with
>>>>>>>>>>> provisioned IOPS, as well as, for those operations (
>>>>>>>>>>> http://aws.amazon.com/ebs/). I would imagine our users would
>>>> like
>>>>>>> the
>>>>>>>>>>> option to employ similar cost models. Could an operator
>>>> implement
>>>>>>>>> such a
>>>>>>>>>>> billing model in the current patch?
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM supports setting
>>>> IOPS/BPS
>>>>>>>>>>>> limitation for a running virtual machine through command line.
>>>>>>>>> However,
>>>>>>>>>>>> CloudStack does not support changing the parameters of a
>>>> created
>>>>>>>>>> offering
>>>>>>>>>>>> (computer offering or disk offering).
>>>>>>>>>>>
>>>>>>>>>>> I meant at the Java interface level. I apologize for being
>>>>> unclear.
>>>>>>>>> Can
>>>>>>>>>>> we more generalize allocation algorithms with a set of
>>>> interfaces
>>>>>>> that
>>>>>>>>>>> describe the service guarantees provided by a resource?
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> (3) It is a good question. Maybe it is better to commit Mike's
>>>>> patch
>>>>>>>>>>> after
>>>>>>>>>>>> disk_io_throttling as Mike needs to consider the limitation in
>>>>>>>>>> hypervisor
>>>>>>>>>>>> type, I think.
>>>>>>>>>>>
>>>>>>>>>>> I will expand on my thoughts in a later response to Mike
>>>> regarding
>>>>>>> the
>>>>>>>>>>> touch points between these two features. I think that
>>>>>>>>> disk_io_throttling
>>>>>>>>>>> will need to be merged before SolidFire, but I think we need
>>>> closer
>>>>>>>>>>> coordination between the branches (possibly have have solidfire
>>>>> track
>>>>>>>>>>> disk_io_throttling) to coordinate on this issue.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> - Wei
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>>>>>>>>
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The things I want to understand are the following:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) Is there value in capturing IOPS policies be captured in
>>>> a
>>>>>>>>> common
>>>>>>>>>>>>> data model (e.g. for billing/usage purposes, expressing
>>>>> offerings).
>>>>>>>>>>>>> 2) Should there be a common interface model for reasoning
>>>> about
>>>>>>>>> IOP
>>>>>>>>>>>>> provisioning at runtime?
>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS configurations
>>>> between
>>>>> a
>>>>>>>>>>>>> hypervisor and storage device reconciled? In particular, a
>>>>>>> scenario
>>>>>>>>>>> where
>>>>>>>>>>>>> is lead to believe (and billed) for more IOPS configured for
>>>> a VM
>>>>>>>>>> than a
>>>>>>>>>>>>> storage device has been configured to deliver. Another
>>>> scenario
>>>>>>>>>> could a
>>>>>>>>>>>>> consistent configuration between a VM and a storage device at
>>>>>>>>> creation
>>>>>>>>>>>>> time, but a later modification to storage device introduces
>>>>> logical
>>>>>>>>>>>>> inconsistency.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I believe Wei's feature deals with controlling the max
>>>> number of
>>>>>>>>> IOPS
>>>>>>>>>>> from
>>>>>>>>>>>>> the hypervisor side.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My feature is focused on controlling IOPS from the storage
>>>> system
>>>>>>>>>> side.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I hope that helps. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>>> jburwell@basho.com
>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My opinion is that no features should be merged until all
>>>>>>>>> functional
>>>>>>>>>>>>>> issues have been resolved and it is ready to turn over to
>>>> test.
>>>>>>>>>> Until
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue is addressed and
>>>>>>>>>> re-reviewed
>>>>>>>>>>>>> by
>>>>>>>>>>>>>> Wido, I don't think this criteria has been satisfied.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also, how does this work intersect/compliment the SolidFire
>>>>> patch
>>>>>>> (
>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
>>>> that
>>>>>>>>> work
>>>>>>>>>> is
>>>>>>>>>>>>>> also involves provisioned IOPS. I would like to ensure we
>>>> don't
>>>>>>>>>> have a
>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>>>>>>>>>> unnecessarily
>>>>>>>>>>>>>> incompatible.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <ustcweizhou@gmail.com
>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sure. I will change it next week.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If anyone object thems for technical reason till Monday, I
>>>> will
>>>>>>>>>> revert
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For the sake of clarity I just want to mention again that we
>>>>>>> should
>>>>>>>>>>>>> change
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we never release a
>>>>> version
>>>>>>>>>> with
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> only total IOps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You laid the groundwork for the I/O throttling and that's
>>>> great!
>>>>>>> We
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> however prevent that we create legacy from day #1.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wido,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 -- this enhancement must to discretely support read and
>>>> write
>>>>>>>>>> IOPS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> don't see how it could be fixed later because I don't see
>>>> how we
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> split total IOPS into read and write. Therefore, we would
>>>> be
>>>>>>> stuck
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> total unless/until we decided to break backwards
>>>> compatibility.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What Wei meant was merging it into master now so that it
>>>> will go
>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release
>>>> so
>>>>>>> that
>>>>>>>>>> 4.2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> will be released with Read and Write instead of Total IOps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is to make the May 31st feature freeze date. But if the
>>>>>>> window
>>>>>>>>>>> moves
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (see other threads) then it won't be necessary to do that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I also completely agree that there is no association between
>>>>>>>>> network
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> disk I/O.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>>> wido@widodh.nl
>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Wido,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks. Good question.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I thought about at the beginning. Finally I decided to
>>>> ignore
>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> difference of read and write mainly because the network
>>>>> throttling
>>>>>>>>>> did
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> care the difference of sent and received bytes as well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That reasoning seems odd. Networking and disk I/O completely
>>>>>>>>>> different.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Disk I/O is much more expensive in most situations then
>>>> network
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> bandwith.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Implementing it will be some copy-paste work. It could be
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> implemented in
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> few days. For the deadline of feature freeze, I will
>>>> implement
>>>>> it
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> that , if needed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It think it's a feature we can't miss. But if it goes into
>>>> the
>>>>> 4.2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> window we have to make sure we don't release with only total
>>>>> IOps
>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> it in 4.3, that will confuse users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Wei,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Virtual machines are running on the same storage device
>>>> (local
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> storage or
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> share strage). Because of the rate limitation of device
>>>> (such as
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> iops), if
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk
>>>>>>> performance
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> other VMs running on the same storage device.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk
>>>> I/O
>>>>> of
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Looking at the code I see you make no difference between
>>>> Read
>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> IOps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Qemu and libvirt support setting both a different rate for
>>>> Read
>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> IOps which could benefit a lot of users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It's also strange, in the polling side you collect both the
>>>> Read
>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Write
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> IOps, but on the throttling side you only go for a global
>>>> value.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Write IOps are usually much more expensive then Read IOps,
>>>> so it
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> like a valid use-case where that an admin would set a lower
>>>>> value
>>>>>>>>> for
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> IOps vs Read IOps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Since this only supports KVM at this point I think it would
>>>> be
>>>>> of
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> great
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> value to at least have the mechanism in place to support
>>>> both,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> implementing
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> this later would be a lot of work.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If a hypervisor doesn't support setting different values for
>>>>> read
>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> write you can always sum both up and set that as the total
>>>>> limit.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you explain why you implemented it this way?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wido
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>>>> global
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>>> issues.apache.org/****
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <ht**tps://
>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> FS (I will update later) :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>> <
>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>> <
>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>>>> **
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>> <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * Are there new dependencies introduced?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * What automated testing (unit and integration) is included
>>>> in
>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * What testing has been done to check for potential
>>>> regressions?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>>>> restore
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (3) Volume operations, including
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To review the code, you can try
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>>>> <
>>>>>>>>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
>>>>>>> cwiki.apache.org/confluence/****
>>>>>>>>> <
>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>>>> **
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>>>>>>>> <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>>> <
>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <ht**tps://
>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <ht**tps://
>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <ht**tps://
>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> **<
>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <**
>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>> *â„¢*
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *â„¢*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the
>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *â„¢*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the
>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> *â„¢*
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
This new type of storage is defined in the Storage.StoragePoolType class
(called Dynamic):
public static enum StoragePoolType {
Filesystem(false), // local directory
NetworkFilesystem(true), // NFS or CIFS
IscsiLUN(true), // shared LUN, with a clusterfs overlay
Iscsi(true), // for e.g., ZFS Comstar
ISO(false), // for iso image
LVM(false), // XenServer local LVM SR
CLVM(true),
RBD(true),
SharedMountPoint(true),
VMFS(true), // VMware VMFS storage
PreSetup(true), // for XenServer, Storage Pool is set up by
customers.
EXT(false), // XenServer local EXT SR
OCFS2(true),
Dynamic(true); // dynamic, zone-wide storage (ex. SolidFire)
boolean shared;
StoragePoolType(boolean shared) {
this.shared = shared;
}
public boolean isShared() {
return shared;
}
}
On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> For example, let's say another storage company wants to implement a
> plug-in to leverage its Quality of Service feature. It would be dynamic,
> zone-wide storage, as well. They would need only implement a storage
> plug-in as I've made the necessary changes to the hypervisor-attach logic
> to support their plug-in.
>
>
> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Oh, sorry to imply the XenServer code is SolidFire specific. It is not.
>>
>> The XenServer attach logic is now aware of dynamic, zone-wide storage
>> (and SolidFire is an implementation of this kind of storage). This kind of
>> storage is new to 4.2 with Edison's storage framework changes.
>>
>> Edison created a new framework that supported the creation and deletion
>> of volumes dynamically. However, when I visited with him in Portland back
>> in April, we realized that it was not complete. We realized there was
>> nothing CloudStack could do with these volumes unless the attach logic was
>> changed to recognize this new type of storage and create the appropriate
>> hypervisor data structure.
>>
>>
>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> It is generally odd to me that any operation in the Storage layer would
>>> understand or care about details. I expect to see the Storage services
>>> expose a set of operations that can be composed/driven by the Hypervisor
>>> implementations to allocate space/create structures per their needs. If
>>> we
>>> don't invert this dependency, we are going to end with a massive n-to-n
>>> problem that will make the system increasingly difficult to maintain and
>>> enhance. Am I understanding that the Xen specific SolidFire code is
>>> located in the CitrixResourceBase class?
>>>
>>> Thanks,
>>> -John
>>>
>>>
>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com
>>> > wrote:
>>>
>>> > To delve into this in a bit more detail:
>>> >
>>> > Prior to 4.2 and aside from one setup method for XenServer, the admin
>>> had
>>> > to first create a volume on the storage system, then go into the
>>> hypervisor
>>> > to set up a data structure to make use of the volume (ex. a storage
>>> > repository on XenServer or a datastore on ESX(i)). VMs and data disks
>>> then
>>> > shared this storage system's volume.
>>> >
>>> > With Edison's new storage framework, storage need no longer be so
>>> static
>>> > and you can easily create a 1:1 relationship between a storage system's
>>> > volume and the VM's data disk (necessary for storage Quality of
>>> Service).
>>> >
>>> > You can now write a plug-in that is called to dynamically create and
>>> delete
>>> > volumes as needed.
>>> >
>>> > The problem that the storage framework did not address is in creating
>>> and
>>> > deleting the hypervisor-specific data structure when performing an
>>> > attach/detach.
>>> >
>>> > That being the case, I've been enhancing it to do so. I've got
>>> XenServer
>>> > worked out and submitted. I've got ESX(i) in my sandbox and can submit
>>> this
>>> > if we extend the 4.2 freeze date.
>>> >
>>> > Does that help a bit? :)
>>> >
>>> >
>>> > On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>>> > mike.tutkowski@solidfire.com
>>> > > wrote:
>>> >
>>> > > Hi John,
>>> > >
>>> > > The storage plug-in - by itself - is hypervisor agnostic.
>>> > >
>>> > > The issue is with the volume-attach logic (in the agent code). The
>>> > storage
>>> > > framework calls into the plug-in to have it create a volume as
>>> needed,
>>> > but
>>> > > when the time comes to attach the volume to a hypervisor, the attach
>>> > logic
>>> > > has to be smart enough to recognize it's being invoked on zone-wide
>>> > storage
>>> > > (where the volume has just been created) and create, say, a storage
>>> > > repository (for XenServer) or a datastore (for VMware) to make use
>>> of the
>>> > > volume that was just created.
>>> > >
>>> > > I've been spending most of my time recently making the attach logic
>>> work
>>> > > in the agent code.
>>> > >
>>> > > Does that clear it up?
>>> > >
>>> > > Thanks!
>>> > >
>>> > >
>>> > > On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
>>> > wrote:
>>> > >
>>> > >> Mike,
>>> > >>
>>> > >> Can you explain why the the storage driver is hypervisor specific?
>>> > >>
>>> > >> Thanks,
>>> > >> -John
>>> > >>
>>> > >> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>>> > mike.tutkowski@solidfire.com>
>>> > >> wrote:
>>> > >>
>>> > >> > Yes, ultimately I would like to support all hypervisors that
>>> > CloudStack
>>> > >> > supports. I think I'm just out of time for 4.2 to get KVM in.
>>> > >> >
>>> > >> > Right now this plug-in supports XenServer. Depending on what we do
>>> > with
>>> > >> > regards to 4.2 feature freeze, I have it working for VMware in my
>>> > >> sandbox,
>>> > >> > as well.
>>> > >> >
>>> > >> > Also, just to be clear, this is all in regards to Disk Offerings.
>>> I
>>> > >> plan to
>>> > >> > support Compute Offerings post 4.2.
>>> > >> >
>>> > >> >
>>> > >> > On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>>> > kelcey@bbits.ca
>>> > >> >wrote:
>>> > >> >
>>> > >> >> Is there any plan on supporting KVM in the patch cycle post 4.2?
>>> > >> >>
>>> > >> >> ----- Original Message -----
>>> > >> >> From: "Mike Tutkowski" <mi...@solidfire.com>
>>> > >> >> To: dev@cloudstack.apache.org
>>> > >> >> Sent: Monday, June 3, 2013 10:12:32 AM
>>> > >> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>> > >> >>
>>> > >> >> I agree on merging Wei's feature first, then mine.
>>> > >> >>
>>> > >> >> If his feature is for KVM only, then it is a non issue as I don't
>>> > >> support
>>> > >> >> KVM in 4.2.
>>> > >> >>
>>> > >> >>
>>> > >> >> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
>>> > >> wrote:
>>> > >> >>
>>> > >> >>> John,
>>> > >> >>>
>>> > >> >>> For the billing, as no one works on billing now, users need to
>>> > >> calculate
>>> > >> >>> the billing by themselves. They can get the service_offering and
>>> > >> >>> disk_offering of a VMs and volumes for calculation. Of course
>>> it is
>>> > >> >> better
>>> > >> >>> to tell user the exact limitation value of individual volume,
>>> and
>>> > >> network
>>> > >> >>> rate limitation for nics as well. I can work on it later. Do you
>>> > >> think it
>>> > >> >>> is a part of I/O throttling?
>>> > >> >>>
>>> > >> >>> Sorry my misunstand the second the question.
>>> > >> >>>
>>> > >> >>> Agree with what you said about the two features.
>>> > >> >>>
>>> > >> >>> -Wei
>>> > >> >>>
>>> > >> >>>
>>> > >> >>> 2013/6/3 John Burwell <jb...@basho.com>
>>> > >> >>>
>>> > >> >>>> Wei,
>>> > >> >>>>
>>> > >> >>>>
>>> > >> >>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
>>> > wrote:
>>> > >> >>>>
>>> > >> >>>>> Hi John, Mike
>>> > >> >>>>>
>>> > >> >>>>> I hope Mike's aswer helps you. I am trying to adding more.
>>> > >> >>>>>
>>> > >> >>>>> (1) I think billing should depend on IO statistics rather than
>>> > IOPS
>>> > >> >>>>> limitation. Please review disk_io_stat if you have time.
>>> > >> >> disk_io_stat
>>> > >> >>>> can
>>> > >> >>>>> get the IO statistics including bytes/iops read/write for an
>>> > >> >> individual
>>> > >> >>>>> virtual machine.
>>> > >> >>>>
>>> > >> >>>> Going by the AWS model, customers are billed more for volumes
>>> with
>>> > >> >>>> provisioned IOPS, as well as, for those operations (
>>> > >> >>>> http://aws.amazon.com/ebs/). I would imagine our users would
>>> like
>>> > >> the
>>> > >> >>>> option to employ similar cost models. Could an operator
>>> implement
>>> > >> >> such a
>>> > >> >>>> billing model in the current patch?
>>> > >> >>>>
>>> > >> >>>>>
>>> > >> >>>>> (2) Do you mean IOPS runtime change? KVM supports setting
>>> IOPS/BPS
>>> > >> >>>>> limitation for a running virtual machine through command line.
>>> > >> >> However,
>>> > >> >>>>> CloudStack does not support changing the parameters of a
>>> created
>>> > >> >>> offering
>>> > >> >>>>> (computer offering or disk offering).
>>> > >> >>>>
>>> > >> >>>> I meant at the Java interface level. I apologize for being
>>> > unclear.
>>> > >> >> Can
>>> > >> >>>> we more generalize allocation algorithms with a set of
>>> interfaces
>>> > >> that
>>> > >> >>>> describe the service guarantees provided by a resource?
>>> > >> >>>>
>>> > >> >>>>>
>>> > >> >>>>> (3) It is a good question. Maybe it is better to commit Mike's
>>> > patch
>>> > >> >>>> after
>>> > >> >>>>> disk_io_throttling as Mike needs to consider the limitation in
>>> > >> >>> hypervisor
>>> > >> >>>>> type, I think.
>>> > >> >>>>
>>> > >> >>>> I will expand on my thoughts in a later response to Mike
>>> regarding
>>> > >> the
>>> > >> >>>> touch points between these two features. I think that
>>> > >> >> disk_io_throttling
>>> > >> >>>> will need to be merged before SolidFire, but I think we need
>>> closer
>>> > >> >>>> coordination between the branches (possibly have have solidfire
>>> > track
>>> > >> >>>> disk_io_throttling) to coordinate on this issue.
>>> > >> >>>>
>>> > >> >>>>>
>>> > >> >>>>> - Wei
>>> > >> >>>>>
>>> > >> >>>>>
>>> > >> >>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>> > >> >>>>>
>>> > >> >>>>>> Mike,
>>> > >> >>>>>>
>>> > >> >>>>>> The things I want to understand are the following:
>>> > >> >>>>>>
>>> > >> >>>>>> 1) Is there value in capturing IOPS policies be captured in
>>> a
>>> > >> >> common
>>> > >> >>>>>> data model (e.g. for billing/usage purposes, expressing
>>> > offerings).
>>> > >> >>>>>> 2) Should there be a common interface model for reasoning
>>> about
>>> > >> >> IOP
>>> > >> >>>>>> provisioning at runtime?
>>> > >> >>>>>> 3) How are conflicting provisioned IOPS configurations
>>> between
>>> > a
>>> > >> >>>>>> hypervisor and storage device reconciled? In particular, a
>>> > >> scenario
>>> > >> >>>> where
>>> > >> >>>>>> is lead to believe (and billed) for more IOPS configured for
>>> a VM
>>> > >> >>> than a
>>> > >> >>>>>> storage device has been configured to deliver. Another
>>> scenario
>>> > >> >>> could a
>>> > >> >>>>>> consistent configuration between a VM and a storage device at
>>> > >> >> creation
>>> > >> >>>>>> time, but a later modification to storage device introduces
>>> > logical
>>> > >> >>>>>> inconsistency.
>>> > >> >>>>>>
>>> > >> >>>>>> Thanks,
>>> > >> >>>>>> -John
>>> > >> >>>>>>
>>> > >> >>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>> > >> >>>> mike.tutkowski@solidfire.com>
>>> > >> >>>>>> wrote:
>>> > >> >>>>>>
>>> > >> >>>>>> Hi John,
>>> > >> >>>>>>
>>> > >> >>>>>> I believe Wei's feature deals with controlling the max
>>> number of
>>> > >> >> IOPS
>>> > >> >>>> from
>>> > >> >>>>>> the hypervisor side.
>>> > >> >>>>>>
>>> > >> >>>>>> My feature is focused on controlling IOPS from the storage
>>> system
>>> > >> >>> side.
>>> > >> >>>>>>
>>> > >> >>>>>> I hope that helps. :)
>>> > >> >>>>>>
>>> > >> >>>>>>
>>> > >> >>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>>> jburwell@basho.com
>>> > >
>>> > >> >>>> wrote:
>>> > >> >>>>>>
>>> > >> >>>>>>> Wei,
>>> > >> >>>>>>>
>>> > >> >>>>>>> My opinion is that no features should be merged until all
>>> > >> >> functional
>>> > >> >>>>>>> issues have been resolved and it is ready to turn over to
>>> test.
>>> > >> >>> Until
>>> > >> >>>>>> the
>>> > >> >>>>>>> total Ops vs discrete read/write ops issue is addressed and
>>> > >> >>> re-reviewed
>>> > >> >>>>>> by
>>> > >> >>>>>>> Wido, I don't think this criteria has been satisfied.
>>> > >> >>>>>>>
>>> > >> >>>>>>> Also, how does this work intersect/compliment the SolidFire
>>> > patch
>>> > >> (
>>> > >> >>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
>>> that
>>> > >> >> work
>>> > >> >>> is
>>> > >> >>>>>>> also involves provisioned IOPS. I would like to ensure we
>>> don't
>>> > >> >>> have a
>>> > >> >>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>>> > >> >>> unnecessarily
>>> > >> >>>>>>> incompatible.
>>> > >> >>>>>>>
>>> > >> >>>>>>> Thanks,
>>> > >> >>>>>>> -John
>>> > >> >>>>>>>
>>> > >> >>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <ustcweizhou@gmail.com
>>> >
>>> > >> >> wrote:
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wido,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Sure. I will change it next week.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> -Wei
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Hi Wei,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wido,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Exactly. I have pushed the features into master.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> If anyone object thems for technical reason till Monday, I
>>> will
>>> > >> >>> revert
>>> > >> >>>>>>>
>>> > >> >>>>>>> them.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> For the sake of clarity I just want to mention again that we
>>> > >> should
>>> > >> >>>>>> change
>>> > >> >>>>>>>
>>> > >> >>>>>>> the total IOps to R/W IOps asap so that we never release a
>>> > version
>>> > >> >>> with
>>> > >> >>>>>>>
>>> > >> >>>>>>> only total IOps.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> You laid the groundwork for the I/O throttling and that's
>>> great!
>>> > >> We
>>> > >> >>>>>> should
>>> > >> >>>>>>>
>>> > >> >>>>>>> however prevent that we create legacy from day #1.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wido
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> -Wei
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wido,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> +1 -- this enhancement must to discretely support read and
>>> write
>>> > >> >>> IOPS.
>>> > >> >>>>>>>
>>> > >> >>>>>>> I
>>> > >> >>>>>>>
>>> > >> >>>>>>> don't see how it could be fixed later because I don't see
>>> how we
>>> > >> >>>>>>>
>>> > >> >>>>>>> correctly
>>> > >> >>>>>>>
>>> > >> >>>>>>> split total IOPS into read and write. Therefore, we would
>>> be
>>> > >> stuck
>>> > >> >>>>>>>
>>> > >> >>>>>>> with a
>>> > >> >>>>>>>
>>> > >> >>>>>>> total unless/until we decided to break backwards
>>> compatibility.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> What Wei meant was merging it into master now so that it
>>> will go
>>> > >> in
>>> > >> >>> the
>>> > >> >>>>>>>
>>> > >> >>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release
>>> so
>>> > >> that
>>> > >> >>> 4.2
>>> > >> >>>>>>>
>>> > >> >>>>>>> will be released with Read and Write instead of Total IOps.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> This is to make the May 31st feature freeze date. But if the
>>> > >> window
>>> > >> >>>> moves
>>> > >> >>>>>>>
>>> > >> >>>>>>> (see other threads) then it won't be necessary to do that.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wido
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> I also completely agree that there is no association between
>>> > >> >> network
>>> > >> >>>>>>>
>>> > >> >>>>>>> and
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> disk I/O.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Thanks,
>>> > >> >>>>>>>
>>> > >> >>>>>>> -John
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>>> wido@widodh.nl
>>> > >
>>> > >> >>>> wrote:
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Hi Wei,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Hi Wido,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Thanks. Good question.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> I thought about at the beginning. Finally I decided to
>>> ignore
>>> > the
>>> > >> >>>>>>>
>>> > >> >>>>>>> difference of read and write mainly because the network
>>> > throttling
>>> > >> >>> did
>>> > >> >>>>>>>
>>> > >> >>>>>>> not
>>> > >> >>>>>>>
>>> > >> >>>>>>> care the difference of sent and received bytes as well.
>>> > >> >>>>>>>
>>> > >> >>>>>>> That reasoning seems odd. Networking and disk I/O completely
>>> > >> >>> different.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Disk I/O is much more expensive in most situations then
>>> network
>>> > >> >>>>>>>
>>> > >> >>>>>>> bandwith.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Implementing it will be some copy-paste work. It could be
>>> > >> >>>>>>>
>>> > >> >>>>>>> implemented in
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> few days. For the deadline of feature freeze, I will
>>> implement
>>> > it
>>> > >> >>>>>>>
>>> > >> >>>>>>> after
>>> > >> >>>>>>>
>>> > >> >>>>>>> that , if needed.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> It think it's a feature we can't miss. But if it goes into
>>> the
>>> > 4.2
>>> > >> >>>>>>>
>>> > >> >>>>>>> window we have to make sure we don't release with only total
>>> > IOps
>>> > >> >> and
>>> > >> >>>>>>>
>>> > >> >>>>>>> fix
>>> > >> >>>>>>>
>>> > >> >>>>>>> it in 4.3, that will confuse users.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wido
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> -Wei
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Hi Wei,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Hi,
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> I would like to merge disk_io_throttling branch into master.
>>> > >> >>>>>>>
>>> > >> >>>>>>> If nobody object, I will merge into master in 48 hours.
>>> > >> >>>>>>>
>>> > >> >>>>>>> The purpose is :
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Virtual machines are running on the same storage device
>>> (local
>>> > >> >>>>>>>
>>> > >> >>>>>>> storage or
>>> > >> >>>>>>>
>>> > >> >>>>>>> share strage). Because of the rate limitation of device
>>> (such as
>>> > >> >>>>>>>
>>> > >> >>>>>>> iops), if
>>> > >> >>>>>>>
>>> > >> >>>>>>> one VM has large disk operation, it may affect the disk
>>> > >> performance
>>> > >> >>>>>>>
>>> > >> >>>>>>> of
>>> > >> >>>>>>>
>>> > >> >>>>>>> other VMs running on the same storage device.
>>> > >> >>>>>>>
>>> > >> >>>>>>> It is neccesary to set the maximum rate and limit the disk
>>> I/O
>>> > of
>>> > >> >>>>>>>
>>> > >> >>>>>>> VMs.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Looking at the code I see you make no difference between
>>> Read
>>> > and
>>> > >> >>>>>>>
>>> > >> >>>>>>> Write
>>> > >> >>>>>>>
>>> > >> >>>>>>> IOps.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Qemu and libvirt support setting both a different rate for
>>> Read
>>> > >> and
>>> > >> >>>>>>>
>>> > >> >>>>>>> Write
>>> > >> >>>>>>>
>>> > >> >>>>>>> IOps which could benefit a lot of users.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> It's also strange, in the polling side you collect both the
>>> Read
>>> > >> >> and
>>> > >> >>>>>>>
>>> > >> >>>>>>> Write
>>> > >> >>>>>>>
>>> > >> >>>>>>> IOps, but on the throttling side you only go for a global
>>> value.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Write IOps are usually much more expensive then Read IOps,
>>> so it
>>> > >> >>>>>>>
>>> > >> >>>>>>> seems
>>> > >> >>>>>>>
>>> > >> >>>>>>> like a valid use-case where that an admin would set a lower
>>> > value
>>> > >> >> for
>>> > >> >>>>>>>
>>> > >> >>>>>>> write
>>> > >> >>>>>>>
>>> > >> >>>>>>> IOps vs Read IOps.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Since this only supports KVM at this point I think it would
>>> be
>>> > of
>>> > >> >>>>>>>
>>> > >> >>>>>>> great
>>> > >> >>>>>>>
>>> > >> >>>>>>> value to at least have the mechanism in place to support
>>> both,
>>> > >> >>>>>>>
>>> > >> >>>>>>> implementing
>>> > >> >>>>>>>
>>> > >> >>>>>>> this later would be a lot of work.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> If a hypervisor doesn't support setting different values for
>>> > read
>>> > >> >> and
>>> > >> >>>>>>>
>>> > >> >>>>>>> write you can always sum both up and set that as the total
>>> > limit.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Can you explain why you implemented it this way?
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wido
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> The feature includes:
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> (1) set the maximum rate of VMs (in disk_offering, and
>>> global
>>> > >> >>>>>>>
>>> > >> >>>>>>> configuration)
>>> > >> >>>>>>>
>>> > >> >>>>>>> (2) change the maximum rate of VMs
>>> > >> >>>>>>>
>>> > >> >>>>>>> (3) limit the disk rate (total bps and iops)
>>> > >> >>>>>>>
>>> > >> >>>>>>> JIRA ticket: https://issues.apache.org/****
>>> > >> >>>>>>>
>>> > >> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
>>> issues.apache.org/****
>>> > >> >>>>>>>
>>> > >> >>>>>>> jira/browse/CLOUDSTACK-1192<
>>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <ht**tps://
>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>> > >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <**
>>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>> > >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> FS (I will update later) :
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>> > >> >>>>>>
>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <
>>> > >> >>>>>>>
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> > <
>>> > >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>> > >> >>>>>>>
>>> > >> >>>>>>> VM+Disk+IO+Throttling<https://****
>>> > >> cwiki.apache.org/confluence/****
>>> > >> >> <
>>> > >> >>>>>>> http://cwiki.apache.org/confluence/**>
>>> > >> >>>>>>>
>>> > >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>>> **
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> > >> >> <
>>> > >> >>>>>>>
>>> > >> >>>>>>
>>> > >> >>>>
>>> > >> >>>
>>> > >> >>
>>> > >>
>>> >
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> > >> >>>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Merge check list :-
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> * Did you check the branch's RAT execution success?
>>> > >> >>>>>>>
>>> > >> >>>>>>> Yes
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> * Are there new dependencies introduced?
>>> > >> >>>>>>>
>>> > >> >>>>>>> No
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> * What automated testing (unit and integration) is included
>>> in
>>> > the
>>> > >> >>>>>>>
>>> > >> >>>>>>> new
>>> > >> >>>>>>>
>>> > >> >>>>>>> feature?
>>> > >> >>>>>>>
>>> > >> >>>>>>> Unit tests are added.
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> * What testing has been done to check for potential
>>> regressions?
>>> > >> >>>>>>>
>>> > >> >>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>> > >> >>>>>>>
>>> > >> >>>>>>> (2) VM operations, including
>>> > >> >>>>>>>
>>> > >> >>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>>> restore
>>> > >> >>>>>>>
>>> > >> >>>>>>> (3) Volume operations, including
>>> > >> >>>>>>>
>>> > >> >>>>>>> Attach, Detach
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> To review the code, you can try
>>> > >> >>>>>>>
>>> > >> >>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>> > >> >>>>>>>
>>> > >> >>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> Best regards,
>>> > >> >>>>>>>
>>> > >> >>>>>>> Wei
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> [1]
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>> > >> >>>>>>
>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <
>>> > >> >>>>>>>
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>>> > <
>>> > >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>> > >> >>>>>>>
>>> > >> >>>>>>> VM+Disk+IO+Throttling<https://****
>>> > >> cwiki.apache.org/confluence/****
>>> > >> >> <
>>> > >> >>>>>>> http://cwiki.apache.org/confluence/**>
>>> > >> >>>>>>>
>>> > >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>>> **
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>>> > >> >> <
>>> > >> >>>>>>>
>>> > >> >>>>>>
>>> > >> >>>>
>>> > >> >>>
>>> > >> >>
>>> > >>
>>> >
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> > >> >>>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> [2] refs/heads/disk_io_throttling
>>> > >> >>>>>>>
>>> > >> >>>>>>> [3]
>>> > >> >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
>>> <
>>> > >> >>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <ht**tps://
>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <ht**tps://
>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>> > >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <**
>>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>> > >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <ht**tps://
>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>> > >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>> > >> >>>>>>>
>>> > >> >>>>>>> **<
>>> > >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>>> > >> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <**
>>> > >> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>> > >> >>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>> > >> >>>>>>>
>>> > >> >>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>> > >> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>> (**CLOUDSTACK-1301
>>> > >> >>>>>>>
>>> > >> >>>>>>> - VM Disk I/O Throttling)
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>>
>>> > >> >>>>>>
>>> > >> >>>>>>
>>> > >> >>>>>> --
>>> > >> >>>>>> *Mike Tutkowski*
>>> > >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >> >>>>>> e: mike.tutkowski@solidfire.com
>>> > >> >>>>>> o: 303.746.7302
>>> > >> >>>>>> Advancing the way the world uses the
>>> > >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >> >>>>>> *â„¢*
>>> > >> >>>>>>
>>> > >> >>>>
>>> > >> >>>>
>>> > >> >>>
>>> > >> >>
>>> > >> >>
>>> > >> >>
>>> > >> >> --
>>> > >> >> *Mike Tutkowski*
>>> > >> >> *Senior CloudStack Developer, SolidFire Inc.*
>>> > >> >> e: mike.tutkowski@solidfire.com
>>> > >> >> o: 303.746.7302
>>> > >> >> Advancing the way the world uses the
>>> > >> >> cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >> >> *â„¢*
>>> > >> >>
>>> > >> >
>>> > >> >
>>> > >> >
>>> > >> > --
>>> > >> > *Mike Tutkowski*
>>> > >> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > >> > e: mike.tutkowski@solidfire.com
>>> > >> > o: 303.746.7302
>>> > >> > Advancing the way the world uses the
>>> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > >> > *â„¢*
>>> > >>
>>> > >>
>>> > >
>>> > >
>>> > > --
>>> > > *Mike Tutkowski*
>>> > > *Senior CloudStack Developer, SolidFire Inc.*
>>> > > e: mike.tutkowski@solidfire.com
>>> > > o: 303.746.7302
>>> > > Advancing the way the world uses the cloud<
>>> > http://solidfire.com/solution/overview/?video=play>
>>> > > *â„¢*
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > *Mike Tutkowski*
>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> > e: mike.tutkowski@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the
>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> > *â„¢*
>>> >
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
For example, let's say another storage company wants to implement a plug-in
to leverage its Quality of Service feature. It would be dynamic, zone-wide
storage, as well. They would need only implement a storage plug-in as I've
made the necessary changes to the hypervisor-attach logic to support their
plug-in.
On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> Oh, sorry to imply the XenServer code is SolidFire specific. It is not.
>
> The XenServer attach logic is now aware of dynamic, zone-wide storage (and
> SolidFire is an implementation of this kind of storage). This kind of
> storage is new to 4.2 with Edison's storage framework changes.
>
> Edison created a new framework that supported the creation and deletion of
> volumes dynamically. However, when I visited with him in Portland back in
> April, we realized that it was not complete. We realized there was nothing
> CloudStack could do with these volumes unless the attach logic was changed
> to recognize this new type of storage and create the appropriate hypervisor
> data structure.
>
>
> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> It is generally odd to me that any operation in the Storage layer would
>> understand or care about details. I expect to see the Storage services
>> expose a set of operations that can be composed/driven by the Hypervisor
>> implementations to allocate space/create structures per their needs. If
>> we
>> don't invert this dependency, we are going to end with a massive n-to-n
>> problem that will make the system increasingly difficult to maintain and
>> enhance. Am I understanding that the Xen specific SolidFire code is
>> located in the CitrixResourceBase class?
>>
>> Thanks,
>> -John
>>
>>
>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>> > wrote:
>>
>> > To delve into this in a bit more detail:
>> >
>> > Prior to 4.2 and aside from one setup method for XenServer, the admin
>> had
>> > to first create a volume on the storage system, then go into the
>> hypervisor
>> > to set up a data structure to make use of the volume (ex. a storage
>> > repository on XenServer or a datastore on ESX(i)). VMs and data disks
>> then
>> > shared this storage system's volume.
>> >
>> > With Edison's new storage framework, storage need no longer be so static
>> > and you can easily create a 1:1 relationship between a storage system's
>> > volume and the VM's data disk (necessary for storage Quality of
>> Service).
>> >
>> > You can now write a plug-in that is called to dynamically create and
>> delete
>> > volumes as needed.
>> >
>> > The problem that the storage framework did not address is in creating
>> and
>> > deleting the hypervisor-specific data structure when performing an
>> > attach/detach.
>> >
>> > That being the case, I've been enhancing it to do so. I've got XenServer
>> > worked out and submitted. I've got ESX(i) in my sandbox and can submit
>> this
>> > if we extend the 4.2 freeze date.
>> >
>> > Does that help a bit? :)
>> >
>> >
>> > On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com
>> > > wrote:
>> >
>> > > Hi John,
>> > >
>> > > The storage plug-in - by itself - is hypervisor agnostic.
>> > >
>> > > The issue is with the volume-attach logic (in the agent code). The
>> > storage
>> > > framework calls into the plug-in to have it create a volume as needed,
>> > but
>> > > when the time comes to attach the volume to a hypervisor, the attach
>> > logic
>> > > has to be smart enough to recognize it's being invoked on zone-wide
>> > storage
>> > > (where the volume has just been created) and create, say, a storage
>> > > repository (for XenServer) or a datastore (for VMware) to make use of
>> the
>> > > volume that was just created.
>> > >
>> > > I've been spending most of my time recently making the attach logic
>> work
>> > > in the agent code.
>> > >
>> > > Does that clear it up?
>> > >
>> > > Thanks!
>> > >
>> > >
>> > > On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
>> > wrote:
>> > >
>> > >> Mike,
>> > >>
>> > >> Can you explain why the the storage driver is hypervisor specific?
>> > >>
>> > >> Thanks,
>> > >> -John
>> > >>
>> > >> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
>> > mike.tutkowski@solidfire.com>
>> > >> wrote:
>> > >>
>> > >> > Yes, ultimately I would like to support all hypervisors that
>> > CloudStack
>> > >> > supports. I think I'm just out of time for 4.2 to get KVM in.
>> > >> >
>> > >> > Right now this plug-in supports XenServer. Depending on what we do
>> > with
>> > >> > regards to 4.2 feature freeze, I have it working for VMware in my
>> > >> sandbox,
>> > >> > as well.
>> > >> >
>> > >> > Also, just to be clear, this is all in regards to Disk Offerings. I
>> > >> plan to
>> > >> > support Compute Offerings post 4.2.
>> > >> >
>> > >> >
>> > >> > On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
>> > kelcey@bbits.ca
>> > >> >wrote:
>> > >> >
>> > >> >> Is there any plan on supporting KVM in the patch cycle post 4.2?
>> > >> >>
>> > >> >> ----- Original Message -----
>> > >> >> From: "Mike Tutkowski" <mi...@solidfire.com>
>> > >> >> To: dev@cloudstack.apache.org
>> > >> >> Sent: Monday, June 3, 2013 10:12:32 AM
>> > >> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> > >> >>
>> > >> >> I agree on merging Wei's feature first, then mine.
>> > >> >>
>> > >> >> If his feature is for KVM only, then it is a non issue as I don't
>> > >> support
>> > >> >> KVM in 4.2.
>> > >> >>
>> > >> >>
>> > >> >> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
>> > >> wrote:
>> > >> >>
>> > >> >>> John,
>> > >> >>>
>> > >> >>> For the billing, as no one works on billing now, users need to
>> > >> calculate
>> > >> >>> the billing by themselves. They can get the service_offering and
>> > >> >>> disk_offering of a VMs and volumes for calculation. Of course it
>> is
>> > >> >> better
>> > >> >>> to tell user the exact limitation value of individual volume, and
>> > >> network
>> > >> >>> rate limitation for nics as well. I can work on it later. Do you
>> > >> think it
>> > >> >>> is a part of I/O throttling?
>> > >> >>>
>> > >> >>> Sorry my misunstand the second the question.
>> > >> >>>
>> > >> >>> Agree with what you said about the two features.
>> > >> >>>
>> > >> >>> -Wei
>> > >> >>>
>> > >> >>>
>> > >> >>> 2013/6/3 John Burwell <jb...@basho.com>
>> > >> >>>
>> > >> >>>> Wei,
>> > >> >>>>
>> > >> >>>>
>> > >> >>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
>> > wrote:
>> > >> >>>>
>> > >> >>>>> Hi John, Mike
>> > >> >>>>>
>> > >> >>>>> I hope Mike's aswer helps you. I am trying to adding more.
>> > >> >>>>>
>> > >> >>>>> (1) I think billing should depend on IO statistics rather than
>> > IOPS
>> > >> >>>>> limitation. Please review disk_io_stat if you have time.
>> > >> >> disk_io_stat
>> > >> >>>> can
>> > >> >>>>> get the IO statistics including bytes/iops read/write for an
>> > >> >> individual
>> > >> >>>>> virtual machine.
>> > >> >>>>
>> > >> >>>> Going by the AWS model, customers are billed more for volumes
>> with
>> > >> >>>> provisioned IOPS, as well as, for those operations (
>> > >> >>>> http://aws.amazon.com/ebs/). I would imagine our users would
>> like
>> > >> the
>> > >> >>>> option to employ similar cost models. Could an operator
>> implement
>> > >> >> such a
>> > >> >>>> billing model in the current patch?
>> > >> >>>>
>> > >> >>>>>
>> > >> >>>>> (2) Do you mean IOPS runtime change? KVM supports setting
>> IOPS/BPS
>> > >> >>>>> limitation for a running virtual machine through command line.
>> > >> >> However,
>> > >> >>>>> CloudStack does not support changing the parameters of a
>> created
>> > >> >>> offering
>> > >> >>>>> (computer offering or disk offering).
>> > >> >>>>
>> > >> >>>> I meant at the Java interface level. I apologize for being
>> > unclear.
>> > >> >> Can
>> > >> >>>> we more generalize allocation algorithms with a set of
>> interfaces
>> > >> that
>> > >> >>>> describe the service guarantees provided by a resource?
>> > >> >>>>
>> > >> >>>>>
>> > >> >>>>> (3) It is a good question. Maybe it is better to commit Mike's
>> > patch
>> > >> >>>> after
>> > >> >>>>> disk_io_throttling as Mike needs to consider the limitation in
>> > >> >>> hypervisor
>> > >> >>>>> type, I think.
>> > >> >>>>
>> > >> >>>> I will expand on my thoughts in a later response to Mike
>> regarding
>> > >> the
>> > >> >>>> touch points between these two features. I think that
>> > >> >> disk_io_throttling
>> > >> >>>> will need to be merged before SolidFire, but I think we need
>> closer
>> > >> >>>> coordination between the branches (possibly have have solidfire
>> > track
>> > >> >>>> disk_io_throttling) to coordinate on this issue.
>> > >> >>>>
>> > >> >>>>>
>> > >> >>>>> - Wei
>> > >> >>>>>
>> > >> >>>>>
>> > >> >>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> > >> >>>>>
>> > >> >>>>>> Mike,
>> > >> >>>>>>
>> > >> >>>>>> The things I want to understand are the following:
>> > >> >>>>>>
>> > >> >>>>>> 1) Is there value in capturing IOPS policies be captured in a
>> > >> >> common
>> > >> >>>>>> data model (e.g. for billing/usage purposes, expressing
>> > offerings).
>> > >> >>>>>> 2) Should there be a common interface model for reasoning
>> about
>> > >> >> IOP
>> > >> >>>>>> provisioning at runtime?
>> > >> >>>>>> 3) How are conflicting provisioned IOPS configurations
>> between
>> > a
>> > >> >>>>>> hypervisor and storage device reconciled? In particular, a
>> > >> scenario
>> > >> >>>> where
>> > >> >>>>>> is lead to believe (and billed) for more IOPS configured for
>> a VM
>> > >> >>> than a
>> > >> >>>>>> storage device has been configured to deliver. Another
>> scenario
>> > >> >>> could a
>> > >> >>>>>> consistent configuration between a VM and a storage device at
>> > >> >> creation
>> > >> >>>>>> time, but a later modification to storage device introduces
>> > logical
>> > >> >>>>>> inconsistency.
>> > >> >>>>>>
>> > >> >>>>>> Thanks,
>> > >> >>>>>> -John
>> > >> >>>>>>
>> > >> >>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> > >> >>>> mike.tutkowski@solidfire.com>
>> > >> >>>>>> wrote:
>> > >> >>>>>>
>> > >> >>>>>> Hi John,
>> > >> >>>>>>
>> > >> >>>>>> I believe Wei's feature deals with controlling the max number
>> of
>> > >> >> IOPS
>> > >> >>>> from
>> > >> >>>>>> the hypervisor side.
>> > >> >>>>>>
>> > >> >>>>>> My feature is focused on controlling IOPS from the storage
>> system
>> > >> >>> side.
>> > >> >>>>>>
>> > >> >>>>>> I hope that helps. :)
>> > >> >>>>>>
>> > >> >>>>>>
>> > >> >>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
>> jburwell@basho.com
>> > >
>> > >> >>>> wrote:
>> > >> >>>>>>
>> > >> >>>>>>> Wei,
>> > >> >>>>>>>
>> > >> >>>>>>> My opinion is that no features should be merged until all
>> > >> >> functional
>> > >> >>>>>>> issues have been resolved and it is ready to turn over to
>> test.
>> > >> >>> Until
>> > >> >>>>>> the
>> > >> >>>>>>> total Ops vs discrete read/write ops issue is addressed and
>> > >> >>> re-reviewed
>> > >> >>>>>> by
>> > >> >>>>>>> Wido, I don't think this criteria has been satisfied.
>> > >> >>>>>>>
>> > >> >>>>>>> Also, how does this work intersect/compliment the SolidFire
>> > patch
>> > >> (
>> > >> >>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
>> that
>> > >> >> work
>> > >> >>> is
>> > >> >>>>>>> also involves provisioned IOPS. I would like to ensure we
>> don't
>> > >> >>> have a
>> > >> >>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>> > >> >>> unnecessarily
>> > >> >>>>>>> incompatible.
>> > >> >>>>>>>
>> > >> >>>>>>> Thanks,
>> > >> >>>>>>> -John
>> > >> >>>>>>>
>> > >> >>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com>
>> > >> >> wrote:
>> > >> >>>>>>>
>> > >> >>>>>>> Wido,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Sure. I will change it next week.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> -Wei
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Hi Wei,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Wido,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Exactly. I have pushed the features into master.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> If anyone object thems for technical reason till Monday, I
>> will
>> > >> >>> revert
>> > >> >>>>>>>
>> > >> >>>>>>> them.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> For the sake of clarity I just want to mention again that we
>> > >> should
>> > >> >>>>>> change
>> > >> >>>>>>>
>> > >> >>>>>>> the total IOps to R/W IOps asap so that we never release a
>> > version
>> > >> >>> with
>> > >> >>>>>>>
>> > >> >>>>>>> only total IOps.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> You laid the groundwork for the I/O throttling and that's
>> great!
>> > >> We
>> > >> >>>>>> should
>> > >> >>>>>>>
>> > >> >>>>>>> however prevent that we create legacy from day #1.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Wido
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> -Wei
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Wido,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> +1 -- this enhancement must to discretely support read and
>> write
>> > >> >>> IOPS.
>> > >> >>>>>>>
>> > >> >>>>>>> I
>> > >> >>>>>>>
>> > >> >>>>>>> don't see how it could be fixed later because I don't see
>> how we
>> > >> >>>>>>>
>> > >> >>>>>>> correctly
>> > >> >>>>>>>
>> > >> >>>>>>> split total IOPS into read and write. Therefore, we would be
>> > >> stuck
>> > >> >>>>>>>
>> > >> >>>>>>> with a
>> > >> >>>>>>>
>> > >> >>>>>>> total unless/until we decided to break backwards
>> compatibility.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> What Wei meant was merging it into master now so that it
>> will go
>> > >> in
>> > >> >>> the
>> > >> >>>>>>>
>> > >> >>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release
>> so
>> > >> that
>> > >> >>> 4.2
>> > >> >>>>>>>
>> > >> >>>>>>> will be released with Read and Write instead of Total IOps.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> This is to make the May 31st feature freeze date. But if the
>> > >> window
>> > >> >>>> moves
>> > >> >>>>>>>
>> > >> >>>>>>> (see other threads) then it won't be necessary to do that.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Wido
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> I also completely agree that there is no association between
>> > >> >> network
>> > >> >>>>>>>
>> > >> >>>>>>> and
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> disk I/O.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Thanks,
>> > >> >>>>>>>
>> > >> >>>>>>> -John
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
>> wido@widodh.nl
>> > >
>> > >> >>>> wrote:
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Hi Wei,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Hi Wido,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Thanks. Good question.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> I thought about at the beginning. Finally I decided to
>> ignore
>> > the
>> > >> >>>>>>>
>> > >> >>>>>>> difference of read and write mainly because the network
>> > throttling
>> > >> >>> did
>> > >> >>>>>>>
>> > >> >>>>>>> not
>> > >> >>>>>>>
>> > >> >>>>>>> care the difference of sent and received bytes as well.
>> > >> >>>>>>>
>> > >> >>>>>>> That reasoning seems odd. Networking and disk I/O completely
>> > >> >>> different.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Disk I/O is much more expensive in most situations then
>> network
>> > >> >>>>>>>
>> > >> >>>>>>> bandwith.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Implementing it will be some copy-paste work. It could be
>> > >> >>>>>>>
>> > >> >>>>>>> implemented in
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> few days. For the deadline of feature freeze, I will
>> implement
>> > it
>> > >> >>>>>>>
>> > >> >>>>>>> after
>> > >> >>>>>>>
>> > >> >>>>>>> that , if needed.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> It think it's a feature we can't miss. But if it goes into
>> the
>> > 4.2
>> > >> >>>>>>>
>> > >> >>>>>>> window we have to make sure we don't release with only total
>> > IOps
>> > >> >> and
>> > >> >>>>>>>
>> > >> >>>>>>> fix
>> > >> >>>>>>>
>> > >> >>>>>>> it in 4.3, that will confuse users.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Wido
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> -Wei
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Hi Wei,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Hi,
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> I would like to merge disk_io_throttling branch into master.
>> > >> >>>>>>>
>> > >> >>>>>>> If nobody object, I will merge into master in 48 hours.
>> > >> >>>>>>>
>> > >> >>>>>>> The purpose is :
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Virtual machines are running on the same storage device
>> (local
>> > >> >>>>>>>
>> > >> >>>>>>> storage or
>> > >> >>>>>>>
>> > >> >>>>>>> share strage). Because of the rate limitation of device
>> (such as
>> > >> >>>>>>>
>> > >> >>>>>>> iops), if
>> > >> >>>>>>>
>> > >> >>>>>>> one VM has large disk operation, it may affect the disk
>> > >> performance
>> > >> >>>>>>>
>> > >> >>>>>>> of
>> > >> >>>>>>>
>> > >> >>>>>>> other VMs running on the same storage device.
>> > >> >>>>>>>
>> > >> >>>>>>> It is neccesary to set the maximum rate and limit the disk
>> I/O
>> > of
>> > >> >>>>>>>
>> > >> >>>>>>> VMs.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Looking at the code I see you make no difference between Read
>> > and
>> > >> >>>>>>>
>> > >> >>>>>>> Write
>> > >> >>>>>>>
>> > >> >>>>>>> IOps.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Qemu and libvirt support setting both a different rate for
>> Read
>> > >> and
>> > >> >>>>>>>
>> > >> >>>>>>> Write
>> > >> >>>>>>>
>> > >> >>>>>>> IOps which could benefit a lot of users.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> It's also strange, in the polling side you collect both the
>> Read
>> > >> >> and
>> > >> >>>>>>>
>> > >> >>>>>>> Write
>> > >> >>>>>>>
>> > >> >>>>>>> IOps, but on the throttling side you only go for a global
>> value.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Write IOps are usually much more expensive then Read IOps,
>> so it
>> > >> >>>>>>>
>> > >> >>>>>>> seems
>> > >> >>>>>>>
>> > >> >>>>>>> like a valid use-case where that an admin would set a lower
>> > value
>> > >> >> for
>> > >> >>>>>>>
>> > >> >>>>>>> write
>> > >> >>>>>>>
>> > >> >>>>>>> IOps vs Read IOps.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Since this only supports KVM at this point I think it would
>> be
>> > of
>> > >> >>>>>>>
>> > >> >>>>>>> great
>> > >> >>>>>>>
>> > >> >>>>>>> value to at least have the mechanism in place to support
>> both,
>> > >> >>>>>>>
>> > >> >>>>>>> implementing
>> > >> >>>>>>>
>> > >> >>>>>>> this later would be a lot of work.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> If a hypervisor doesn't support setting different values for
>> > read
>> > >> >> and
>> > >> >>>>>>>
>> > >> >>>>>>> write you can always sum both up and set that as the total
>> > limit.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Can you explain why you implemented it this way?
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Wido
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> The feature includes:
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>> > >> >>>>>>>
>> > >> >>>>>>> configuration)
>> > >> >>>>>>>
>> > >> >>>>>>> (2) change the maximum rate of VMs
>> > >> >>>>>>>
>> > >> >>>>>>> (3) limit the disk rate (total bps and iops)
>> > >> >>>>>>>
>> > >> >>>>>>> JIRA ticket: https://issues.apache.org/****
>> > >> >>>>>>>
>> > >> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>> > >> >>>>>>>
>> > >> >>>>>>> jira/browse/CLOUDSTACK-1192<
>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> > >> >>>>>>>
>> > >> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192
>> <
>> > >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> > >> >>>>>>>
>> > >> >>>>>>> <**
>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> > >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> FS (I will update later) :
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> > >> >>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> > >> >>>>>>>
>> > >> >>>>>>> <
>> > >> >>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > <
>> > >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> > >> >>>>>>>
>> > >> >>>>>>> VM+Disk+IO+Throttling<https://****
>> > >> cwiki.apache.org/confluence/****
>> > >> >> <
>> > >> >>>>>>> http://cwiki.apache.org/confluence/**>
>> > >> >>>>>>>
>> > >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>> **
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > >> >> <
>> > >> >>>>>>>
>> > >> >>>>>>
>> > >> >>>>
>> > >> >>>
>> > >> >>
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> > >> >>>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Merge check list :-
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> * Did you check the branch's RAT execution success?
>> > >> >>>>>>>
>> > >> >>>>>>> Yes
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> * Are there new dependencies introduced?
>> > >> >>>>>>>
>> > >> >>>>>>> No
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> * What automated testing (unit and integration) is included
>> in
>> > the
>> > >> >>>>>>>
>> > >> >>>>>>> new
>> > >> >>>>>>>
>> > >> >>>>>>> feature?
>> > >> >>>>>>>
>> > >> >>>>>>> Unit tests are added.
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> * What testing has been done to check for potential
>> regressions?
>> > >> >>>>>>>
>> > >> >>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> > >> >>>>>>>
>> > >> >>>>>>> (2) VM operations, including
>> > >> >>>>>>>
>> > >> >>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
>> restore
>> > >> >>>>>>>
>> > >> >>>>>>> (3) Volume operations, including
>> > >> >>>>>>>
>> > >> >>>>>>> Attach, Detach
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> To review the code, you can try
>> > >> >>>>>>>
>> > >> >>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>> > >> >>>>>>>
>> > >> >>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> Best regards,
>> > >> >>>>>>>
>> > >> >>>>>>> Wei
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> [1]
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> > >> >>>>>>
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> > >> >>>>>>>
>> > >> >>>>>>> <
>> > >> >>>>>>>
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
>> > <
>> > >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> > >> >>>>>>>
>> > >> >>>>>>> VM+Disk+IO+Throttling<https://****
>> > >> cwiki.apache.org/confluence/****
>> > >> >> <
>> > >> >>>>>>> http://cwiki.apache.org/confluence/**>
>> > >> >>>>>>>
>> > >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.
>> **
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> > >> >> <
>> > >> >>>>>>>
>> > >> >>>>>>
>> > >> >>>>
>> > >> >>>
>> > >> >>
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> > >> >>>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> [2] refs/heads/disk_io_throttling
>> > >> >>>>>>>
>> > >> >>>>>>> [3]
>> > >> >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
>> > >> >>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>> > >> >>>>>>>
>> > >> >>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301
>> <
>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> > >> >>>>>>>
>> > >> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301
>> <
>> > >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> > >> >>>>>>>
>> > >> >>>>>>> <**
>> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> > >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> <ht**tps://
>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> > >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>> > >> >>>>>>>
>> > >> >>>>>>> **<
>> > >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>> > >> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> > >> >>>>>>>
>> > >> >>>>>>> <**
>> > >> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> > >> >>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> > >> >>>>>>>
>> > >> >>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> > >> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> (**CLOUDSTACK-1301
>> > >> >>>>>>>
>> > >> >>>>>>> - VM Disk I/O Throttling)
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>
>> > >> >>>>>>
>> > >> >>>>>> --
>> > >> >>>>>> *Mike Tutkowski*
>> > >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> > >> >>>>>> e: mike.tutkowski@solidfire.com
>> > >> >>>>>> o: 303.746.7302
>> > >> >>>>>> Advancing the way the world uses the
>> > >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >> >>>>>> *â„¢*
>> > >> >>>>>>
>> > >> >>>>
>> > >> >>>>
>> > >> >>>
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >> --
>> > >> >> *Mike Tutkowski*
>> > >> >> *Senior CloudStack Developer, SolidFire Inc.*
>> > >> >> e: mike.tutkowski@solidfire.com
>> > >> >> o: 303.746.7302
>> > >> >> Advancing the way the world uses the
>> > >> >> cloud<http://solidfire.com/solution/overview/?video=play>
>> > >> >> *â„¢*
>> > >> >>
>> > >> >
>> > >> >
>> > >> >
>> > >> > --
>> > >> > *Mike Tutkowski*
>> > >> > *Senior CloudStack Developer, SolidFire Inc.*
>> > >> > e: mike.tutkowski@solidfire.com
>> > >> > o: 303.746.7302
>> > >> > Advancing the way the world uses the
>> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > >> > *â„¢*
>> > >>
>> > >>
>> > >
>> > >
>> > > --
>> > > *Mike Tutkowski*
>> > > *Senior CloudStack Developer, SolidFire Inc.*
>> > > e: mike.tutkowski@solidfire.com
>> > > o: 303.746.7302
>> > > Advancing the way the world uses the cloud<
>> > http://solidfire.com/solution/overview/?video=play>
>> > > *â„¢*
>> > >
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>> >
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Oh, sorry to imply the XenServer code is SolidFire specific. It is not.
The XenServer attach logic is now aware of dynamic, zone-wide storage (and
SolidFire is an implementation of this kind of storage). This kind of
storage is new to 4.2 with Edison's storage framework changes.
Edison created a new framework that supported the creation and deletion of
volumes dynamically. However, when I visited with him in Portland back in
April, we realized that it was not complete. We realized there was nothing
CloudStack could do with these volumes unless the attach logic was changed
to recognize this new type of storage and create the appropriate hypervisor
data structure.
On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> It is generally odd to me that any operation in the Storage layer would
> understand or care about details. I expect to see the Storage services
> expose a set of operations that can be composed/driven by the Hypervisor
> implementations to allocate space/create structures per their needs. If we
> don't invert this dependency, we are going to end with a massive n-to-n
> problem that will make the system increasingly difficult to maintain and
> enhance. Am I understanding that the Xen specific SolidFire code is
> located in the CitrixResourceBase class?
>
> Thanks,
> -John
>
>
> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> > wrote:
>
> > To delve into this in a bit more detail:
> >
> > Prior to 4.2 and aside from one setup method for XenServer, the admin had
> > to first create a volume on the storage system, then go into the
> hypervisor
> > to set up a data structure to make use of the volume (ex. a storage
> > repository on XenServer or a datastore on ESX(i)). VMs and data disks
> then
> > shared this storage system's volume.
> >
> > With Edison's new storage framework, storage need no longer be so static
> > and you can easily create a 1:1 relationship between a storage system's
> > volume and the VM's data disk (necessary for storage Quality of Service).
> >
> > You can now write a plug-in that is called to dynamically create and
> delete
> > volumes as needed.
> >
> > The problem that the storage framework did not address is in creating and
> > deleting the hypervisor-specific data structure when performing an
> > attach/detach.
> >
> > That being the case, I've been enhancing it to do so. I've got XenServer
> > worked out and submitted. I've got ESX(i) in my sandbox and can submit
> this
> > if we extend the 4.2 freeze date.
> >
> > Does that help a bit? :)
> >
> >
> > On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com
> > > wrote:
> >
> > > Hi John,
> > >
> > > The storage plug-in - by itself - is hypervisor agnostic.
> > >
> > > The issue is with the volume-attach logic (in the agent code). The
> > storage
> > > framework calls into the plug-in to have it create a volume as needed,
> > but
> > > when the time comes to attach the volume to a hypervisor, the attach
> > logic
> > > has to be smart enough to recognize it's being invoked on zone-wide
> > storage
> > > (where the volume has just been created) and create, say, a storage
> > > repository (for XenServer) or a datastore (for VMware) to make use of
> the
> > > volume that was just created.
> > >
> > > I've been spending most of my time recently making the attach logic
> work
> > > in the agent code.
> > >
> > > Does that clear it up?
> > >
> > > Thanks!
> > >
> > >
> > > On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
> > wrote:
> > >
> > >> Mike,
> > >>
> > >> Can you explain why the the storage driver is hypervisor specific?
> > >>
> > >> Thanks,
> > >> -John
> > >>
> > >> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > >> wrote:
> > >>
> > >> > Yes, ultimately I would like to support all hypervisors that
> > CloudStack
> > >> > supports. I think I'm just out of time for 4.2 to get KVM in.
> > >> >
> > >> > Right now this plug-in supports XenServer. Depending on what we do
> > with
> > >> > regards to 4.2 feature freeze, I have it working for VMware in my
> > >> sandbox,
> > >> > as well.
> > >> >
> > >> > Also, just to be clear, this is all in regards to Disk Offerings. I
> > >> plan to
> > >> > support Compute Offerings post 4.2.
> > >> >
> > >> >
> > >> > On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
> > kelcey@bbits.ca
> > >> >wrote:
> > >> >
> > >> >> Is there any plan on supporting KVM in the patch cycle post 4.2?
> > >> >>
> > >> >> ----- Original Message -----
> > >> >> From: "Mike Tutkowski" <mi...@solidfire.com>
> > >> >> To: dev@cloudstack.apache.org
> > >> >> Sent: Monday, June 3, 2013 10:12:32 AM
> > >> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> > >> >>
> > >> >> I agree on merging Wei's feature first, then mine.
> > >> >>
> > >> >> If his feature is for KVM only, then it is a non issue as I don't
> > >> support
> > >> >> KVM in 4.2.
> > >> >>
> > >> >>
> > >> >> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
> > >> wrote:
> > >> >>
> > >> >>> John,
> > >> >>>
> > >> >>> For the billing, as no one works on billing now, users need to
> > >> calculate
> > >> >>> the billing by themselves. They can get the service_offering and
> > >> >>> disk_offering of a VMs and volumes for calculation. Of course it
> is
> > >> >> better
> > >> >>> to tell user the exact limitation value of individual volume, and
> > >> network
> > >> >>> rate limitation for nics as well. I can work on it later. Do you
> > >> think it
> > >> >>> is a part of I/O throttling?
> > >> >>>
> > >> >>> Sorry my misunstand the second the question.
> > >> >>>
> > >> >>> Agree with what you said about the two features.
> > >> >>>
> > >> >>> -Wei
> > >> >>>
> > >> >>>
> > >> >>> 2013/6/3 John Burwell <jb...@basho.com>
> > >> >>>
> > >> >>>> Wei,
> > >> >>>>
> > >> >>>>
> > >> >>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
> > wrote:
> > >> >>>>
> > >> >>>>> Hi John, Mike
> > >> >>>>>
> > >> >>>>> I hope Mike's aswer helps you. I am trying to adding more.
> > >> >>>>>
> > >> >>>>> (1) I think billing should depend on IO statistics rather than
> > IOPS
> > >> >>>>> limitation. Please review disk_io_stat if you have time.
> > >> >> disk_io_stat
> > >> >>>> can
> > >> >>>>> get the IO statistics including bytes/iops read/write for an
> > >> >> individual
> > >> >>>>> virtual machine.
> > >> >>>>
> > >> >>>> Going by the AWS model, customers are billed more for volumes
> with
> > >> >>>> provisioned IOPS, as well as, for those operations (
> > >> >>>> http://aws.amazon.com/ebs/). I would imagine our users would
> like
> > >> the
> > >> >>>> option to employ similar cost models. Could an operator
> implement
> > >> >> such a
> > >> >>>> billing model in the current patch?
> > >> >>>>
> > >> >>>>>
> > >> >>>>> (2) Do you mean IOPS runtime change? KVM supports setting
> IOPS/BPS
> > >> >>>>> limitation for a running virtual machine through command line.
> > >> >> However,
> > >> >>>>> CloudStack does not support changing the parameters of a created
> > >> >>> offering
> > >> >>>>> (computer offering or disk offering).
> > >> >>>>
> > >> >>>> I meant at the Java interface level. I apologize for being
> > unclear.
> > >> >> Can
> > >> >>>> we more generalize allocation algorithms with a set of interfaces
> > >> that
> > >> >>>> describe the service guarantees provided by a resource?
> > >> >>>>
> > >> >>>>>
> > >> >>>>> (3) It is a good question. Maybe it is better to commit Mike's
> > patch
> > >> >>>> after
> > >> >>>>> disk_io_throttling as Mike needs to consider the limitation in
> > >> >>> hypervisor
> > >> >>>>> type, I think.
> > >> >>>>
> > >> >>>> I will expand on my thoughts in a later response to Mike
> regarding
> > >> the
> > >> >>>> touch points between these two features. I think that
> > >> >> disk_io_throttling
> > >> >>>> will need to be merged before SolidFire, but I think we need
> closer
> > >> >>>> coordination between the branches (possibly have have solidfire
> > track
> > >> >>>> disk_io_throttling) to coordinate on this issue.
> > >> >>>>
> > >> >>>>>
> > >> >>>>> - Wei
> > >> >>>>>
> > >> >>>>>
> > >> >>>>> 2013/6/3 John Burwell <jb...@basho.com>
> > >> >>>>>
> > >> >>>>>> Mike,
> > >> >>>>>>
> > >> >>>>>> The things I want to understand are the following:
> > >> >>>>>>
> > >> >>>>>> 1) Is there value in capturing IOPS policies be captured in a
> > >> >> common
> > >> >>>>>> data model (e.g. for billing/usage purposes, expressing
> > offerings).
> > >> >>>>>> 2) Should there be a common interface model for reasoning
> about
> > >> >> IOP
> > >> >>>>>> provisioning at runtime?
> > >> >>>>>> 3) How are conflicting provisioned IOPS configurations
> between
> > a
> > >> >>>>>> hypervisor and storage device reconciled? In particular, a
> > >> scenario
> > >> >>>> where
> > >> >>>>>> is lead to believe (and billed) for more IOPS configured for a
> VM
> > >> >>> than a
> > >> >>>>>> storage device has been configured to deliver. Another
> scenario
> > >> >>> could a
> > >> >>>>>> consistent configuration between a VM and a storage device at
> > >> >> creation
> > >> >>>>>> time, but a later modification to storage device introduces
> > logical
> > >> >>>>>> inconsistency.
> > >> >>>>>>
> > >> >>>>>> Thanks,
> > >> >>>>>> -John
> > >> >>>>>>
> > >> >>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> > >> >>>> mike.tutkowski@solidfire.com>
> > >> >>>>>> wrote:
> > >> >>>>>>
> > >> >>>>>> Hi John,
> > >> >>>>>>
> > >> >>>>>> I believe Wei's feature deals with controlling the max number
> of
> > >> >> IOPS
> > >> >>>> from
> > >> >>>>>> the hypervisor side.
> > >> >>>>>>
> > >> >>>>>> My feature is focused on controlling IOPS from the storage
> system
> > >> >>> side.
> > >> >>>>>>
> > >> >>>>>> I hope that helps. :)
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <
> jburwell@basho.com
> > >
> > >> >>>> wrote:
> > >> >>>>>>
> > >> >>>>>>> Wei,
> > >> >>>>>>>
> > >> >>>>>>> My opinion is that no features should be merged until all
> > >> >> functional
> > >> >>>>>>> issues have been resolved and it is ready to turn over to
> test.
> > >> >>> Until
> > >> >>>>>> the
> > >> >>>>>>> total Ops vs discrete read/write ops issue is addressed and
> > >> >>> re-reviewed
> > >> >>>>>> by
> > >> >>>>>>> Wido, I don't think this criteria has been satisfied.
> > >> >>>>>>>
> > >> >>>>>>> Also, how does this work intersect/compliment the SolidFire
> > patch
> > >> (
> > >> >>>>>>> https://reviews.apache.org/r/11479/)? As I understand it
> that
> > >> >> work
> > >> >>> is
> > >> >>>>>>> also involves provisioned IOPS. I would like to ensure we
> don't
> > >> >>> have a
> > >> >>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
> > >> >>> unnecessarily
> > >> >>>>>>> incompatible.
> > >> >>>>>>>
> > >> >>>>>>> Thanks,
> > >> >>>>>>> -John
> > >> >>>>>>>
> > >> >>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com>
> > >> >> wrote:
> > >> >>>>>>>
> > >> >>>>>>> Wido,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Sure. I will change it next week.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> -Wei
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Hi Wei,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Wido,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Exactly. I have pushed the features into master.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> If anyone object thems for technical reason till Monday, I
> will
> > >> >>> revert
> > >> >>>>>>>
> > >> >>>>>>> them.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> For the sake of clarity I just want to mention again that we
> > >> should
> > >> >>>>>> change
> > >> >>>>>>>
> > >> >>>>>>> the total IOps to R/W IOps asap so that we never release a
> > version
> > >> >>> with
> > >> >>>>>>>
> > >> >>>>>>> only total IOps.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> You laid the groundwork for the I/O throttling and that's
> great!
> > >> We
> > >> >>>>>> should
> > >> >>>>>>>
> > >> >>>>>>> however prevent that we create legacy from day #1.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Wido
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> -Wei
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Wido,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> +1 -- this enhancement must to discretely support read and
> write
> > >> >>> IOPS.
> > >> >>>>>>>
> > >> >>>>>>> I
> > >> >>>>>>>
> > >> >>>>>>> don't see how it could be fixed later because I don't see how
> we
> > >> >>>>>>>
> > >> >>>>>>> correctly
> > >> >>>>>>>
> > >> >>>>>>> split total IOPS into read and write. Therefore, we would be
> > >> stuck
> > >> >>>>>>>
> > >> >>>>>>> with a
> > >> >>>>>>>
> > >> >>>>>>> total unless/until we decided to break backwards
> compatibility.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> What Wei meant was merging it into master now so that it will
> go
> > >> in
> > >> >>> the
> > >> >>>>>>>
> > >> >>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so
> > >> that
> > >> >>> 4.2
> > >> >>>>>>>
> > >> >>>>>>> will be released with Read and Write instead of Total IOps.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> This is to make the May 31st feature freeze date. But if the
> > >> window
> > >> >>>> moves
> > >> >>>>>>>
> > >> >>>>>>> (see other threads) then it won't be necessary to do that.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Wido
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> I also completely agree that there is no association between
> > >> >> network
> > >> >>>>>>>
> > >> >>>>>>> and
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> disk I/O.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Thanks,
> > >> >>>>>>>
> > >> >>>>>>> -John
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <
> wido@widodh.nl
> > >
> > >> >>>> wrote:
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Hi Wei,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Hi Wido,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Thanks. Good question.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> I thought about at the beginning. Finally I decided to ignore
> > the
> > >> >>>>>>>
> > >> >>>>>>> difference of read and write mainly because the network
> > throttling
> > >> >>> did
> > >> >>>>>>>
> > >> >>>>>>> not
> > >> >>>>>>>
> > >> >>>>>>> care the difference of sent and received bytes as well.
> > >> >>>>>>>
> > >> >>>>>>> That reasoning seems odd. Networking and disk I/O completely
> > >> >>> different.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Disk I/O is much more expensive in most situations then
> network
> > >> >>>>>>>
> > >> >>>>>>> bandwith.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Implementing it will be some copy-paste work. It could be
> > >> >>>>>>>
> > >> >>>>>>> implemented in
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> few days. For the deadline of feature freeze, I will implement
> > it
> > >> >>>>>>>
> > >> >>>>>>> after
> > >> >>>>>>>
> > >> >>>>>>> that , if needed.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> It think it's a feature we can't miss. But if it goes into the
> > 4.2
> > >> >>>>>>>
> > >> >>>>>>> window we have to make sure we don't release with only total
> > IOps
> > >> >> and
> > >> >>>>>>>
> > >> >>>>>>> fix
> > >> >>>>>>>
> > >> >>>>>>> it in 4.3, that will confuse users.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Wido
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> -Wei
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Hi Wei,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Hi,
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> I would like to merge disk_io_throttling branch into master.
> > >> >>>>>>>
> > >> >>>>>>> If nobody object, I will merge into master in 48 hours.
> > >> >>>>>>>
> > >> >>>>>>> The purpose is :
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Virtual machines are running on the same storage device (local
> > >> >>>>>>>
> > >> >>>>>>> storage or
> > >> >>>>>>>
> > >> >>>>>>> share strage). Because of the rate limitation of device (such
> as
> > >> >>>>>>>
> > >> >>>>>>> iops), if
> > >> >>>>>>>
> > >> >>>>>>> one VM has large disk operation, it may affect the disk
> > >> performance
> > >> >>>>>>>
> > >> >>>>>>> of
> > >> >>>>>>>
> > >> >>>>>>> other VMs running on the same storage device.
> > >> >>>>>>>
> > >> >>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
> > of
> > >> >>>>>>>
> > >> >>>>>>> VMs.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Looking at the code I see you make no difference between Read
> > and
> > >> >>>>>>>
> > >> >>>>>>> Write
> > >> >>>>>>>
> > >> >>>>>>> IOps.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Qemu and libvirt support setting both a different rate for
> Read
> > >> and
> > >> >>>>>>>
> > >> >>>>>>> Write
> > >> >>>>>>>
> > >> >>>>>>> IOps which could benefit a lot of users.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> It's also strange, in the polling side you collect both the
> Read
> > >> >> and
> > >> >>>>>>>
> > >> >>>>>>> Write
> > >> >>>>>>>
> > >> >>>>>>> IOps, but on the throttling side you only go for a global
> value.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Write IOps are usually much more expensive then Read IOps, so
> it
> > >> >>>>>>>
> > >> >>>>>>> seems
> > >> >>>>>>>
> > >> >>>>>>> like a valid use-case where that an admin would set a lower
> > value
> > >> >> for
> > >> >>>>>>>
> > >> >>>>>>> write
> > >> >>>>>>>
> > >> >>>>>>> IOps vs Read IOps.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Since this only supports KVM at this point I think it would be
> > of
> > >> >>>>>>>
> > >> >>>>>>> great
> > >> >>>>>>>
> > >> >>>>>>> value to at least have the mechanism in place to support both,
> > >> >>>>>>>
> > >> >>>>>>> implementing
> > >> >>>>>>>
> > >> >>>>>>> this later would be a lot of work.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> If a hypervisor doesn't support setting different values for
> > read
> > >> >> and
> > >> >>>>>>>
> > >> >>>>>>> write you can always sum both up and set that as the total
> > limit.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Can you explain why you implemented it this way?
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Wido
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> The feature includes:
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> > >> >>>>>>>
> > >> >>>>>>> configuration)
> > >> >>>>>>>
> > >> >>>>>>> (2) change the maximum rate of VMs
> > >> >>>>>>>
> > >> >>>>>>> (3) limit the disk rate (total bps and iops)
> > >> >>>>>>>
> > >> >>>>>>> JIRA ticket: https://issues.apache.org/****
> > >> >>>>>>>
> > >> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> > >> >>>>>>>
> > >> >>>>>>> jira/browse/CLOUDSTACK-1192<
> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> > >> >>>>>>>
> > >> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> > >> >>>>>>>
> > >> >>>>>>> <**
> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> > >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> FS (I will update later) :
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > >> >>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > >> >>>>>>>
> > >> >>>>>>> <
> > >> >>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > <
> > >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > >> >>>>>>>
> > >> >>>>>>> VM+Disk+IO+Throttling<https://****
> > >> cwiki.apache.org/confluence/****
> > >> >> <
> > >> >>>>>>> http://cwiki.apache.org/confluence/**>
> > >> >>>>>>>
> > >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > >> >>>>>>>
> > >> >>>>>>>
> > >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >> >> <
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>
> > >> >>>
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Merge check list :-
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> * Did you check the branch's RAT execution success?
> > >> >>>>>>>
> > >> >>>>>>> Yes
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> * Are there new dependencies introduced?
> > >> >>>>>>>
> > >> >>>>>>> No
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> * What automated testing (unit and integration) is included in
> > the
> > >> >>>>>>>
> > >> >>>>>>> new
> > >> >>>>>>>
> > >> >>>>>>> feature?
> > >> >>>>>>>
> > >> >>>>>>> Unit tests are added.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> * What testing has been done to check for potential
> regressions?
> > >> >>>>>>>
> > >> >>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> > >> >>>>>>>
> > >> >>>>>>> (2) VM operations, including
> > >> >>>>>>>
> > >> >>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate,
> restore
> > >> >>>>>>>
> > >> >>>>>>> (3) Volume operations, including
> > >> >>>>>>>
> > >> >>>>>>> Attach, Detach
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> To review the code, you can try
> > >> >>>>>>>
> > >> >>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> > >> >>>>>>>
> > >> >>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> Best regards,
> > >> >>>>>>>
> > >> >>>>>>> Wei
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> [1]
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > >> >>>>>>
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > >> >>>>>>>
> > >> >>>>>>> <
> > >> >>>>>>>
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> > <
> > >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > >> >>>>>>>
> > >> >>>>>>> VM+Disk+IO+Throttling<https://****
> > >> cwiki.apache.org/confluence/****
> > >> >> <
> > >> >>>>>>> http://cwiki.apache.org/confluence/**>
> > >> >>>>>>>
> > >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > >> >>>>>>>
> > >> >>>>>>>
> > >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> > >> >> <
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>
> > >> >>>
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> [2] refs/heads/disk_io_throttling
> > >> >>>>>>>
> > >> >>>>>>> [3]
> > >> >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> > >> >>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> > >> >>>>>>>
> > >> >>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> > >> >>>>>>>
> > >> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> > >> >>>>>>>
> > >> >>>>>>> <**
> > >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> > >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> <ht**tps://
> issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> > >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> > >> >>>>>>>
> > >> >>>>>>> **<
> > >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> > >> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> > >> >>>>>>>
> > >> >>>>>>> <**
> > >> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> > >> >>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> > >> >>>>>>>
> > >> >>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > >> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> (**CLOUDSTACK-1301
> > >> >>>>>>>
> > >> >>>>>>> - VM Disk I/O Throttling)
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> --
> > >> >>>>>> *Mike Tutkowski*
> > >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > >> >>>>>> e: mike.tutkowski@solidfire.com
> > >> >>>>>> o: 303.746.7302
> > >> >>>>>> Advancing the way the world uses the
> > >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> >>>>>> *â„¢*
> > >> >>>>>>
> > >> >>>>
> > >> >>>>
> > >> >>>
> > >> >>
> > >> >>
> > >> >>
> > >> >> --
> > >> >> *Mike Tutkowski*
> > >> >> *Senior CloudStack Developer, SolidFire Inc.*
> > >> >> e: mike.tutkowski@solidfire.com
> > >> >> o: 303.746.7302
> > >> >> Advancing the way the world uses the
> > >> >> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> >> *â„¢*
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > *Mike Tutkowski*
> > >> > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > e: mike.tutkowski@solidfire.com
> > >> > o: 303.746.7302
> > >> > Advancing the way the world uses the
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > *â„¢*
> > >>
> > >>
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the cloud<
> > http://solidfire.com/solution/overview/?video=play>
> > > *â„¢*
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
It is generally odd to me that any operation in the Storage layer would
understand or care about details. I expect to see the Storage services
expose a set of operations that can be composed/driven by the Hypervisor
implementations to allocate space/create structures per their needs. If we
don't invert this dependency, we are going to end with a massive n-to-n
problem that will make the system increasingly difficult to maintain and
enhance. Am I understanding that the Xen specific SolidFire code is
located in the CitrixResourceBase class?
Thanks,
-John
On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> To delve into this in a bit more detail:
>
> Prior to 4.2 and aside from one setup method for XenServer, the admin had
> to first create a volume on the storage system, then go into the hypervisor
> to set up a data structure to make use of the volume (ex. a storage
> repository on XenServer or a datastore on ESX(i)). VMs and data disks then
> shared this storage system's volume.
>
> With Edison's new storage framework, storage need no longer be so static
> and you can easily create a 1:1 relationship between a storage system's
> volume and the VM's data disk (necessary for storage Quality of Service).
>
> You can now write a plug-in that is called to dynamically create and delete
> volumes as needed.
>
> The problem that the storage framework did not address is in creating and
> deleting the hypervisor-specific data structure when performing an
> attach/detach.
>
> That being the case, I've been enhancing it to do so. I've got XenServer
> worked out and submitted. I've got ESX(i) in my sandbox and can submit this
> if we extend the 4.2 freeze date.
>
> Does that help a bit? :)
>
>
> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> > wrote:
>
> > Hi John,
> >
> > The storage plug-in - by itself - is hypervisor agnostic.
> >
> > The issue is with the volume-attach logic (in the agent code). The
> storage
> > framework calls into the plug-in to have it create a volume as needed,
> but
> > when the time comes to attach the volume to a hypervisor, the attach
> logic
> > has to be smart enough to recognize it's being invoked on zone-wide
> storage
> > (where the volume has just been created) and create, say, a storage
> > repository (for XenServer) or a datastore (for VMware) to make use of the
> > volume that was just created.
> >
> > I've been spending most of my time recently making the attach logic work
> > in the agent code.
> >
> > Does that clear it up?
> >
> > Thanks!
> >
> >
> > On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Can you explain why the the storage driver is hypervisor specific?
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >> > Yes, ultimately I would like to support all hypervisors that
> CloudStack
> >> > supports. I think I'm just out of time for 4.2 to get KVM in.
> >> >
> >> > Right now this plug-in supports XenServer. Depending on what we do
> with
> >> > regards to 4.2 feature freeze, I have it working for VMware in my
> >> sandbox,
> >> > as well.
> >> >
> >> > Also, just to be clear, this is all in regards to Disk Offerings. I
> >> plan to
> >> > support Compute Offerings post 4.2.
> >> >
> >> >
> >> > On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <
> kelcey@bbits.ca
> >> >wrote:
> >> >
> >> >> Is there any plan on supporting KVM in the patch cycle post 4.2?
> >> >>
> >> >> ----- Original Message -----
> >> >> From: "Mike Tutkowski" <mi...@solidfire.com>
> >> >> To: dev@cloudstack.apache.org
> >> >> Sent: Monday, June 3, 2013 10:12:32 AM
> >> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >> >>
> >> >> I agree on merging Wei's feature first, then mine.
> >> >>
> >> >> If his feature is for KVM only, then it is a non issue as I don't
> >> support
> >> >> KVM in 4.2.
> >> >>
> >> >>
> >> >> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
> >> wrote:
> >> >>
> >> >>> John,
> >> >>>
> >> >>> For the billing, as no one works on billing now, users need to
> >> calculate
> >> >>> the billing by themselves. They can get the service_offering and
> >> >>> disk_offering of a VMs and volumes for calculation. Of course it is
> >> >> better
> >> >>> to tell user the exact limitation value of individual volume, and
> >> network
> >> >>> rate limitation for nics as well. I can work on it later. Do you
> >> think it
> >> >>> is a part of I/O throttling?
> >> >>>
> >> >>> Sorry my misunstand the second the question.
> >> >>>
> >> >>> Agree with what you said about the two features.
> >> >>>
> >> >>> -Wei
> >> >>>
> >> >>>
> >> >>> 2013/6/3 John Burwell <jb...@basho.com>
> >> >>>
> >> >>>> Wei,
> >> >>>>
> >> >>>>
> >> >>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com>
> wrote:
> >> >>>>
> >> >>>>> Hi John, Mike
> >> >>>>>
> >> >>>>> I hope Mike's aswer helps you. I am trying to adding more.
> >> >>>>>
> >> >>>>> (1) I think billing should depend on IO statistics rather than
> IOPS
> >> >>>>> limitation. Please review disk_io_stat if you have time.
> >> >> disk_io_stat
> >> >>>> can
> >> >>>>> get the IO statistics including bytes/iops read/write for an
> >> >> individual
> >> >>>>> virtual machine.
> >> >>>>
> >> >>>> Going by the AWS model, customers are billed more for volumes with
> >> >>>> provisioned IOPS, as well as, for those operations (
> >> >>>> http://aws.amazon.com/ebs/). I would imagine our users would like
> >> the
> >> >>>> option to employ similar cost models. Could an operator implement
> >> >> such a
> >> >>>> billing model in the current patch?
> >> >>>>
> >> >>>>>
> >> >>>>> (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
> >> >>>>> limitation for a running virtual machine through command line.
> >> >> However,
> >> >>>>> CloudStack does not support changing the parameters of a created
> >> >>> offering
> >> >>>>> (computer offering or disk offering).
> >> >>>>
> >> >>>> I meant at the Java interface level. I apologize for being
> unclear.
> >> >> Can
> >> >>>> we more generalize allocation algorithms with a set of interfaces
> >> that
> >> >>>> describe the service guarantees provided by a resource?
> >> >>>>
> >> >>>>>
> >> >>>>> (3) It is a good question. Maybe it is better to commit Mike's
> patch
> >> >>>> after
> >> >>>>> disk_io_throttling as Mike needs to consider the limitation in
> >> >>> hypervisor
> >> >>>>> type, I think.
> >> >>>>
> >> >>>> I will expand on my thoughts in a later response to Mike regarding
> >> the
> >> >>>> touch points between these two features. I think that
> >> >> disk_io_throttling
> >> >>>> will need to be merged before SolidFire, but I think we need closer
> >> >>>> coordination between the branches (possibly have have solidfire
> track
> >> >>>> disk_io_throttling) to coordinate on this issue.
> >> >>>>
> >> >>>>>
> >> >>>>> - Wei
> >> >>>>>
> >> >>>>>
> >> >>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >> >>>>>
> >> >>>>>> Mike,
> >> >>>>>>
> >> >>>>>> The things I want to understand are the following:
> >> >>>>>>
> >> >>>>>> 1) Is there value in capturing IOPS policies be captured in a
> >> >> common
> >> >>>>>> data model (e.g. for billing/usage purposes, expressing
> offerings).
> >> >>>>>> 2) Should there be a common interface model for reasoning about
> >> >> IOP
> >> >>>>>> provisioning at runtime?
> >> >>>>>> 3) How are conflicting provisioned IOPS configurations between
> a
> >> >>>>>> hypervisor and storage device reconciled? In particular, a
> >> scenario
> >> >>>> where
> >> >>>>>> is lead to believe (and billed) for more IOPS configured for a VM
> >> >>> than a
> >> >>>>>> storage device has been configured to deliver. Another scenario
> >> >>> could a
> >> >>>>>> consistent configuration between a VM and a storage device at
> >> >> creation
> >> >>>>>> time, but a later modification to storage device introduces
> logical
> >> >>>>>> inconsistency.
> >> >>>>>>
> >> >>>>>> Thanks,
> >> >>>>>> -John
> >> >>>>>>
> >> >>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >> >>>> mike.tutkowski@solidfire.com>
> >> >>>>>> wrote:
> >> >>>>>>
> >> >>>>>> Hi John,
> >> >>>>>>
> >> >>>>>> I believe Wei's feature deals with controlling the max number of
> >> >> IOPS
> >> >>>> from
> >> >>>>>> the hypervisor side.
> >> >>>>>>
> >> >>>>>> My feature is focused on controlling IOPS from the storage system
> >> >>> side.
> >> >>>>>>
> >> >>>>>> I hope that helps. :)
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jburwell@basho.com
> >
> >> >>>> wrote:
> >> >>>>>>
> >> >>>>>>> Wei,
> >> >>>>>>>
> >> >>>>>>> My opinion is that no features should be merged until all
> >> >> functional
> >> >>>>>>> issues have been resolved and it is ready to turn over to test.
> >> >>> Until
> >> >>>>>> the
> >> >>>>>>> total Ops vs discrete read/write ops issue is addressed and
> >> >>> re-reviewed
> >> >>>>>> by
> >> >>>>>>> Wido, I don't think this criteria has been satisfied.
> >> >>>>>>>
> >> >>>>>>> Also, how does this work intersect/compliment the SolidFire
> patch
> >> (
> >> >>>>>>> https://reviews.apache.org/r/11479/)? As I understand it that
> >> >> work
> >> >>> is
> >> >>>>>>> also involves provisioned IOPS. I would like to ensure we don't
> >> >>> have a
> >> >>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
> >> >>> unnecessarily
> >> >>>>>>> incompatible.
> >> >>>>>>>
> >> >>>>>>> Thanks,
> >> >>>>>>> -John
> >> >>>>>>>
> >> >>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com>
> >> >> wrote:
> >> >>>>>>>
> >> >>>>>>> Wido,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Sure. I will change it next week.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> -Wei
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Hi Wei,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Wido,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Exactly. I have pushed the features into master.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> If anyone object thems for technical reason till Monday, I will
> >> >>> revert
> >> >>>>>>>
> >> >>>>>>> them.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> For the sake of clarity I just want to mention again that we
> >> should
> >> >>>>>> change
> >> >>>>>>>
> >> >>>>>>> the total IOps to R/W IOps asap so that we never release a
> version
> >> >>> with
> >> >>>>>>>
> >> >>>>>>> only total IOps.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> You laid the groundwork for the I/O throttling and that's great!
> >> We
> >> >>>>>> should
> >> >>>>>>>
> >> >>>>>>> however prevent that we create legacy from day #1.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Wido
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> -Wei
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Wido,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> +1 -- this enhancement must to discretely support read and write
> >> >>> IOPS.
> >> >>>>>>>
> >> >>>>>>> I
> >> >>>>>>>
> >> >>>>>>> don't see how it could be fixed later because I don't see how we
> >> >>>>>>>
> >> >>>>>>> correctly
> >> >>>>>>>
> >> >>>>>>> split total IOPS into read and write. Therefore, we would be
> >> stuck
> >> >>>>>>>
> >> >>>>>>> with a
> >> >>>>>>>
> >> >>>>>>> total unless/until we decided to break backwards compatibility.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> What Wei meant was merging it into master now so that it will go
> >> in
> >> >>> the
> >> >>>>>>>
> >> >>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so
> >> that
> >> >>> 4.2
> >> >>>>>>>
> >> >>>>>>> will be released with Read and Write instead of Total IOps.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> This is to make the May 31st feature freeze date. But if the
> >> window
> >> >>>> moves
> >> >>>>>>>
> >> >>>>>>> (see other threads) then it won't be necessary to do that.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Wido
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> I also completely agree that there is no association between
> >> >> network
> >> >>>>>>>
> >> >>>>>>> and
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> disk I/O.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Thanks,
> >> >>>>>>>
> >> >>>>>>> -John
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wido@widodh.nl
> >
> >> >>>> wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Hi Wei,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Hi Wido,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Thanks. Good question.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> I thought about at the beginning. Finally I decided to ignore
> the
> >> >>>>>>>
> >> >>>>>>> difference of read and write mainly because the network
> throttling
> >> >>> did
> >> >>>>>>>
> >> >>>>>>> not
> >> >>>>>>>
> >> >>>>>>> care the difference of sent and received bytes as well.
> >> >>>>>>>
> >> >>>>>>> That reasoning seems odd. Networking and disk I/O completely
> >> >>> different.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Disk I/O is much more expensive in most situations then network
> >> >>>>>>>
> >> >>>>>>> bandwith.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Implementing it will be some copy-paste work. It could be
> >> >>>>>>>
> >> >>>>>>> implemented in
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> few days. For the deadline of feature freeze, I will implement
> it
> >> >>>>>>>
> >> >>>>>>> after
> >> >>>>>>>
> >> >>>>>>> that , if needed.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> It think it's a feature we can't miss. But if it goes into the
> 4.2
> >> >>>>>>>
> >> >>>>>>> window we have to make sure we don't release with only total
> IOps
> >> >> and
> >> >>>>>>>
> >> >>>>>>> fix
> >> >>>>>>>
> >> >>>>>>> it in 4.3, that will confuse users.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Wido
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> -Wei
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Hi Wei,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Hi,
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> I would like to merge disk_io_throttling branch into master.
> >> >>>>>>>
> >> >>>>>>> If nobody object, I will merge into master in 48 hours.
> >> >>>>>>>
> >> >>>>>>> The purpose is :
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Virtual machines are running on the same storage device (local
> >> >>>>>>>
> >> >>>>>>> storage or
> >> >>>>>>>
> >> >>>>>>> share strage). Because of the rate limitation of device (such as
> >> >>>>>>>
> >> >>>>>>> iops), if
> >> >>>>>>>
> >> >>>>>>> one VM has large disk operation, it may affect the disk
> >> performance
> >> >>>>>>>
> >> >>>>>>> of
> >> >>>>>>>
> >> >>>>>>> other VMs running on the same storage device.
> >> >>>>>>>
> >> >>>>>>> It is neccesary to set the maximum rate and limit the disk I/O
> of
> >> >>>>>>>
> >> >>>>>>> VMs.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Looking at the code I see you make no difference between Read
> and
> >> >>>>>>>
> >> >>>>>>> Write
> >> >>>>>>>
> >> >>>>>>> IOps.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Qemu and libvirt support setting both a different rate for Read
> >> and
> >> >>>>>>>
> >> >>>>>>> Write
> >> >>>>>>>
> >> >>>>>>> IOps which could benefit a lot of users.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> It's also strange, in the polling side you collect both the Read
> >> >> and
> >> >>>>>>>
> >> >>>>>>> Write
> >> >>>>>>>
> >> >>>>>>> IOps, but on the throttling side you only go for a global value.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Write IOps are usually much more expensive then Read IOps, so it
> >> >>>>>>>
> >> >>>>>>> seems
> >> >>>>>>>
> >> >>>>>>> like a valid use-case where that an admin would set a lower
> value
> >> >> for
> >> >>>>>>>
> >> >>>>>>> write
> >> >>>>>>>
> >> >>>>>>> IOps vs Read IOps.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Since this only supports KVM at this point I think it would be
> of
> >> >>>>>>>
> >> >>>>>>> great
> >> >>>>>>>
> >> >>>>>>> value to at least have the mechanism in place to support both,
> >> >>>>>>>
> >> >>>>>>> implementing
> >> >>>>>>>
> >> >>>>>>> this later would be a lot of work.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> If a hypervisor doesn't support setting different values for
> read
> >> >> and
> >> >>>>>>>
> >> >>>>>>> write you can always sum both up and set that as the total
> limit.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Can you explain why you implemented it this way?
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Wido
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> The feature includes:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> >> >>>>>>>
> >> >>>>>>> configuration)
> >> >>>>>>>
> >> >>>>>>> (2) change the maximum rate of VMs
> >> >>>>>>>
> >> >>>>>>> (3) limit the disk rate (total bps and iops)
> >> >>>>>>>
> >> >>>>>>> JIRA ticket: https://issues.apache.org/****
> >> >>>>>>>
> >> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> >> >>>>>>>
> >> >>>>>>> jira/browse/CLOUDSTACK-1192<
> >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >> >>>>>>>
> >> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >> >>>>>>>
> >> >>>>>>> <**
> >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> FS (I will update later) :
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >> >>>>>>>
> >> >>>>>>> <
> >> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> <
> >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >> >>>>>>>
> >> >>>>>>> VM+Disk+IO+Throttling<https://****
> >> cwiki.apache.org/confluence/****
> >> >> <
> >> >>>>>>> http://cwiki.apache.org/confluence/**>
> >> >>>>>>>
> >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >> >>>>>>>
> >> >>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >> >> <
> >> >>>>>>>
> >> >>>>>>
> >> >>>>
> >> >>>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Merge check list :-
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> * Did you check the branch's RAT execution success?
> >> >>>>>>>
> >> >>>>>>> Yes
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> * Are there new dependencies introduced?
> >> >>>>>>>
> >> >>>>>>> No
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> * What automated testing (unit and integration) is included in
> the
> >> >>>>>>>
> >> >>>>>>> new
> >> >>>>>>>
> >> >>>>>>> feature?
> >> >>>>>>>
> >> >>>>>>> Unit tests are added.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> * What testing has been done to check for potential regressions?
> >> >>>>>>>
> >> >>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >> >>>>>>>
> >> >>>>>>> (2) VM operations, including
> >> >>>>>>>
> >> >>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >> >>>>>>>
> >> >>>>>>> (3) Volume operations, including
> >> >>>>>>>
> >> >>>>>>> Attach, Detach
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> To review the code, you can try
> >> >>>>>>>
> >> >>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >> >>>>>>>
> >> >>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Best regards,
> >> >>>>>>>
> >> >>>>>>> Wei
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> [1]
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >> >>>>>>>
> >> >>>>>>> <
> >> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> <
> >> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >> >>>>>>>
> >> >>>>>>> VM+Disk+IO+Throttling<https://****
> >> cwiki.apache.org/confluence/****
> >> >> <
> >> >>>>>>> http://cwiki.apache.org/confluence/**>
> >> >>>>>>>
> >> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >> >>>>>>>
> >> >>>>>>>
> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >> >> <
> >> >>>>>>>
> >> >>>>>>
> >> >>>>
> >> >>>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> [2] refs/heads/disk_io_throttling
> >> >>>>>>>
> >> >>>>>>> [3]
> >> >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> >> >>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> >> >>>>>>>
> >> >>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >> >>>>>>>
> >> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >> >>>>>>>
> >> >>>>>>> <**
> >> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> >> >>>>>>>
> >> >>>>>>> **<
> >> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> >> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >> >>>>>>>
> >> >>>>>>> <**
> >> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >> >>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >> >>>>>>>
> >> >>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> (**CLOUDSTACK-1301
> >> >>>>>>>
> >> >>>>>>> - VM Disk I/O Throttling)
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> --
> >> >>>>>> *Mike Tutkowski*
> >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >> >>>>>> e: mike.tutkowski@solidfire.com
> >> >>>>>> o: 303.746.7302
> >> >>>>>> Advancing the way the world uses the
> >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >> >>>>>> *â„¢*
> >> >>>>>>
> >> >>>>
> >> >>>>
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> *Mike Tutkowski*
> >> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> >> e: mike.tutkowski@solidfire.com
> >> >> o: 303.746.7302
> >> >> Advancing the way the world uses the
> >> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> >> *â„¢*
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *â„¢*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
To delve into this in a bit more detail:
Prior to 4.2 and aside from one setup method for XenServer, the admin had
to first create a volume on the storage system, then go into the hypervisor
to set up a data structure to make use of the volume (ex. a storage
repository on XenServer or a datastore on ESX(i)). VMs and data disks then
shared this storage system's volume.
With Edison's new storage framework, storage need no longer be so static
and you can easily create a 1:1 relationship between a storage system's
volume and the VM's data disk (necessary for storage Quality of Service).
You can now write a plug-in that is called to dynamically create and delete
volumes as needed.
The problem that the storage framework did not address is in creating and
deleting the hypervisor-specific data structure when performing an
attach/detach.
That being the case, I've been enhancing it to do so. I've got XenServer
worked out and submitted. I've got ESX(i) in my sandbox and can submit this
if we extend the 4.2 freeze date.
Does that help a bit? :)
On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> Hi John,
>
> The storage plug-in - by itself - is hypervisor agnostic.
>
> The issue is with the volume-attach logic (in the agent code). The storage
> framework calls into the plug-in to have it create a volume as needed, but
> when the time comes to attach the volume to a hypervisor, the attach logic
> has to be smart enough to recognize it's being invoked on zone-wide storage
> (where the volume has just been created) and create, say, a storage
> repository (for XenServer) or a datastore (for VMware) to make use of the
> volume that was just created.
>
> I've been spending most of my time recently making the attach logic work
> in the agent code.
>
> Does that clear it up?
>
> Thanks!
>
>
> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> Can you explain why the the storage driver is hypervisor specific?
>>
>> Thanks,
>> -John
>>
>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > Yes, ultimately I would like to support all hypervisors that CloudStack
>> > supports. I think I'm just out of time for 4.2 to get KVM in.
>> >
>> > Right now this plug-in supports XenServer. Depending on what we do with
>> > regards to 4.2 feature freeze, I have it working for VMware in my
>> sandbox,
>> > as well.
>> >
>> > Also, just to be clear, this is all in regards to Disk Offerings. I
>> plan to
>> > support Compute Offerings post 4.2.
>> >
>> >
>> > On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <kelcey@bbits.ca
>> >wrote:
>> >
>> >> Is there any plan on supporting KVM in the patch cycle post 4.2?
>> >>
>> >> ----- Original Message -----
>> >> From: "Mike Tutkowski" <mi...@solidfire.com>
>> >> To: dev@cloudstack.apache.org
>> >> Sent: Monday, June 3, 2013 10:12:32 AM
>> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
>> >>
>> >> I agree on merging Wei's feature first, then mine.
>> >>
>> >> If his feature is for KVM only, then it is a non issue as I don't
>> support
>> >> KVM in 4.2.
>> >>
>> >>
>> >> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com>
>> wrote:
>> >>
>> >>> John,
>> >>>
>> >>> For the billing, as no one works on billing now, users need to
>> calculate
>> >>> the billing by themselves. They can get the service_offering and
>> >>> disk_offering of a VMs and volumes for calculation. Of course it is
>> >> better
>> >>> to tell user the exact limitation value of individual volume, and
>> network
>> >>> rate limitation for nics as well. I can work on it later. Do you
>> think it
>> >>> is a part of I/O throttling?
>> >>>
>> >>> Sorry my misunstand the second the question.
>> >>>
>> >>> Agree with what you said about the two features.
>> >>>
>> >>> -Wei
>> >>>
>> >>>
>> >>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>
>> >>>> Wei,
>> >>>>
>> >>>>
>> >>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
>> >>>>
>> >>>>> Hi John, Mike
>> >>>>>
>> >>>>> I hope Mike's aswer helps you. I am trying to adding more.
>> >>>>>
>> >>>>> (1) I think billing should depend on IO statistics rather than IOPS
>> >>>>> limitation. Please review disk_io_stat if you have time.
>> >> disk_io_stat
>> >>>> can
>> >>>>> get the IO statistics including bytes/iops read/write for an
>> >> individual
>> >>>>> virtual machine.
>> >>>>
>> >>>> Going by the AWS model, customers are billed more for volumes with
>> >>>> provisioned IOPS, as well as, for those operations (
>> >>>> http://aws.amazon.com/ebs/). I would imagine our users would like
>> the
>> >>>> option to employ similar cost models. Could an operator implement
>> >> such a
>> >>>> billing model in the current patch?
>> >>>>
>> >>>>>
>> >>>>> (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
>> >>>>> limitation for a running virtual machine through command line.
>> >> However,
>> >>>>> CloudStack does not support changing the parameters of a created
>> >>> offering
>> >>>>> (computer offering or disk offering).
>> >>>>
>> >>>> I meant at the Java interface level. I apologize for being unclear.
>> >> Can
>> >>>> we more generalize allocation algorithms with a set of interfaces
>> that
>> >>>> describe the service guarantees provided by a resource?
>> >>>>
>> >>>>>
>> >>>>> (3) It is a good question. Maybe it is better to commit Mike's patch
>> >>>> after
>> >>>>> disk_io_throttling as Mike needs to consider the limitation in
>> >>> hypervisor
>> >>>>> type, I think.
>> >>>>
>> >>>> I will expand on my thoughts in a later response to Mike regarding
>> the
>> >>>> touch points between these two features. I think that
>> >> disk_io_throttling
>> >>>> will need to be merged before SolidFire, but I think we need closer
>> >>>> coordination between the branches (possibly have have solidfire track
>> >>>> disk_io_throttling) to coordinate on this issue.
>> >>>>
>> >>>>>
>> >>>>> - Wei
>> >>>>>
>> >>>>>
>> >>>>> 2013/6/3 John Burwell <jb...@basho.com>
>> >>>>>
>> >>>>>> Mike,
>> >>>>>>
>> >>>>>> The things I want to understand are the following:
>> >>>>>>
>> >>>>>> 1) Is there value in capturing IOPS policies be captured in a
>> >> common
>> >>>>>> data model (e.g. for billing/usage purposes, expressing offerings).
>> >>>>>> 2) Should there be a common interface model for reasoning about
>> >> IOP
>> >>>>>> provisioning at runtime?
>> >>>>>> 3) How are conflicting provisioned IOPS configurations between a
>> >>>>>> hypervisor and storage device reconciled? In particular, a
>> scenario
>> >>>> where
>> >>>>>> is lead to believe (and billed) for more IOPS configured for a VM
>> >>> than a
>> >>>>>> storage device has been configured to deliver. Another scenario
>> >>> could a
>> >>>>>> consistent configuration between a VM and a storage device at
>> >> creation
>> >>>>>> time, but a later modification to storage device introduces logical
>> >>>>>> inconsistency.
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> -John
>> >>>>>>
>> >>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> >>>> mike.tutkowski@solidfire.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>> Hi John,
>> >>>>>>
>> >>>>>> I believe Wei's feature deals with controlling the max number of
>> >> IOPS
>> >>>> from
>> >>>>>> the hypervisor side.
>> >>>>>>
>> >>>>>> My feature is focused on controlling IOPS from the storage system
>> >>> side.
>> >>>>>>
>> >>>>>> I hope that helps. :)
>> >>>>>>
>> >>>>>>
>> >>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> Wei,
>> >>>>>>>
>> >>>>>>> My opinion is that no features should be merged until all
>> >> functional
>> >>>>>>> issues have been resolved and it is ready to turn over to test.
>> >>> Until
>> >>>>>> the
>> >>>>>>> total Ops vs discrete read/write ops issue is addressed and
>> >>> re-reviewed
>> >>>>>> by
>> >>>>>>> Wido, I don't think this criteria has been satisfied.
>> >>>>>>>
>> >>>>>>> Also, how does this work intersect/compliment the SolidFire patch
>> (
>> >>>>>>> https://reviews.apache.org/r/11479/)? As I understand it that
>> >> work
>> >>> is
>> >>>>>>> also involves provisioned IOPS. I would like to ensure we don't
>> >>> have a
>> >>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>> >>> unnecessarily
>> >>>>>>> incompatible.
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> -John
>> >>>>>>>
>> >>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com>
>> >> wrote:
>> >>>>>>>
>> >>>>>>> Wido,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Sure. I will change it next week.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> -Wei
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Hi Wei,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Wido,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Exactly. I have pushed the features into master.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> If anyone object thems for technical reason till Monday, I will
>> >>> revert
>> >>>>>>>
>> >>>>>>> them.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> For the sake of clarity I just want to mention again that we
>> should
>> >>>>>> change
>> >>>>>>>
>> >>>>>>> the total IOps to R/W IOps asap so that we never release a version
>> >>> with
>> >>>>>>>
>> >>>>>>> only total IOps.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> You laid the groundwork for the I/O throttling and that's great!
>> We
>> >>>>>> should
>> >>>>>>>
>> >>>>>>> however prevent that we create legacy from day #1.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Wido
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> -Wei
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Wido,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> +1 -- this enhancement must to discretely support read and write
>> >>> IOPS.
>> >>>>>>>
>> >>>>>>> I
>> >>>>>>>
>> >>>>>>> don't see how it could be fixed later because I don't see how we
>> >>>>>>>
>> >>>>>>> correctly
>> >>>>>>>
>> >>>>>>> split total IOPS into read and write. Therefore, we would be
>> stuck
>> >>>>>>>
>> >>>>>>> with a
>> >>>>>>>
>> >>>>>>> total unless/until we decided to break backwards compatibility.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> What Wei meant was merging it into master now so that it will go
>> in
>> >>> the
>> >>>>>>>
>> >>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so
>> that
>> >>> 4.2
>> >>>>>>>
>> >>>>>>> will be released with Read and Write instead of Total IOps.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> This is to make the May 31st feature freeze date. But if the
>> window
>> >>>> moves
>> >>>>>>>
>> >>>>>>> (see other threads) then it won't be necessary to do that.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Wido
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I also completely agree that there is no association between
>> >> network
>> >>>>>>>
>> >>>>>>> and
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> disk I/O.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>
>> >>>>>>> -John
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
>> >>>> wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Hi Wei,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Hi Wido,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Thanks. Good question.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I thought about at the beginning. Finally I decided to ignore the
>> >>>>>>>
>> >>>>>>> difference of read and write mainly because the network throttling
>> >>> did
>> >>>>>>>
>> >>>>>>> not
>> >>>>>>>
>> >>>>>>> care the difference of sent and received bytes as well.
>> >>>>>>>
>> >>>>>>> That reasoning seems odd. Networking and disk I/O completely
>> >>> different.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Disk I/O is much more expensive in most situations then network
>> >>>>>>>
>> >>>>>>> bandwith.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Implementing it will be some copy-paste work. It could be
>> >>>>>>>
>> >>>>>>> implemented in
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> few days. For the deadline of feature freeze, I will implement it
>> >>>>>>>
>> >>>>>>> after
>> >>>>>>>
>> >>>>>>> that , if needed.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> It think it's a feature we can't miss. But if it goes into the 4.2
>> >>>>>>>
>> >>>>>>> window we have to make sure we don't release with only total IOps
>> >> and
>> >>>>>>>
>> >>>>>>> fix
>> >>>>>>>
>> >>>>>>> it in 4.3, that will confuse users.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Wido
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> -Wei
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Hi Wei,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Hi,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I would like to merge disk_io_throttling branch into master.
>> >>>>>>>
>> >>>>>>> If nobody object, I will merge into master in 48 hours.
>> >>>>>>>
>> >>>>>>> The purpose is :
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Virtual machines are running on the same storage device (local
>> >>>>>>>
>> >>>>>>> storage or
>> >>>>>>>
>> >>>>>>> share strage). Because of the rate limitation of device (such as
>> >>>>>>>
>> >>>>>>> iops), if
>> >>>>>>>
>> >>>>>>> one VM has large disk operation, it may affect the disk
>> performance
>> >>>>>>>
>> >>>>>>> of
>> >>>>>>>
>> >>>>>>> other VMs running on the same storage device.
>> >>>>>>>
>> >>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
>> >>>>>>>
>> >>>>>>> VMs.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Looking at the code I see you make no difference between Read and
>> >>>>>>>
>> >>>>>>> Write
>> >>>>>>>
>> >>>>>>> IOps.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Qemu and libvirt support setting both a different rate for Read
>> and
>> >>>>>>>
>> >>>>>>> Write
>> >>>>>>>
>> >>>>>>> IOps which could benefit a lot of users.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> It's also strange, in the polling side you collect both the Read
>> >> and
>> >>>>>>>
>> >>>>>>> Write
>> >>>>>>>
>> >>>>>>> IOps, but on the throttling side you only go for a global value.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Write IOps are usually much more expensive then Read IOps, so it
>> >>>>>>>
>> >>>>>>> seems
>> >>>>>>>
>> >>>>>>> like a valid use-case where that an admin would set a lower value
>> >> for
>> >>>>>>>
>> >>>>>>> write
>> >>>>>>>
>> >>>>>>> IOps vs Read IOps.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Since this only supports KVM at this point I think it would be of
>> >>>>>>>
>> >>>>>>> great
>> >>>>>>>
>> >>>>>>> value to at least have the mechanism in place to support both,
>> >>>>>>>
>> >>>>>>> implementing
>> >>>>>>>
>> >>>>>>> this later would be a lot of work.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> If a hypervisor doesn't support setting different values for read
>> >> and
>> >>>>>>>
>> >>>>>>> write you can always sum both up and set that as the total limit.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Can you explain why you implemented it this way?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Wido
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> The feature includes:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>> >>>>>>>
>> >>>>>>> configuration)
>> >>>>>>>
>> >>>>>>> (2) change the maximum rate of VMs
>> >>>>>>>
>> >>>>>>> (3) limit the disk rate (total bps and iops)
>> >>>>>>>
>> >>>>>>> JIRA ticket: https://issues.apache.org/****
>> >>>>>>>
>> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>> >>>>>>>
>> >>>>>>> jira/browse/CLOUDSTACK-1192<
>> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>> >>>>>>>
>> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>> >>>>>>>
>> >>>>>>> <**
>> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> FS (I will update later) :
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>
>> >>>>>>> <
>> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>
>> >>>>>>> VM+Disk+IO+Throttling<https://****
>> cwiki.apache.org/confluence/****
>> >> <
>> >>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>
>> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>> >>>>>>>
>> >>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >> <
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Merge check list :-
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> * Did you check the branch's RAT execution success?
>> >>>>>>>
>> >>>>>>> Yes
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> * Are there new dependencies introduced?
>> >>>>>>>
>> >>>>>>> No
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> * What automated testing (unit and integration) is included in the
>> >>>>>>>
>> >>>>>>> new
>> >>>>>>>
>> >>>>>>> feature?
>> >>>>>>>
>> >>>>>>> Unit tests are added.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> * What testing has been done to check for potential regressions?
>> >>>>>>>
>> >>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> >>>>>>>
>> >>>>>>> (2) VM operations, including
>> >>>>>>>
>> >>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>> >>>>>>>
>> >>>>>>> (3) Volume operations, including
>> >>>>>>>
>> >>>>>>> Attach, Detach
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> To review the code, you can try
>> >>>>>>>
>> >>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>> >>>>>>>
>> >>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Best regards,
>> >>>>>>>
>> >>>>>>> Wei
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> [1]
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>> >>>>>>>
>> >>>>>>> <
>> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>> >>>>>>>
>> >>>>>>> VM+Disk+IO+Throttling<https://****
>> cwiki.apache.org/confluence/****
>> >> <
>> >>>>>>> http://cwiki.apache.org/confluence/**>
>> >>>>>>>
>> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>> >>>>>>>
>> >>>>>>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> >> <
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>>
>> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> [2] refs/heads/disk_io_throttling
>> >>>>>>>
>> >>>>>>> [3]
>> >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
>> >>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>> >>>>>>>
>> >>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>> >>>>>>>
>> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>> >>>>>>>
>> >>>>>>> <**
>> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>
>> >>>>>>> **<
>> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>> >>>>>>>
>> >>>>>>> <**
>> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> >>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>> >>>>>>>
>> >>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> (**CLOUDSTACK-1301
>> >>>>>>>
>> >>>>>>> - VM Disk I/O Throttling)
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Mike Tutkowski*
>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>> o: 303.746.7302
>> >>>>>> Advancing the way the world uses the
>> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>> *â„¢*
>> >>>>>>
>> >>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the
>> >> cloud<http://solidfire.com/solution/overview/?video=play>
>> >> *â„¢*
>> >>
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
The storage plug-in - by itself - is hypervisor agnostic.
The issue is with the volume-attach logic (in the agent code). The storage
framework calls into the plug-in to have it create a volume as needed, but
when the time comes to attach the volume to a hypervisor, the attach logic
has to be smart enough to recognize it's being invoked on zone-wide storage
(where the volume has just been created) and create, say, a storage
repository (for XenServer) or a datastore (for VMware) to make use of the
volume that was just created.
I've been spending most of my time recently making the attach logic work in
the agent code.
Does that clear it up?
Thanks!
On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <jb...@basho.com> wrote:
> Mike,
>
> Can you explain why the the storage driver is hypervisor specific?
>
> Thanks,
> -John
>
> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Yes, ultimately I would like to support all hypervisors that CloudStack
> > supports. I think I'm just out of time for 4.2 to get KVM in.
> >
> > Right now this plug-in supports XenServer. Depending on what we do with
> > regards to 4.2 feature freeze, I have it working for VMware in my
> sandbox,
> > as well.
> >
> > Also, just to be clear, this is all in regards to Disk Offerings. I plan
> to
> > support Compute Offerings post 4.2.
> >
> >
> > On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <kelcey@bbits.ca
> >wrote:
> >
> >> Is there any plan on supporting KVM in the patch cycle post 4.2?
> >>
> >> ----- Original Message -----
> >> From: "Mike Tutkowski" <mi...@solidfire.com>
> >> To: dev@cloudstack.apache.org
> >> Sent: Monday, June 3, 2013 10:12:32 AM
> >> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>
> >> I agree on merging Wei's feature first, then mine.
> >>
> >> If his feature is for KVM only, then it is a non issue as I don't
> support
> >> KVM in 4.2.
> >>
> >>
> >> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com> wrote:
> >>
> >>> John,
> >>>
> >>> For the billing, as no one works on billing now, users need to
> calculate
> >>> the billing by themselves. They can get the service_offering and
> >>> disk_offering of a VMs and volumes for calculation. Of course it is
> >> better
> >>> to tell user the exact limitation value of individual volume, and
> network
> >>> rate limitation for nics as well. I can work on it later. Do you think
> it
> >>> is a part of I/O throttling?
> >>>
> >>> Sorry my misunstand the second the question.
> >>>
> >>> Agree with what you said about the two features.
> >>>
> >>> -Wei
> >>>
> >>>
> >>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>
> >>>> Wei,
> >>>>
> >>>>
> >>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
> >>>>
> >>>>> Hi John, Mike
> >>>>>
> >>>>> I hope Mike's aswer helps you. I am trying to adding more.
> >>>>>
> >>>>> (1) I think billing should depend on IO statistics rather than IOPS
> >>>>> limitation. Please review disk_io_stat if you have time.
> >> disk_io_stat
> >>>> can
> >>>>> get the IO statistics including bytes/iops read/write for an
> >> individual
> >>>>> virtual machine.
> >>>>
> >>>> Going by the AWS model, customers are billed more for volumes with
> >>>> provisioned IOPS, as well as, for those operations (
> >>>> http://aws.amazon.com/ebs/). I would imagine our users would like
> the
> >>>> option to employ similar cost models. Could an operator implement
> >> such a
> >>>> billing model in the current patch?
> >>>>
> >>>>>
> >>>>> (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
> >>>>> limitation for a running virtual machine through command line.
> >> However,
> >>>>> CloudStack does not support changing the parameters of a created
> >>> offering
> >>>>> (computer offering or disk offering).
> >>>>
> >>>> I meant at the Java interface level. I apologize for being unclear.
> >> Can
> >>>> we more generalize allocation algorithms with a set of interfaces that
> >>>> describe the service guarantees provided by a resource?
> >>>>
> >>>>>
> >>>>> (3) It is a good question. Maybe it is better to commit Mike's patch
> >>>> after
> >>>>> disk_io_throttling as Mike needs to consider the limitation in
> >>> hypervisor
> >>>>> type, I think.
> >>>>
> >>>> I will expand on my thoughts in a later response to Mike regarding the
> >>>> touch points between these two features. I think that
> >> disk_io_throttling
> >>>> will need to be merged before SolidFire, but I think we need closer
> >>>> coordination between the branches (possibly have have solidfire track
> >>>> disk_io_throttling) to coordinate on this issue.
> >>>>
> >>>>>
> >>>>> - Wei
> >>>>>
> >>>>>
> >>>>> 2013/6/3 John Burwell <jb...@basho.com>
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> The things I want to understand are the following:
> >>>>>>
> >>>>>> 1) Is there value in capturing IOPS policies be captured in a
> >> common
> >>>>>> data model (e.g. for billing/usage purposes, expressing offerings).
> >>>>>> 2) Should there be a common interface model for reasoning about
> >> IOP
> >>>>>> provisioning at runtime?
> >>>>>> 3) How are conflicting provisioned IOPS configurations between a
> >>>>>> hypervisor and storage device reconciled? In particular, a scenario
> >>>> where
> >>>>>> is lead to believe (and billed) for more IOPS configured for a VM
> >>> than a
> >>>>>> storage device has been configured to deliver. Another scenario
> >>> could a
> >>>>>> consistent configuration between a VM and a storage device at
> >> creation
> >>>>>> time, but a later modification to storage device introduces logical
> >>>>>> inconsistency.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Hi John,
> >>>>>>
> >>>>>> I believe Wei's feature deals with controlling the max number of
> >> IOPS
> >>>> from
> >>>>>> the hypervisor side.
> >>>>>>
> >>>>>> My feature is focused on controlling IOPS from the storage system
> >>> side.
> >>>>>>
> >>>>>> I hope that helps. :)
> >>>>>>
> >>>>>>
> >>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Wei,
> >>>>>>>
> >>>>>>> My opinion is that no features should be merged until all
> >> functional
> >>>>>>> issues have been resolved and it is ready to turn over to test.
> >>> Until
> >>>>>> the
> >>>>>>> total Ops vs discrete read/write ops issue is addressed and
> >>> re-reviewed
> >>>>>> by
> >>>>>>> Wido, I don't think this criteria has been satisfied.
> >>>>>>>
> >>>>>>> Also, how does this work intersect/compliment the SolidFire patch (
> >>>>>>> https://reviews.apache.org/r/11479/)? As I understand it that
> >> work
> >>> is
> >>>>>>> also involves provisioned IOPS. I would like to ensure we don't
> >>> have a
> >>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
> >>> unnecessarily
> >>>>>>> incompatible.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> -John
> >>>>>>>
> >>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com>
> >> wrote:
> >>>>>>>
> >>>>>>> Wido,
> >>>>>>>
> >>>>>>>
> >>>>>>> Sure. I will change it next week.
> >>>>>>>
> >>>>>>>
> >>>>>>> -Wei
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Wei,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Wido,
> >>>>>>>
> >>>>>>>
> >>>>>>> Exactly. I have pushed the features into master.
> >>>>>>>
> >>>>>>>
> >>>>>>> If anyone object thems for technical reason till Monday, I will
> >>> revert
> >>>>>>>
> >>>>>>> them.
> >>>>>>>
> >>>>>>>
> >>>>>>> For the sake of clarity I just want to mention again that we should
> >>>>>> change
> >>>>>>>
> >>>>>>> the total IOps to R/W IOps asap so that we never release a version
> >>> with
> >>>>>>>
> >>>>>>> only total IOps.
> >>>>>>>
> >>>>>>>
> >>>>>>> You laid the groundwork for the I/O throttling and that's great! We
> >>>>>> should
> >>>>>>>
> >>>>>>> however prevent that we create legacy from day #1.
> >>>>>>>
> >>>>>>>
> >>>>>>> Wido
> >>>>>>>
> >>>>>>>
> >>>>>>> -Wei
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Wido,
> >>>>>>>
> >>>>>>>
> >>>>>>> +1 -- this enhancement must to discretely support read and write
> >>> IOPS.
> >>>>>>>
> >>>>>>> I
> >>>>>>>
> >>>>>>> don't see how it could be fixed later because I don't see how we
> >>>>>>>
> >>>>>>> correctly
> >>>>>>>
> >>>>>>> split total IOPS into read and write. Therefore, we would be stuck
> >>>>>>>
> >>>>>>> with a
> >>>>>>>
> >>>>>>> total unless/until we decided to break backwards compatibility.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> What Wei meant was merging it into master now so that it will go in
> >>> the
> >>>>>>>
> >>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that
> >>> 4.2
> >>>>>>>
> >>>>>>> will be released with Read and Write instead of Total IOps.
> >>>>>>>
> >>>>>>>
> >>>>>>> This is to make the May 31st feature freeze date. But if the window
> >>>> moves
> >>>>>>>
> >>>>>>> (see other threads) then it won't be necessary to do that.
> >>>>>>>
> >>>>>>>
> >>>>>>> Wido
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I also completely agree that there is no association between
> >> network
> >>>>>>>
> >>>>>>> and
> >>>>>>>
> >>>>>>>
> >>>>>>> disk I/O.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> -John
> >>>>>>>
> >>>>>>>
> >>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
> >>>> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Wei,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Wido,
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks. Good question.
> >>>>>>>
> >>>>>>>
> >>>>>>> I thought about at the beginning. Finally I decided to ignore the
> >>>>>>>
> >>>>>>> difference of read and write mainly because the network throttling
> >>> did
> >>>>>>>
> >>>>>>> not
> >>>>>>>
> >>>>>>> care the difference of sent and received bytes as well.
> >>>>>>>
> >>>>>>> That reasoning seems odd. Networking and disk I/O completely
> >>> different.
> >>>>>>>
> >>>>>>>
> >>>>>>> Disk I/O is much more expensive in most situations then network
> >>>>>>>
> >>>>>>> bandwith.
> >>>>>>>
> >>>>>>>
> >>>>>>> Implementing it will be some copy-paste work. It could be
> >>>>>>>
> >>>>>>> implemented in
> >>>>>>>
> >>>>>>>
> >>>>>>> few days. For the deadline of feature freeze, I will implement it
> >>>>>>>
> >>>>>>> after
> >>>>>>>
> >>>>>>> that , if needed.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> It think it's a feature we can't miss. But if it goes into the 4.2
> >>>>>>>
> >>>>>>> window we have to make sure we don't release with only total IOps
> >> and
> >>>>>>>
> >>>>>>> fix
> >>>>>>>
> >>>>>>> it in 4.3, that will confuse users.
> >>>>>>>
> >>>>>>>
> >>>>>>> Wido
> >>>>>>>
> >>>>>>>
> >>>>>>> -Wei
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Wei,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>>
> >>>>>>> I would like to merge disk_io_throttling branch into master.
> >>>>>>>
> >>>>>>> If nobody object, I will merge into master in 48 hours.
> >>>>>>>
> >>>>>>> The purpose is :
> >>>>>>>
> >>>>>>>
> >>>>>>> Virtual machines are running on the same storage device (local
> >>>>>>>
> >>>>>>> storage or
> >>>>>>>
> >>>>>>> share strage). Because of the rate limitation of device (such as
> >>>>>>>
> >>>>>>> iops), if
> >>>>>>>
> >>>>>>> one VM has large disk operation, it may affect the disk performance
> >>>>>>>
> >>>>>>> of
> >>>>>>>
> >>>>>>> other VMs running on the same storage device.
> >>>>>>>
> >>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
> >>>>>>>
> >>>>>>> VMs.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Looking at the code I see you make no difference between Read and
> >>>>>>>
> >>>>>>> Write
> >>>>>>>
> >>>>>>> IOps.
> >>>>>>>
> >>>>>>>
> >>>>>>> Qemu and libvirt support setting both a different rate for Read and
> >>>>>>>
> >>>>>>> Write
> >>>>>>>
> >>>>>>> IOps which could benefit a lot of users.
> >>>>>>>
> >>>>>>>
> >>>>>>> It's also strange, in the polling side you collect both the Read
> >> and
> >>>>>>>
> >>>>>>> Write
> >>>>>>>
> >>>>>>> IOps, but on the throttling side you only go for a global value.
> >>>>>>>
> >>>>>>>
> >>>>>>> Write IOps are usually much more expensive then Read IOps, so it
> >>>>>>>
> >>>>>>> seems
> >>>>>>>
> >>>>>>> like a valid use-case where that an admin would set a lower value
> >> for
> >>>>>>>
> >>>>>>> write
> >>>>>>>
> >>>>>>> IOps vs Read IOps.
> >>>>>>>
> >>>>>>>
> >>>>>>> Since this only supports KVM at this point I think it would be of
> >>>>>>>
> >>>>>>> great
> >>>>>>>
> >>>>>>> value to at least have the mechanism in place to support both,
> >>>>>>>
> >>>>>>> implementing
> >>>>>>>
> >>>>>>> this later would be a lot of work.
> >>>>>>>
> >>>>>>>
> >>>>>>> If a hypervisor doesn't support setting different values for read
> >> and
> >>>>>>>
> >>>>>>> write you can always sum both up and set that as the total limit.
> >>>>>>>
> >>>>>>>
> >>>>>>> Can you explain why you implemented it this way?
> >>>>>>>
> >>>>>>>
> >>>>>>> Wido
> >>>>>>>
> >>>>>>>
> >>>>>>> The feature includes:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
> >>>>>>>
> >>>>>>> configuration)
> >>>>>>>
> >>>>>>> (2) change the maximum rate of VMs
> >>>>>>>
> >>>>>>> (3) limit the disk rate (total bps and iops)
> >>>>>>>
> >>>>>>> JIRA ticket: https://issues.apache.org/****
> >>>>>>>
> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> >>>>>>>
> >>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>
> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>
> >>>>>>> <**
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> FS (I will update later) :
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>
> >>>>>>> <
> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>
> >>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****
> >> <
> >>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>
> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >> <
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Merge check list :-
> >>>>>>>
> >>>>>>>
> >>>>>>> * Did you check the branch's RAT execution success?
> >>>>>>>
> >>>>>>> Yes
> >>>>>>>
> >>>>>>>
> >>>>>>> * Are there new dependencies introduced?
> >>>>>>>
> >>>>>>> No
> >>>>>>>
> >>>>>>>
> >>>>>>> * What automated testing (unit and integration) is included in the
> >>>>>>>
> >>>>>>> new
> >>>>>>>
> >>>>>>> feature?
> >>>>>>>
> >>>>>>> Unit tests are added.
> >>>>>>>
> >>>>>>>
> >>>>>>> * What testing has been done to check for potential regressions?
> >>>>>>>
> >>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>>>>>
> >>>>>>> (2) VM operations, including
> >>>>>>>
> >>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >>>>>>>
> >>>>>>> (3) Volume operations, including
> >>>>>>>
> >>>>>>> Attach, Detach
> >>>>>>>
> >>>>>>>
> >>>>>>> To review the code, you can try
> >>>>>>>
> >>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>
> >>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>
> >>>>>>>
> >>>>>>> Best regards,
> >>>>>>>
> >>>>>>> Wei
> >>>>>>>
> >>>>>>>
> >>>>>>> [1]
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>
> >>>>>>> <
> >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>>>>>
> >>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****
> >> <
> >>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>
> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >>>>>>>
> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >> <
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>
> >>>>>>> [3]
> >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> >>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> >>>>>>>
> >>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>
> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>
> >>>>>>> <**
> >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> >>>>>>>
> >>>>>>> **<
> >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>
> >>>>>>> <**
> >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>
> >>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> (**CLOUDSTACK-1301
> >>>>>>>
> >>>>>>> - VM Disk I/O Throttling)
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *â„¢*
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *â„¢*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
Can you explain why the the storage driver is hypervisor specific?
Thanks,
-John
On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <mi...@solidfire.com> wrote:
> Yes, ultimately I would like to support all hypervisors that CloudStack
> supports. I think I'm just out of time for 4.2 to get KVM in.
>
> Right now this plug-in supports XenServer. Depending on what we do with
> regards to 4.2 feature freeze, I have it working for VMware in my sandbox,
> as well.
>
> Also, just to be clear, this is all in regards to Disk Offerings. I plan to
> support Compute Offerings post 4.2.
>
>
> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <ke...@bbits.ca>wrote:
>
>> Is there any plan on supporting KVM in the patch cycle post 4.2?
>>
>> ----- Original Message -----
>> From: "Mike Tutkowski" <mi...@solidfire.com>
>> To: dev@cloudstack.apache.org
>> Sent: Monday, June 3, 2013 10:12:32 AM
>> Subject: Re: [MERGE] disk_io_throttling to MASTER
>>
>> I agree on merging Wei's feature first, then mine.
>>
>> If his feature is for KVM only, then it is a non issue as I don't support
>> KVM in 4.2.
>>
>>
>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> John,
>>>
>>> For the billing, as no one works on billing now, users need to calculate
>>> the billing by themselves. They can get the service_offering and
>>> disk_offering of a VMs and volumes for calculation. Of course it is
>> better
>>> to tell user the exact limitation value of individual volume, and network
>>> rate limitation for nics as well. I can work on it later. Do you think it
>>> is a part of I/O throttling?
>>>
>>> Sorry my misunstand the second the question.
>>>
>>> Agree with what you said about the two features.
>>>
>>> -Wei
>>>
>>>
>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>
>>>> Wei,
>>>>
>>>>
>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
>>>>
>>>>> Hi John, Mike
>>>>>
>>>>> I hope Mike's aswer helps you. I am trying to adding more.
>>>>>
>>>>> (1) I think billing should depend on IO statistics rather than IOPS
>>>>> limitation. Please review disk_io_stat if you have time.
>> disk_io_stat
>>>> can
>>>>> get the IO statistics including bytes/iops read/write for an
>> individual
>>>>> virtual machine.
>>>>
>>>> Going by the AWS model, customers are billed more for volumes with
>>>> provisioned IOPS, as well as, for those operations (
>>>> http://aws.amazon.com/ebs/). I would imagine our users would like the
>>>> option to employ similar cost models. Could an operator implement
>> such a
>>>> billing model in the current patch?
>>>>
>>>>>
>>>>> (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
>>>>> limitation for a running virtual machine through command line.
>> However,
>>>>> CloudStack does not support changing the parameters of a created
>>> offering
>>>>> (computer offering or disk offering).
>>>>
>>>> I meant at the Java interface level. I apologize for being unclear.
>> Can
>>>> we more generalize allocation algorithms with a set of interfaces that
>>>> describe the service guarantees provided by a resource?
>>>>
>>>>>
>>>>> (3) It is a good question. Maybe it is better to commit Mike's patch
>>>> after
>>>>> disk_io_throttling as Mike needs to consider the limitation in
>>> hypervisor
>>>>> type, I think.
>>>>
>>>> I will expand on my thoughts in a later response to Mike regarding the
>>>> touch points between these two features. I think that
>> disk_io_throttling
>>>> will need to be merged before SolidFire, but I think we need closer
>>>> coordination between the branches (possibly have have solidfire track
>>>> disk_io_throttling) to coordinate on this issue.
>>>>
>>>>>
>>>>> - Wei
>>>>>
>>>>>
>>>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> The things I want to understand are the following:
>>>>>>
>>>>>> 1) Is there value in capturing IOPS policies be captured in a
>> common
>>>>>> data model (e.g. for billing/usage purposes, expressing offerings).
>>>>>> 2) Should there be a common interface model for reasoning about
>> IOP
>>>>>> provisioning at runtime?
>>>>>> 3) How are conflicting provisioned IOPS configurations between a
>>>>>> hypervisor and storage device reconciled? In particular, a scenario
>>>> where
>>>>>> is lead to believe (and billed) for more IOPS configured for a VM
>>> than a
>>>>>> storage device has been configured to deliver. Another scenario
>>> could a
>>>>>> consistent configuration between a VM and a storage device at
>> creation
>>>>>> time, but a later modification to storage device introduces logical
>>>>>> inconsistency.
>>>>>>
>>>>>> Thanks,
>>>>>> -John
>>>>>>
>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> I believe Wei's feature deals with controlling the max number of
>> IOPS
>>>> from
>>>>>> the hypervisor side.
>>>>>>
>>>>>> My feature is focused on controlling IOPS from the storage system
>>> side.
>>>>>>
>>>>>> I hope that helps. :)
>>>>>>
>>>>>>
>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
>>>> wrote:
>>>>>>
>>>>>>> Wei,
>>>>>>>
>>>>>>> My opinion is that no features should be merged until all
>> functional
>>>>>>> issues have been resolved and it is ready to turn over to test.
>>> Until
>>>>>> the
>>>>>>> total Ops vs discrete read/write ops issue is addressed and
>>> re-reviewed
>>>>>> by
>>>>>>> Wido, I don't think this criteria has been satisfied.
>>>>>>>
>>>>>>> Also, how does this work intersect/compliment the SolidFire patch (
>>>>>>> https://reviews.apache.org/r/11479/)? As I understand it that
>> work
>>> is
>>>>>>> also involves provisioned IOPS. I would like to ensure we don't
>>> have a
>>>>>>> scenario where provisioned IOPS in KVM and SolidFire are
>>> unnecessarily
>>>>>>> incompatible.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>>
>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com>
>> wrote:
>>>>>>>
>>>>>>> Wido,
>>>>>>>
>>>>>>>
>>>>>>> Sure. I will change it next week.
>>>>>>>
>>>>>>>
>>>>>>> -Wei
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>>>
>>>>>>>
>>>>>>> Hi Wei,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>>>
>>>>>>>
>>>>>>> Wido,
>>>>>>>
>>>>>>>
>>>>>>> Exactly. I have pushed the features into master.
>>>>>>>
>>>>>>>
>>>>>>> If anyone object thems for technical reason till Monday, I will
>>> revert
>>>>>>>
>>>>>>> them.
>>>>>>>
>>>>>>>
>>>>>>> For the sake of clarity I just want to mention again that we should
>>>>>> change
>>>>>>>
>>>>>>> the total IOps to R/W IOps asap so that we never release a version
>>> with
>>>>>>>
>>>>>>> only total IOps.
>>>>>>>
>>>>>>>
>>>>>>> You laid the groundwork for the I/O throttling and that's great! We
>>>>>> should
>>>>>>>
>>>>>>> however prevent that we create legacy from day #1.
>>>>>>>
>>>>>>>
>>>>>>> Wido
>>>>>>>
>>>>>>>
>>>>>>> -Wei
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>
>>>>>>>
>>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>>>
>>>>>>>
>>>>>>> Wido,
>>>>>>>
>>>>>>>
>>>>>>> +1 -- this enhancement must to discretely support read and write
>>> IOPS.
>>>>>>>
>>>>>>> I
>>>>>>>
>>>>>>> don't see how it could be fixed later because I don't see how we
>>>>>>>
>>>>>>> correctly
>>>>>>>
>>>>>>> split total IOPS into read and write. Therefore, we would be stuck
>>>>>>>
>>>>>>> with a
>>>>>>>
>>>>>>> total unless/until we decided to break backwards compatibility.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> What Wei meant was merging it into master now so that it will go in
>>> the
>>>>>>>
>>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that
>>> 4.2
>>>>>>>
>>>>>>> will be released with Read and Write instead of Total IOps.
>>>>>>>
>>>>>>>
>>>>>>> This is to make the May 31st feature freeze date. But if the window
>>>> moves
>>>>>>>
>>>>>>> (see other threads) then it won't be necessary to do that.
>>>>>>>
>>>>>>>
>>>>>>> Wido
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I also completely agree that there is no association between
>> network
>>>>>>>
>>>>>>> and
>>>>>>>
>>>>>>>
>>>>>>> disk I/O.
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -John
>>>>>>>
>>>>>>>
>>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi Wei,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi Wido,
>>>>>>>
>>>>>>>
>>>>>>> Thanks. Good question.
>>>>>>>
>>>>>>>
>>>>>>> I thought about at the beginning. Finally I decided to ignore the
>>>>>>>
>>>>>>> difference of read and write mainly because the network throttling
>>> did
>>>>>>>
>>>>>>> not
>>>>>>>
>>>>>>> care the difference of sent and received bytes as well.
>>>>>>>
>>>>>>> That reasoning seems odd. Networking and disk I/O completely
>>> different.
>>>>>>>
>>>>>>>
>>>>>>> Disk I/O is much more expensive in most situations then network
>>>>>>>
>>>>>>> bandwith.
>>>>>>>
>>>>>>>
>>>>>>> Implementing it will be some copy-paste work. It could be
>>>>>>>
>>>>>>> implemented in
>>>>>>>
>>>>>>>
>>>>>>> few days. For the deadline of feature freeze, I will implement it
>>>>>>>
>>>>>>> after
>>>>>>>
>>>>>>> that , if needed.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>>>>>>
>>>>>>> window we have to make sure we don't release with only total IOps
>> and
>>>>>>>
>>>>>>> fix
>>>>>>>
>>>>>>> it in 4.3, that will confuse users.
>>>>>>>
>>>>>>>
>>>>>>> Wido
>>>>>>>
>>>>>>>
>>>>>>> -Wei
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>>
>>>>>>>
>>>>>>> Hi Wei,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>
>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>
>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>
>>>>>>> The purpose is :
>>>>>>>
>>>>>>>
>>>>>>> Virtual machines are running on the same storage device (local
>>>>>>>
>>>>>>> storage or
>>>>>>>
>>>>>>> share strage). Because of the rate limitation of device (such as
>>>>>>>
>>>>>>> iops), if
>>>>>>>
>>>>>>> one VM has large disk operation, it may affect the disk performance
>>>>>>>
>>>>>>> of
>>>>>>>
>>>>>>> other VMs running on the same storage device.
>>>>>>>
>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
>>>>>>>
>>>>>>> VMs.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Looking at the code I see you make no difference between Read and
>>>>>>>
>>>>>>> Write
>>>>>>>
>>>>>>> IOps.
>>>>>>>
>>>>>>>
>>>>>>> Qemu and libvirt support setting both a different rate for Read and
>>>>>>>
>>>>>>> Write
>>>>>>>
>>>>>>> IOps which could benefit a lot of users.
>>>>>>>
>>>>>>>
>>>>>>> It's also strange, in the polling side you collect both the Read
>> and
>>>>>>>
>>>>>>> Write
>>>>>>>
>>>>>>> IOps, but on the throttling side you only go for a global value.
>>>>>>>
>>>>>>>
>>>>>>> Write IOps are usually much more expensive then Read IOps, so it
>>>>>>>
>>>>>>> seems
>>>>>>>
>>>>>>> like a valid use-case where that an admin would set a lower value
>> for
>>>>>>>
>>>>>>> write
>>>>>>>
>>>>>>> IOps vs Read IOps.
>>>>>>>
>>>>>>>
>>>>>>> Since this only supports KVM at this point I think it would be of
>>>>>>>
>>>>>>> great
>>>>>>>
>>>>>>> value to at least have the mechanism in place to support both,
>>>>>>>
>>>>>>> implementing
>>>>>>>
>>>>>>> this later would be a lot of work.
>>>>>>>
>>>>>>>
>>>>>>> If a hypervisor doesn't support setting different values for read
>> and
>>>>>>>
>>>>>>> write you can always sum both up and set that as the total limit.
>>>>>>>
>>>>>>>
>>>>>>> Can you explain why you implemented it this way?
>>>>>>>
>>>>>>>
>>>>>>> Wido
>>>>>>>
>>>>>>>
>>>>>>> The feature includes:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>>>
>>>>>>> configuration)
>>>>>>>
>>>>>>> (2) change the maximum rate of VMs
>>>>>>>
>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>
>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>
>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>>>>>>>
>>>>>>> jira/browse/CLOUDSTACK-1192<
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>
>>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>
>>>>>>> <**
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> FS (I will update later) :
>>>>>>>
>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>
>>>>>>> <
>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>
>>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****
>> <
>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>
>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> <
>>>>>>>
>>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Merge check list :-
>>>>>>>
>>>>>>>
>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>
>>>>>>> Yes
>>>>>>>
>>>>>>>
>>>>>>> * Are there new dependencies introduced?
>>>>>>>
>>>>>>> No
>>>>>>>
>>>>>>>
>>>>>>> * What automated testing (unit and integration) is included in the
>>>>>>>
>>>>>>> new
>>>>>>>
>>>>>>> feature?
>>>>>>>
>>>>>>> Unit tests are added.
>>>>>>>
>>>>>>>
>>>>>>> * What testing has been done to check for potential regressions?
>>>>>>>
>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>
>>>>>>> (2) VM operations, including
>>>>>>>
>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>>>
>>>>>>> (3) Volume operations, including
>>>>>>>
>>>>>>> Attach, Detach
>>>>>>>
>>>>>>>
>>>>>>> To review the code, you can try
>>>>>>>
>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>
>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Wei
>>>>>>>
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>
>>>>>>> <
>>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>
>>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****
>> <
>>>>>>> http://cwiki.apache.org/confluence/**>
>>>>>>>
>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>>>>
>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
>> <
>>>>>>>
>>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>
>>>>>>> [3]
>>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>>>>>
>>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>
>>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>
>>>>>>> <**
>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>>>>>
>>>>>>> **<
>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>
>>>>>>> <**
>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>
>>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> (**CLOUDSTACK-1301
>>>>>>>
>>>>>>> - VM Disk I/O Throttling)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *â„¢*
>>>>>>
>>>>
>>>>
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yes, ultimately I would like to support all hypervisors that CloudStack
supports. I think I'm just out of time for 4.2 to get KVM in.
Right now this plug-in supports XenServer. Depending on what we do with
regards to 4.2 feature freeze, I have it working for VMware in my sandbox,
as well.
Also, just to be clear, this is all in regards to Disk Offerings. I plan to
support Compute Offerings post 4.2.
On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <ke...@bbits.ca>wrote:
> Is there any plan on supporting KVM in the patch cycle post 4.2?
>
> ----- Original Message -----
> From: "Mike Tutkowski" <mi...@solidfire.com>
> To: dev@cloudstack.apache.org
> Sent: Monday, June 3, 2013 10:12:32 AM
> Subject: Re: [MERGE] disk_io_throttling to MASTER
>
> I agree on merging Wei's feature first, then mine.
>
> If his feature is for KVM only, then it is a non issue as I don't support
> KVM in 4.2.
>
>
> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com> wrote:
>
> > John,
> >
> > For the billing, as no one works on billing now, users need to calculate
> > the billing by themselves. They can get the service_offering and
> > disk_offering of a VMs and volumes for calculation. Of course it is
> better
> > to tell user the exact limitation value of individual volume, and network
> > rate limitation for nics as well. I can work on it later. Do you think it
> > is a part of I/O throttling?
> >
> > Sorry my misunstand the second the question.
> >
> > Agree with what you said about the two features.
> >
> > -Wei
> >
> >
> > 2013/6/3 John Burwell <jb...@basho.com>
> >
> > > Wei,
> > >
> > >
> > > On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
> > >
> > > > Hi John, Mike
> > > >
> > > > I hope Mike's aswer helps you. I am trying to adding more.
> > > >
> > > > (1) I think billing should depend on IO statistics rather than IOPS
> > > > limitation. Please review disk_io_stat if you have time.
> disk_io_stat
> > > can
> > > > get the IO statistics including bytes/iops read/write for an
> individual
> > > > virtual machine.
> > >
> > > Going by the AWS model, customers are billed more for volumes with
> > > provisioned IOPS, as well as, for those operations (
> > > http://aws.amazon.com/ebs/). I would imagine our users would like the
> > > option to employ similar cost models. Could an operator implement
> such a
> > > billing model in the current patch?
> > >
> > > >
> > > > (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
> > > > limitation for a running virtual machine through command line.
> However,
> > > > CloudStack does not support changing the parameters of a created
> > offering
> > > > (computer offering or disk offering).
> > >
> > > I meant at the Java interface level. I apologize for being unclear.
> Can
> > > we more generalize allocation algorithms with a set of interfaces that
> > > describe the service guarantees provided by a resource?
> > >
> > > >
> > > > (3) It is a good question. Maybe it is better to commit Mike's patch
> > > after
> > > > disk_io_throttling as Mike needs to consider the limitation in
> > hypervisor
> > > > type, I think.
> > >
> > > I will expand on my thoughts in a later response to Mike regarding the
> > > touch points between these two features. I think that
> disk_io_throttling
> > > will need to be merged before SolidFire, but I think we need closer
> > > coordination between the branches (possibly have have solidfire track
> > > disk_io_throttling) to coordinate on this issue.
> > >
> > > >
> > > > - Wei
> > > >
> > > >
> > > > 2013/6/3 John Burwell <jb...@basho.com>
> > > >
> > > >> Mike,
> > > >>
> > > >> The things I want to understand are the following:
> > > >>
> > > >> 1) Is there value in capturing IOPS policies be captured in a
> common
> > > >> data model (e.g. for billing/usage purposes, expressing offerings).
> > > >> 2) Should there be a common interface model for reasoning about
> IOP
> > > >> provisioning at runtime?
> > > >> 3) How are conflicting provisioned IOPS configurations between a
> > > >> hypervisor and storage device reconciled? In particular, a scenario
> > > where
> > > >> is lead to believe (and billed) for more IOPS configured for a VM
> > than a
> > > >> storage device has been configured to deliver. Another scenario
> > could a
> > > >> consistent configuration between a VM and a storage device at
> creation
> > > >> time, but a later modification to storage device introduces logical
> > > >> inconsistency.
> > > >>
> > > >> Thanks,
> > > >> -John
> > > >>
> > > >> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> > > mike.tutkowski@solidfire.com>
> > > >> wrote:
> > > >>
> > > >> Hi John,
> > > >>
> > > >> I believe Wei's feature deals with controlling the max number of
> IOPS
> > > from
> > > >> the hypervisor side.
> > > >>
> > > >> My feature is focused on controlling IOPS from the storage system
> > side.
> > > >>
> > > >> I hope that helps. :)
> > > >>
> > > >>
> > > >> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
> > > wrote:
> > > >>
> > > >>> Wei,
> > > >>>
> > > >>> My opinion is that no features should be merged until all
> functional
> > > >>> issues have been resolved and it is ready to turn over to test.
> > Until
> > > >> the
> > > >>> total Ops vs discrete read/write ops issue is addressed and
> > re-reviewed
> > > >> by
> > > >>> Wido, I don't think this criteria has been satisfied.
> > > >>>
> > > >>> Also, how does this work intersect/compliment the SolidFire patch (
> > > >>> https://reviews.apache.org/r/11479/)? As I understand it that
> work
> > is
> > > >>> also involves provisioned IOPS. I would like to ensure we don't
> > have a
> > > >>> scenario where provisioned IOPS in KVM and SolidFire are
> > unnecessarily
> > > >>> incompatible.
> > > >>>
> > > >>> Thanks,
> > > >>> -John
> > > >>>
> > > >>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com>
> wrote:
> > > >>>
> > > >>> Wido,
> > > >>>
> > > >>>
> > > >>> Sure. I will change it next week.
> > > >>>
> > > >>>
> > > >>> -Wei
> > > >>>
> > > >>>
> > > >>>
> > > >>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> > > >>>
> > > >>>
> > > >>> Hi Wei,
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> > > >>>
> > > >>>
> > > >>> Wido,
> > > >>>
> > > >>>
> > > >>> Exactly. I have pushed the features into master.
> > > >>>
> > > >>>
> > > >>> If anyone object thems for technical reason till Monday, I will
> > revert
> > > >>>
> > > >>> them.
> > > >>>
> > > >>>
> > > >>> For the sake of clarity I just want to mention again that we should
> > > >> change
> > > >>>
> > > >>> the total IOps to R/W IOps asap so that we never release a version
> > with
> > > >>>
> > > >>> only total IOps.
> > > >>>
> > > >>>
> > > >>> You laid the groundwork for the I/O throttling and that's great! We
> > > >> should
> > > >>>
> > > >>> however prevent that we create legacy from day #1.
> > > >>>
> > > >>>
> > > >>> Wido
> > > >>>
> > > >>>
> > > >>> -Wei
> > > >>>
> > > >>>
> > > >>>
> > > >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > > >>>
> > > >>>
> > > >>> On 05/31/2013 03:59 PM, John Burwell wrote:
> > > >>>
> > > >>>
> > > >>> Wido,
> > > >>>
> > > >>>
> > > >>> +1 -- this enhancement must to discretely support read and write
> > IOPS.
> > > >>>
> > > >>> I
> > > >>>
> > > >>> don't see how it could be fixed later because I don't see how we
> > > >>>
> > > >>> correctly
> > > >>>
> > > >>> split total IOPS into read and write. Therefore, we would be stuck
> > > >>>
> > > >>> with a
> > > >>>
> > > >>> total unless/until we decided to break backwards compatibility.
> > > >>>
> > > >>>
> > > >>>
> > > >>> What Wei meant was merging it into master now so that it will go in
> > the
> > > >>>
> > > >>> 4.2 branch and add Read / Write IOps before the 4.2 release so that
> > 4.2
> > > >>>
> > > >>> will be released with Read and Write instead of Total IOps.
> > > >>>
> > > >>>
> > > >>> This is to make the May 31st feature freeze date. But if the window
> > > moves
> > > >>>
> > > >>> (see other threads) then it won't be necessary to do that.
> > > >>>
> > > >>>
> > > >>> Wido
> > > >>>
> > > >>>
> > > >>>
> > > >>> I also completely agree that there is no association between
> network
> > > >>>
> > > >>> and
> > > >>>
> > > >>>
> > > >>> disk I/O.
> > > >>>
> > > >>>
> > > >>> Thanks,
> > > >>>
> > > >>> -John
> > > >>>
> > > >>>
> > > >>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
> > > wrote:
> > > >>>
> > > >>>
> > > >>> Hi Wei,
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> > > >>>
> > > >>>
> > > >>> Hi Wido,
> > > >>>
> > > >>>
> > > >>> Thanks. Good question.
> > > >>>
> > > >>>
> > > >>> I thought about at the beginning. Finally I decided to ignore the
> > > >>>
> > > >>> difference of read and write mainly because the network throttling
> > did
> > > >>>
> > > >>> not
> > > >>>
> > > >>> care the difference of sent and received bytes as well.
> > > >>>
> > > >>> That reasoning seems odd. Networking and disk I/O completely
> > different.
> > > >>>
> > > >>>
> > > >>> Disk I/O is much more expensive in most situations then network
> > > >>>
> > > >>> bandwith.
> > > >>>
> > > >>>
> > > >>> Implementing it will be some copy-paste work. It could be
> > > >>>
> > > >>> implemented in
> > > >>>
> > > >>>
> > > >>> few days. For the deadline of feature freeze, I will implement it
> > > >>>
> > > >>> after
> > > >>>
> > > >>> that , if needed.
> > > >>>
> > > >>>
> > > >>>
> > > >>> It think it's a feature we can't miss. But if it goes into the 4.2
> > > >>>
> > > >>> window we have to make sure we don't release with only total IOps
> and
> > > >>>
> > > >>> fix
> > > >>>
> > > >>> it in 4.3, that will confuse users.
> > > >>>
> > > >>>
> > > >>> Wido
> > > >>>
> > > >>>
> > > >>> -Wei
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > > >>>
> > > >>>
> > > >>> Hi Wei,
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> > > >>>
> > > >>>
> > > >>> Hi,
> > > >>>
> > > >>>
> > > >>> I would like to merge disk_io_throttling branch into master.
> > > >>>
> > > >>> If nobody object, I will merge into master in 48 hours.
> > > >>>
> > > >>> The purpose is :
> > > >>>
> > > >>>
> > > >>> Virtual machines are running on the same storage device (local
> > > >>>
> > > >>> storage or
> > > >>>
> > > >>> share strage). Because of the rate limitation of device (such as
> > > >>>
> > > >>> iops), if
> > > >>>
> > > >>> one VM has large disk operation, it may affect the disk performance
> > > >>>
> > > >>> of
> > > >>>
> > > >>> other VMs running on the same storage device.
> > > >>>
> > > >>> It is neccesary to set the maximum rate and limit the disk I/O of
> > > >>>
> > > >>> VMs.
> > > >>>
> > > >>>
> > > >>>
> > > >>> Looking at the code I see you make no difference between Read and
> > > >>>
> > > >>> Write
> > > >>>
> > > >>> IOps.
> > > >>>
> > > >>>
> > > >>> Qemu and libvirt support setting both a different rate for Read and
> > > >>>
> > > >>> Write
> > > >>>
> > > >>> IOps which could benefit a lot of users.
> > > >>>
> > > >>>
> > > >>> It's also strange, in the polling side you collect both the Read
> and
> > > >>>
> > > >>> Write
> > > >>>
> > > >>> IOps, but on the throttling side you only go for a global value.
> > > >>>
> > > >>>
> > > >>> Write IOps are usually much more expensive then Read IOps, so it
> > > >>>
> > > >>> seems
> > > >>>
> > > >>> like a valid use-case where that an admin would set a lower value
> for
> > > >>>
> > > >>> write
> > > >>>
> > > >>> IOps vs Read IOps.
> > > >>>
> > > >>>
> > > >>> Since this only supports KVM at this point I think it would be of
> > > >>>
> > > >>> great
> > > >>>
> > > >>> value to at least have the mechanism in place to support both,
> > > >>>
> > > >>> implementing
> > > >>>
> > > >>> this later would be a lot of work.
> > > >>>
> > > >>>
> > > >>> If a hypervisor doesn't support setting different values for read
> and
> > > >>>
> > > >>> write you can always sum both up and set that as the total limit.
> > > >>>
> > > >>>
> > > >>> Can you explain why you implemented it this way?
> > > >>>
> > > >>>
> > > >>> Wido
> > > >>>
> > > >>>
> > > >>> The feature includes:
> > > >>>
> > > >>>
> > > >>>
> > > >>> (1) set the maximum rate of VMs (in disk_offering, and global
> > > >>>
> > > >>> configuration)
> > > >>>
> > > >>> (2) change the maximum rate of VMs
> > > >>>
> > > >>> (3) limit the disk rate (total bps and iops)
> > > >>>
> > > >>> JIRA ticket: https://issues.apache.org/****
> > > >>>
> > > >>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> > > >>>
> > > >>> jira/browse/CLOUDSTACK-1192<
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> > > >>>
> > > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> > > >>>
> > > >>> <**
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> > > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > > >>>
> > > >>>
> > > >>>
> > > >>> FS (I will update later) :
> > > >>>
> > > >>>
> > > >>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > > >>>
> > > >>> <
> > > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> > > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > > >>>
> > > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****
> <
> > > >>> http://cwiki.apache.org/confluence/**>
> > > >>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > > >>>
> > > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> <
> > > >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> Merge check list :-
> > > >>>
> > > >>>
> > > >>> * Did you check the branch's RAT execution success?
> > > >>>
> > > >>> Yes
> > > >>>
> > > >>>
> > > >>> * Are there new dependencies introduced?
> > > >>>
> > > >>> No
> > > >>>
> > > >>>
> > > >>> * What automated testing (unit and integration) is included in the
> > > >>>
> > > >>> new
> > > >>>
> > > >>> feature?
> > > >>>
> > > >>> Unit tests are added.
> > > >>>
> > > >>>
> > > >>> * What testing has been done to check for potential regressions?
> > > >>>
> > > >>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> > > >>>
> > > >>> (2) VM operations, including
> > > >>>
> > > >>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> > > >>>
> > > >>> (3) Volume operations, including
> > > >>>
> > > >>> Attach, Detach
> > > >>>
> > > >>>
> > > >>> To review the code, you can try
> > > >>>
> > > >>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> > > >>>
> > > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > > >>>
> > > >>>
> > > >>> Best regards,
> > > >>>
> > > >>> Wei
> > > >>>
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>>
> > > >>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > > >>>
> > > >>> <
> > > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> > > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > > >>>
> > > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****
> <
> > > >>> http://cwiki.apache.org/confluence/**>
> > > >>>
> > > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > > >>>
> > > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> <
> > > >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> [2] refs/heads/disk_io_throttling
> > > >>>
> > > >>> [3]
> > > >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> > > >> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> > > >>>
> > > >>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> > > >>>
> > > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> > > >>>
> > > >>> <**
> > > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> > > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > > >>>
> > > >>>
> > > >>>
> > > >>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> > > >>>
> > > >>> **<
> > > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> > > >> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> > > >>>
> > > >>> <**
> > > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> > > >> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> > > >>>
> > > >>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > > >>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > > >>>
> > > >>>
> > > >>>
> > > >>> (**CLOUDSTACK-1301
> > > >>>
> > > >>> - VM Disk I/O Throttling)
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> *Mike Tutkowski*
> > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >> e: mike.tutkowski@solidfire.com
> > > >> o: 303.746.7302
> > > >> Advancing the way the world uses the
> > > >> cloud<http://solidfire.com/solution/overview/?video=play>
> > > >> *â„¢*
> > > >>
> > >
> > >
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Kelcey Jamison Damage <ke...@bbits.ca>.
Is there any plan on supporting KVM in the patch cycle post 4.2?
----- Original Message -----
From: "Mike Tutkowski" <mi...@solidfire.com>
To: dev@cloudstack.apache.org
Sent: Monday, June 3, 2013 10:12:32 AM
Subject: Re: [MERGE] disk_io_throttling to MASTER
I agree on merging Wei's feature first, then mine.
If his feature is for KVM only, then it is a non issue as I don't support
KVM in 4.2.
On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com> wrote:
> John,
>
> For the billing, as no one works on billing now, users need to calculate
> the billing by themselves. They can get the service_offering and
> disk_offering of a VMs and volumes for calculation. Of course it is better
> to tell user the exact limitation value of individual volume, and network
> rate limitation for nics as well. I can work on it later. Do you think it
> is a part of I/O throttling?
>
> Sorry my misunstand the second the question.
>
> Agree with what you said about the two features.
>
> -Wei
>
>
> 2013/6/3 John Burwell <jb...@basho.com>
>
> > Wei,
> >
> >
> > On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
> >
> > > Hi John, Mike
> > >
> > > I hope Mike's aswer helps you. I am trying to adding more.
> > >
> > > (1) I think billing should depend on IO statistics rather than IOPS
> > > limitation. Please review disk_io_stat if you have time. disk_io_stat
> > can
> > > get the IO statistics including bytes/iops read/write for an individual
> > > virtual machine.
> >
> > Going by the AWS model, customers are billed more for volumes with
> > provisioned IOPS, as well as, for those operations (
> > http://aws.amazon.com/ebs/). I would imagine our users would like the
> > option to employ similar cost models. Could an operator implement such a
> > billing model in the current patch?
> >
> > >
> > > (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
> > > limitation for a running virtual machine through command line. However,
> > > CloudStack does not support changing the parameters of a created
> offering
> > > (computer offering or disk offering).
> >
> > I meant at the Java interface level. I apologize for being unclear. Can
> > we more generalize allocation algorithms with a set of interfaces that
> > describe the service guarantees provided by a resource?
> >
> > >
> > > (3) It is a good question. Maybe it is better to commit Mike's patch
> > after
> > > disk_io_throttling as Mike needs to consider the limitation in
> hypervisor
> > > type, I think.
> >
> > I will expand on my thoughts in a later response to Mike regarding the
> > touch points between these two features. I think that disk_io_throttling
> > will need to be merged before SolidFire, but I think we need closer
> > coordination between the branches (possibly have have solidfire track
> > disk_io_throttling) to coordinate on this issue.
> >
> > >
> > > - Wei
> > >
> > >
> > > 2013/6/3 John Burwell <jb...@basho.com>
> > >
> > >> Mike,
> > >>
> > >> The things I want to understand are the following:
> > >>
> > >> 1) Is there value in capturing IOPS policies be captured in a common
> > >> data model (e.g. for billing/usage purposes, expressing offerings).
> > >> 2) Should there be a common interface model for reasoning about IOP
> > >> provisioning at runtime?
> > >> 3) How are conflicting provisioned IOPS configurations between a
> > >> hypervisor and storage device reconciled? In particular, a scenario
> > where
> > >> is lead to believe (and billed) for more IOPS configured for a VM
> than a
> > >> storage device has been configured to deliver. Another scenario
> could a
> > >> consistent configuration between a VM and a storage device at creation
> > >> time, but a later modification to storage device introduces logical
> > >> inconsistency.
> > >>
> > >> Thanks,
> > >> -John
> > >>
> > >> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > >> wrote:
> > >>
> > >> Hi John,
> > >>
> > >> I believe Wei's feature deals with controlling the max number of IOPS
> > from
> > >> the hypervisor side.
> > >>
> > >> My feature is focused on controlling IOPS from the storage system
> side.
> > >>
> > >> I hope that helps. :)
> > >>
> > >>
> > >> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
> > wrote:
> > >>
> > >>> Wei,
> > >>>
> > >>> My opinion is that no features should be merged until all functional
> > >>> issues have been resolved and it is ready to turn over to test.
> Until
> > >> the
> > >>> total Ops vs discrete read/write ops issue is addressed and
> re-reviewed
> > >> by
> > >>> Wido, I don't think this criteria has been satisfied.
> > >>>
> > >>> Also, how does this work intersect/compliment the SolidFire patch (
> > >>> https://reviews.apache.org/r/11479/)? As I understand it that work
> is
> > >>> also involves provisioned IOPS. I would like to ensure we don't
> have a
> > >>> scenario where provisioned IOPS in KVM and SolidFire are
> unnecessarily
> > >>> incompatible.
> > >>>
> > >>> Thanks,
> > >>> -John
> > >>>
> > >>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
> > >>>
> > >>> Wido,
> > >>>
> > >>>
> > >>> Sure. I will change it next week.
> > >>>
> > >>>
> > >>> -Wei
> > >>>
> > >>>
> > >>>
> > >>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> > >>>
> > >>>
> > >>> Hi Wei,
> > >>>
> > >>>
> > >>>
> > >>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> > >>>
> > >>>
> > >>> Wido,
> > >>>
> > >>>
> > >>> Exactly. I have pushed the features into master.
> > >>>
> > >>>
> > >>> If anyone object thems for technical reason till Monday, I will
> revert
> > >>>
> > >>> them.
> > >>>
> > >>>
> > >>> For the sake of clarity I just want to mention again that we should
> > >> change
> > >>>
> > >>> the total IOps to R/W IOps asap so that we never release a version
> with
> > >>>
> > >>> only total IOps.
> > >>>
> > >>>
> > >>> You laid the groundwork for the I/O throttling and that's great! We
> > >> should
> > >>>
> > >>> however prevent that we create legacy from day #1.
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>> -Wei
> > >>>
> > >>>
> > >>>
> > >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > >>>
> > >>>
> > >>> On 05/31/2013 03:59 PM, John Burwell wrote:
> > >>>
> > >>>
> > >>> Wido,
> > >>>
> > >>>
> > >>> +1 -- this enhancement must to discretely support read and write
> IOPS.
> > >>>
> > >>> I
> > >>>
> > >>> don't see how it could be fixed later because I don't see how we
> > >>>
> > >>> correctly
> > >>>
> > >>> split total IOPS into read and write. Therefore, we would be stuck
> > >>>
> > >>> with a
> > >>>
> > >>> total unless/until we decided to break backwards compatibility.
> > >>>
> > >>>
> > >>>
> > >>> What Wei meant was merging it into master now so that it will go in
> the
> > >>>
> > >>> 4.2 branch and add Read / Write IOps before the 4.2 release so that
> 4.2
> > >>>
> > >>> will be released with Read and Write instead of Total IOps.
> > >>>
> > >>>
> > >>> This is to make the May 31st feature freeze date. But if the window
> > moves
> > >>>
> > >>> (see other threads) then it won't be necessary to do that.
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>>
> > >>> I also completely agree that there is no association between network
> > >>>
> > >>> and
> > >>>
> > >>>
> > >>> disk I/O.
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>> -John
> > >>>
> > >>>
> > >>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
> > wrote:
> > >>>
> > >>>
> > >>> Hi Wei,
> > >>>
> > >>>
> > >>>
> > >>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> > >>>
> > >>>
> > >>> Hi Wido,
> > >>>
> > >>>
> > >>> Thanks. Good question.
> > >>>
> > >>>
> > >>> I thought about at the beginning. Finally I decided to ignore the
> > >>>
> > >>> difference of read and write mainly because the network throttling
> did
> > >>>
> > >>> not
> > >>>
> > >>> care the difference of sent and received bytes as well.
> > >>>
> > >>> That reasoning seems odd. Networking and disk I/O completely
> different.
> > >>>
> > >>>
> > >>> Disk I/O is much more expensive in most situations then network
> > >>>
> > >>> bandwith.
> > >>>
> > >>>
> > >>> Implementing it will be some copy-paste work. It could be
> > >>>
> > >>> implemented in
> > >>>
> > >>>
> > >>> few days. For the deadline of feature freeze, I will implement it
> > >>>
> > >>> after
> > >>>
> > >>> that , if needed.
> > >>>
> > >>>
> > >>>
> > >>> It think it's a feature we can't miss. But if it goes into the 4.2
> > >>>
> > >>> window we have to make sure we don't release with only total IOps and
> > >>>
> > >>> fix
> > >>>
> > >>> it in 4.3, that will confuse users.
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>> -Wei
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > >>>
> > >>>
> > >>> Hi Wei,
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> > >>>
> > >>>
> > >>> Hi,
> > >>>
> > >>>
> > >>> I would like to merge disk_io_throttling branch into master.
> > >>>
> > >>> If nobody object, I will merge into master in 48 hours.
> > >>>
> > >>> The purpose is :
> > >>>
> > >>>
> > >>> Virtual machines are running on the same storage device (local
> > >>>
> > >>> storage or
> > >>>
> > >>> share strage). Because of the rate limitation of device (such as
> > >>>
> > >>> iops), if
> > >>>
> > >>> one VM has large disk operation, it may affect the disk performance
> > >>>
> > >>> of
> > >>>
> > >>> other VMs running on the same storage device.
> > >>>
> > >>> It is neccesary to set the maximum rate and limit the disk I/O of
> > >>>
> > >>> VMs.
> > >>>
> > >>>
> > >>>
> > >>> Looking at the code I see you make no difference between Read and
> > >>>
> > >>> Write
> > >>>
> > >>> IOps.
> > >>>
> > >>>
> > >>> Qemu and libvirt support setting both a different rate for Read and
> > >>>
> > >>> Write
> > >>>
> > >>> IOps which could benefit a lot of users.
> > >>>
> > >>>
> > >>> It's also strange, in the polling side you collect both the Read and
> > >>>
> > >>> Write
> > >>>
> > >>> IOps, but on the throttling side you only go for a global value.
> > >>>
> > >>>
> > >>> Write IOps are usually much more expensive then Read IOps, so it
> > >>>
> > >>> seems
> > >>>
> > >>> like a valid use-case where that an admin would set a lower value for
> > >>>
> > >>> write
> > >>>
> > >>> IOps vs Read IOps.
> > >>>
> > >>>
> > >>> Since this only supports KVM at this point I think it would be of
> > >>>
> > >>> great
> > >>>
> > >>> value to at least have the mechanism in place to support both,
> > >>>
> > >>> implementing
> > >>>
> > >>> this later would be a lot of work.
> > >>>
> > >>>
> > >>> If a hypervisor doesn't support setting different values for read and
> > >>>
> > >>> write you can always sum both up and set that as the total limit.
> > >>>
> > >>>
> > >>> Can you explain why you implemented it this way?
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>> The feature includes:
> > >>>
> > >>>
> > >>>
> > >>> (1) set the maximum rate of VMs (in disk_offering, and global
> > >>>
> > >>> configuration)
> > >>>
> > >>> (2) change the maximum rate of VMs
> > >>>
> > >>> (3) limit the disk rate (total bps and iops)
> > >>>
> > >>> JIRA ticket: https://issues.apache.org/****
> > >>>
> > >>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> > >>>
> > >>> jira/browse/CLOUDSTACK-1192<
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> > >>>
> > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> > >>>
> > >>> <**
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > >>>
> > >>>
> > >>>
> > >>> FS (I will update later) :
> > >>>
> > >>>
> > >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > >>>
> > >>> <
> > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > >>>
> > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > >>> http://cwiki.apache.org/confluence/**>
> > >>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > >>>
> > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> Merge check list :-
> > >>>
> > >>>
> > >>> * Did you check the branch's RAT execution success?
> > >>>
> > >>> Yes
> > >>>
> > >>>
> > >>> * Are there new dependencies introduced?
> > >>>
> > >>> No
> > >>>
> > >>>
> > >>> * What automated testing (unit and integration) is included in the
> > >>>
> > >>> new
> > >>>
> > >>> feature?
> > >>>
> > >>> Unit tests are added.
> > >>>
> > >>>
> > >>> * What testing has been done to check for potential regressions?
> > >>>
> > >>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> > >>>
> > >>> (2) VM operations, including
> > >>>
> > >>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> > >>>
> > >>> (3) Volume operations, including
> > >>>
> > >>> Attach, Detach
> > >>>
> > >>>
> > >>> To review the code, you can try
> > >>>
> > >>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> > >>>
> > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >>>
> > >>>
> > >>> Best regards,
> > >>>
> > >>> Wei
> > >>>
> > >>>
> > >>> [1]
> > >>>
> > >>>
> > >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > >>>
> > >>> <
> > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > >>>
> > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > >>> http://cwiki.apache.org/confluence/**>
> > >>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > >>>
> > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> [2] refs/heads/disk_io_throttling
> > >>>
> > >>> [3]
> > >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> > >> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> > >>>
> > >>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> > >>>
> > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> > >>>
> > >>> <**
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > >>>
> > >>>
> > >>>
> > >>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> > >>>
> > >>> **<
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> > >> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> > >>>
> > >>> <**
> > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> > >> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> > >>>
> > >>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > >>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > >>>
> > >>>
> > >>>
> > >>> (**CLOUDSTACK-1301
> > >>>
> > >>> - VM Disk I/O Throttling)
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >> --
> > >> *Mike Tutkowski*
> > >> *Senior CloudStack Developer, SolidFire Inc.*
> > >> e: mike.tutkowski@solidfire.com
> > >> o: 303.746.7302
> > >> Advancing the way the world uses the
> > >> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> *â„¢*
> > >>
> >
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Perhaps I should add more detail. :)
The SolidFire plug-in is based on and requires Edison's new storage
framework (also going in with 4.2).
Leveraging Edison's framework, the plug-in is invoked at certain times to
create and delete volumes on the SAN (and also to create and delete SRs in
XenServer or Datastores in VMware to make use of those volumes).
The Min, Max, and Burst IOPS can be filled in by an admin when he/she
creates a Disk Offering or he/she can pass these IOPS fields onto end users
to fill in (in the Add Volume dialog). These fields are - ultimately -
optional as most storage systems do not yet support them (but they are
intentionally meant for future use by other storage systems per an earlier
discussion with the CloudStack e-mail list).
Talk to you later! :)
On Sun, Jun 2, 2013 at 6:38 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> Hi John,
>
> I believe Wei's feature deals with controlling the max number of IOPS from
> the hypervisor side.
>
> My feature is focused on controlling IOPS from the storage system side.
>
> I hope that helps. :)
>
>
> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com> wrote:
>
>> Wei,
>>
>> My opinion is that no features should be merged until all functional
>> issues have been resolved and it is ready to turn over to test. Until the
>> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
>> Wido, I don't think this criteria has been satisfied.
>>
>> Also, how does this work intersect/compliment the SolidFire patch (
>> https://reviews.apache.org/r/11479/)? As I understand it that work is
>> also involves provisioned IOPS. I would like to ensure we don't have a
>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
>> incompatible.
>>
>> Thanks,
>> -John
>>
>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>>
>> Wido,
>>
>>
>> Sure. I will change it next week.
>>
>>
>> -Wei
>>
>>
>>
>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>
>>
>> Hi Wei,
>>
>>
>>
>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>
>>
>> Wido,
>>
>>
>> Exactly. I have pushed the features into master.
>>
>>
>> If anyone object thems for technical reason till Monday, I will revert
>>
>> them.
>>
>>
>> For the sake of clarity I just want to mention again that we should change
>>
>> the total IOps to R/W IOps asap so that we never release a version with
>>
>> only total IOps.
>>
>>
>> You laid the groundwork for the I/O throttling and that's great! We should
>>
>> however prevent that we create legacy from day #1.
>>
>>
>> Wido
>>
>>
>> -Wei
>>
>>
>>
>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>
>>
>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>
>>
>> Wido,
>>
>>
>> +1 -- this enhancement must to discretely support read and write IOPS.
>>
>> I
>>
>> don't see how it could be fixed later because I don't see how we
>>
>> correctly
>>
>> split total IOPS into read and write. Therefore, we would be stuck
>>
>> with a
>>
>> total unless/until we decided to break backwards compatibility.
>>
>>
>>
>> What Wei meant was merging it into master now so that it will go in the
>>
>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>
>> will be released with Read and Write instead of Total IOps.
>>
>>
>> This is to make the May 31st feature freeze date. But if the window moves
>>
>> (see other threads) then it won't be necessary to do that.
>>
>>
>> Wido
>>
>>
>>
>> I also completely agree that there is no association between network
>>
>> and
>>
>>
>> disk I/O.
>>
>>
>> Thanks,
>>
>> -John
>>
>>
>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>
>>
>> Hi Wei,
>>
>>
>>
>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>
>>
>> Hi Wido,
>>
>>
>> Thanks. Good question.
>>
>>
>> I thought about at the beginning. Finally I decided to ignore the
>>
>> difference of read and write mainly because the network throttling did
>>
>> not
>>
>> care the difference of sent and received bytes as well.
>>
>> That reasoning seems odd. Networking and disk I/O completely different.
>>
>>
>> Disk I/O is much more expensive in most situations then network
>>
>> bandwith.
>>
>>
>> Implementing it will be some copy-paste work. It could be
>>
>> implemented in
>>
>>
>> few days. For the deadline of feature freeze, I will implement it
>>
>> after
>>
>> that , if needed.
>>
>>
>>
>> It think it's a feature we can't miss. But if it goes into the 4.2
>>
>> window we have to make sure we don't release with only total IOps and
>>
>> fix
>>
>> it in 4.3, that will confuse users.
>>
>>
>> Wido
>>
>>
>> -Wei
>>
>>
>>
>>
>>
>>
>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>
>>
>> Hi Wei,
>>
>>
>>
>>
>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>
>>
>> Hi,
>>
>>
>> I would like to merge disk_io_throttling branch into master.
>>
>> If nobody object, I will merge into master in 48 hours.
>>
>> The purpose is :
>>
>>
>> Virtual machines are running on the same storage device (local
>>
>> storage or
>>
>> share strage). Because of the rate limitation of device (such as
>>
>> iops), if
>>
>> one VM has large disk operation, it may affect the disk performance
>>
>> of
>>
>> other VMs running on the same storage device.
>>
>> It is neccesary to set the maximum rate and limit the disk I/O of
>>
>> VMs.
>>
>>
>>
>> Looking at the code I see you make no difference between Read and
>>
>> Write
>>
>> IOps.
>>
>>
>> Qemu and libvirt support setting both a different rate for Read and
>>
>> Write
>>
>> IOps which could benefit a lot of users.
>>
>>
>> It's also strange, in the polling side you collect both the Read and
>>
>> Write
>>
>> IOps, but on the throttling side you only go for a global value.
>>
>>
>> Write IOps are usually much more expensive then Read IOps, so it
>>
>> seems
>>
>> like a valid use-case where that an admin would set a lower value for
>>
>> write
>>
>> IOps vs Read IOps.
>>
>>
>> Since this only supports KVM at this point I think it would be of
>>
>> great
>>
>> value to at least have the mechanism in place to support both,
>>
>> implementing
>>
>> this later would be a lot of work.
>>
>>
>> If a hypervisor doesn't support setting different values for read and
>>
>> write you can always sum both up and set that as the total limit.
>>
>>
>> Can you explain why you implemented it this way?
>>
>>
>> Wido
>>
>>
>> The feature includes:
>>
>>
>>
>> (1) set the maximum rate of VMs (in disk_offering, and global
>>
>> configuration)
>>
>> (2) change the maximum rate of VMs
>>
>> (3) limit the disk rate (total bps and iops)
>>
>> JIRA ticket: https://issues.apache.org/****
>>
>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>>
>> jira/browse/CLOUDSTACK-1192<
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>
>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>
>> <**
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>
>>
>>
>> FS (I will update later) :
>>
>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>
>> <
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>
>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>> http://cwiki.apache.org/confluence/**>
>>
>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >
>>
>>
>>
>> Merge check list :-
>>
>>
>> * Did you check the branch's RAT execution success?
>>
>> Yes
>>
>>
>> * Are there new dependencies introduced?
>>
>> No
>>
>>
>> * What automated testing (unit and integration) is included in the
>>
>> new
>>
>> feature?
>>
>> Unit tests are added.
>>
>>
>> * What testing has been done to check for potential regressions?
>>
>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>
>> (2) VM operations, including
>>
>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>
>> (3) Volume operations, including
>>
>> Attach, Detach
>>
>>
>> To review the code, you can try
>>
>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>
>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>
>>
>> Best regards,
>>
>> Wei
>>
>>
>> [1]
>>
>>
>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>
>> <
>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>
>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>> http://cwiki.apache.org/confluence/**>
>>
>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>
>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>> >
>>
>>
>>
>> [2] refs/heads/disk_io_throttling
>>
>> [3]
>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>
>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>
>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>
>> <**
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>
>>
>>
>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>
>> **<
>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>
>> <**
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>
>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>
>>
>>
>> (**CLOUDSTACK-1301
>>
>> - VM Disk I/O Throttling)
>>
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Wei,
I apologize for conflating usage with billing. It seems that we need to enhance the usage information being captured to reflect that an operation was provisioned. It would be interesting to know if/how our users would like track usage of volumes with provisioned IOPS.
Thanks,
-John
On Jun 3, 2013, at 10:55 AM, Wei ZHOU <us...@gmail.com> wrote:
> John,
>
> For the billing, as no one works on billing now, users need to calculate
> the billing by themselves. They can get the service_offering and
> disk_offering of a VMs and volumes for calculation. Of course it is better
> to tell user the exact limitation value of individual volume, and network
> rate limitation for nics as well. I can work on it later. Do you think it
> is a part of I/O throttling?
>
> Sorry my misunstand the second the question.
>
> Agree with what you said about the two features.
>
> -Wei
>
>
> 2013/6/3 John Burwell <jb...@basho.com>
>
>> Wei,
>>
>>
>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
>>
>>> Hi John, Mike
>>>
>>> I hope Mike's aswer helps you. I am trying to adding more.
>>>
>>> (1) I think billing should depend on IO statistics rather than IOPS
>>> limitation. Please review disk_io_stat if you have time. disk_io_stat
>> can
>>> get the IO statistics including bytes/iops read/write for an individual
>>> virtual machine.
>>
>> Going by the AWS model, customers are billed more for volumes with
>> provisioned IOPS, as well as, for those operations (
>> http://aws.amazon.com/ebs/). I would imagine our users would like the
>> option to employ similar cost models. Could an operator implement such a
>> billing model in the current patch?
>>
>>>
>>> (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
>>> limitation for a running virtual machine through command line. However,
>>> CloudStack does not support changing the parameters of a created offering
>>> (computer offering or disk offering).
>>
>> I meant at the Java interface level. I apologize for being unclear. Can
>> we more generalize allocation algorithms with a set of interfaces that
>> describe the service guarantees provided by a resource?
>>
>>>
>>> (3) It is a good question. Maybe it is better to commit Mike's patch
>> after
>>> disk_io_throttling as Mike needs to consider the limitation in hypervisor
>>> type, I think.
>>
>> I will expand on my thoughts in a later response to Mike regarding the
>> touch points between these two features. I think that disk_io_throttling
>> will need to be merged before SolidFire, but I think we need closer
>> coordination between the branches (possibly have have solidfire track
>> disk_io_throttling) to coordinate on this issue.
>>
>>>
>>> - Wei
>>>
>>>
>>> 2013/6/3 John Burwell <jb...@basho.com>
>>>
>>>> Mike,
>>>>
>>>> The things I want to understand are the following:
>>>>
>>>> 1) Is there value in capturing IOPS policies be captured in a common
>>>> data model (e.g. for billing/usage purposes, expressing offerings).
>>>> 2) Should there be a common interface model for reasoning about IOP
>>>> provisioning at runtime?
>>>> 3) How are conflicting provisioned IOPS configurations between a
>>>> hypervisor and storage device reconciled? In particular, a scenario
>> where
>>>> is lead to believe (and billed) for more IOPS configured for a VM than a
>>>> storage device has been configured to deliver. Another scenario could a
>>>> consistent configuration between a VM and a storage device at creation
>>>> time, but a later modification to storage device introduces logical
>>>> inconsistency.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>> wrote:
>>>>
>>>> Hi John,
>>>>
>>>> I believe Wei's feature deals with controlling the max number of IOPS
>> from
>>>> the hypervisor side.
>>>>
>>>> My feature is focused on controlling IOPS from the storage system side.
>>>>
>>>> I hope that helps. :)
>>>>
>>>>
>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
>> wrote:
>>>>
>>>>> Wei,
>>>>>
>>>>> My opinion is that no features should be merged until all functional
>>>>> issues have been resolved and it is ready to turn over to test. Until
>>>> the
>>>>> total Ops vs discrete read/write ops issue is addressed and re-reviewed
>>>> by
>>>>> Wido, I don't think this criteria has been satisfied.
>>>>>
>>>>> Also, how does this work intersect/compliment the SolidFire patch (
>>>>> https://reviews.apache.org/r/11479/)? As I understand it that work is
>>>>> also involves provisioned IOPS. I would like to ensure we don't have a
>>>>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
>>>>> incompatible.
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>>>>>
>>>>> Wido,
>>>>>
>>>>>
>>>>> Sure. I will change it next week.
>>>>>
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>>
>>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>>
>>>>>
>>>>> Hi Wei,
>>>>>
>>>>>
>>>>>
>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>>
>>>>>
>>>>> Wido,
>>>>>
>>>>>
>>>>> Exactly. I have pushed the features into master.
>>>>>
>>>>>
>>>>> If anyone object thems for technical reason till Monday, I will revert
>>>>>
>>>>> them.
>>>>>
>>>>>
>>>>> For the sake of clarity I just want to mention again that we should
>>>> change
>>>>>
>>>>> the total IOps to R/W IOps asap so that we never release a version with
>>>>>
>>>>> only total IOps.
>>>>>
>>>>>
>>>>> You laid the groundwork for the I/O throttling and that's great! We
>>>> should
>>>>>
>>>>> however prevent that we create legacy from day #1.
>>>>>
>>>>>
>>>>> Wido
>>>>>
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>>
>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>
>>>>>
>>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>>
>>>>>
>>>>> Wido,
>>>>>
>>>>>
>>>>> +1 -- this enhancement must to discretely support read and write IOPS.
>>>>>
>>>>> I
>>>>>
>>>>> don't see how it could be fixed later because I don't see how we
>>>>>
>>>>> correctly
>>>>>
>>>>> split total IOPS into read and write. Therefore, we would be stuck
>>>>>
>>>>> with a
>>>>>
>>>>> total unless/until we decided to break backwards compatibility.
>>>>>
>>>>>
>>>>>
>>>>> What Wei meant was merging it into master now so that it will go in the
>>>>>
>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>>>>
>>>>> will be released with Read and Write instead of Total IOps.
>>>>>
>>>>>
>>>>> This is to make the May 31st feature freeze date. But if the window
>> moves
>>>>>
>>>>> (see other threads) then it won't be necessary to do that.
>>>>>
>>>>>
>>>>> Wido
>>>>>
>>>>>
>>>>>
>>>>> I also completely agree that there is no association between network
>>>>>
>>>>> and
>>>>>
>>>>>
>>>>> disk I/O.
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -John
>>>>>
>>>>>
>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
>> wrote:
>>>>>
>>>>>
>>>>> Hi Wei,
>>>>>
>>>>>
>>>>>
>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>
>>>>>
>>>>> Hi Wido,
>>>>>
>>>>>
>>>>> Thanks. Good question.
>>>>>
>>>>>
>>>>> I thought about at the beginning. Finally I decided to ignore the
>>>>>
>>>>> difference of read and write mainly because the network throttling did
>>>>>
>>>>> not
>>>>>
>>>>> care the difference of sent and received bytes as well.
>>>>>
>>>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>>>
>>>>>
>>>>> Disk I/O is much more expensive in most situations then network
>>>>>
>>>>> bandwith.
>>>>>
>>>>>
>>>>> Implementing it will be some copy-paste work. It could be
>>>>>
>>>>> implemented in
>>>>>
>>>>>
>>>>> few days. For the deadline of feature freeze, I will implement it
>>>>>
>>>>> after
>>>>>
>>>>> that , if needed.
>>>>>
>>>>>
>>>>>
>>>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>>>>
>>>>> window we have to make sure we don't release with only total IOps and
>>>>>
>>>>> fix
>>>>>
>>>>> it in 4.3, that will confuse users.
>>>>>
>>>>>
>>>>> Wido
>>>>>
>>>>>
>>>>> -Wei
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>
>>>>>
>>>>> Hi Wei,
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>
>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>
>>>>> The purpose is :
>>>>>
>>>>>
>>>>> Virtual machines are running on the same storage device (local
>>>>>
>>>>> storage or
>>>>>
>>>>> share strage). Because of the rate limitation of device (such as
>>>>>
>>>>> iops), if
>>>>>
>>>>> one VM has large disk operation, it may affect the disk performance
>>>>>
>>>>> of
>>>>>
>>>>> other VMs running on the same storage device.
>>>>>
>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
>>>>>
>>>>> VMs.
>>>>>
>>>>>
>>>>>
>>>>> Looking at the code I see you make no difference between Read and
>>>>>
>>>>> Write
>>>>>
>>>>> IOps.
>>>>>
>>>>>
>>>>> Qemu and libvirt support setting both a different rate for Read and
>>>>>
>>>>> Write
>>>>>
>>>>> IOps which could benefit a lot of users.
>>>>>
>>>>>
>>>>> It's also strange, in the polling side you collect both the Read and
>>>>>
>>>>> Write
>>>>>
>>>>> IOps, but on the throttling side you only go for a global value.
>>>>>
>>>>>
>>>>> Write IOps are usually much more expensive then Read IOps, so it
>>>>>
>>>>> seems
>>>>>
>>>>> like a valid use-case where that an admin would set a lower value for
>>>>>
>>>>> write
>>>>>
>>>>> IOps vs Read IOps.
>>>>>
>>>>>
>>>>> Since this only supports KVM at this point I think it would be of
>>>>>
>>>>> great
>>>>>
>>>>> value to at least have the mechanism in place to support both,
>>>>>
>>>>> implementing
>>>>>
>>>>> this later would be a lot of work.
>>>>>
>>>>>
>>>>> If a hypervisor doesn't support setting different values for read and
>>>>>
>>>>> write you can always sum both up and set that as the total limit.
>>>>>
>>>>>
>>>>> Can you explain why you implemented it this way?
>>>>>
>>>>>
>>>>> Wido
>>>>>
>>>>>
>>>>> The feature includes:
>>>>>
>>>>>
>>>>>
>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>
>>>>> configuration)
>>>>>
>>>>> (2) change the maximum rate of VMs
>>>>>
>>>>> (3) limit the disk rate (total bps and iops)
>>>>>
>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>
>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>>>>>
>>>>> jira/browse/CLOUDSTACK-1192<
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>
>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>
>>>>> <**
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>
>>>>>
>>>>>
>>>>> FS (I will update later) :
>>>>>
>>>>>
>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>
>>>>> <
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>
>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>>>> http://cwiki.apache.org/confluence/**>
>>>>>
>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>>
>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Merge check list :-
>>>>>
>>>>>
>>>>> * Did you check the branch's RAT execution success?
>>>>>
>>>>> Yes
>>>>>
>>>>>
>>>>> * Are there new dependencies introduced?
>>>>>
>>>>> No
>>>>>
>>>>>
>>>>> * What automated testing (unit and integration) is included in the
>>>>>
>>>>> new
>>>>>
>>>>> feature?
>>>>>
>>>>> Unit tests are added.
>>>>>
>>>>>
>>>>> * What testing has been done to check for potential regressions?
>>>>>
>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>
>>>>> (2) VM operations, including
>>>>>
>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>
>>>>> (3) Volume operations, including
>>>>>
>>>>> Attach, Detach
>>>>>
>>>>>
>>>>> To review the code, you can try
>>>>>
>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>
>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Wei
>>>>>
>>>>>
>>>>> [1]
>>>>>
>>>>>
>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>
>>>>> <
>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>
>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>>>> http://cwiki.apache.org/confluence/**>
>>>>>
>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>>
>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [2] refs/heads/disk_io_throttling
>>>>>
>>>>> [3]
>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>>>
>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>
>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>
>>>>> <**
>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>
>>>>>
>>>>>
>>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>>>
>>>>> **<
>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>
>>>>> <**
>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>
>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>
>>>>>
>>>>>
>>>>> (**CLOUDSTACK-1301
>>>>>
>>>>> - VM Disk I/O Throttling)
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *â„¢*
>>>>
>>
>>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I agree on merging Wei's feature first, then mine.
If his feature is for KVM only, then it is a non issue as I don't support
KVM in 4.2.
On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <us...@gmail.com> wrote:
> John,
>
> For the billing, as no one works on billing now, users need to calculate
> the billing by themselves. They can get the service_offering and
> disk_offering of a VMs and volumes for calculation. Of course it is better
> to tell user the exact limitation value of individual volume, and network
> rate limitation for nics as well. I can work on it later. Do you think it
> is a part of I/O throttling?
>
> Sorry my misunstand the second the question.
>
> Agree with what you said about the two features.
>
> -Wei
>
>
> 2013/6/3 John Burwell <jb...@basho.com>
>
> > Wei,
> >
> >
> > On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
> >
> > > Hi John, Mike
> > >
> > > I hope Mike's aswer helps you. I am trying to adding more.
> > >
> > > (1) I think billing should depend on IO statistics rather than IOPS
> > > limitation. Please review disk_io_stat if you have time. disk_io_stat
> > can
> > > get the IO statistics including bytes/iops read/write for an individual
> > > virtual machine.
> >
> > Going by the AWS model, customers are billed more for volumes with
> > provisioned IOPS, as well as, for those operations (
> > http://aws.amazon.com/ebs/). I would imagine our users would like the
> > option to employ similar cost models. Could an operator implement such a
> > billing model in the current patch?
> >
> > >
> > > (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
> > > limitation for a running virtual machine through command line. However,
> > > CloudStack does not support changing the parameters of a created
> offering
> > > (computer offering or disk offering).
> >
> > I meant at the Java interface level. I apologize for being unclear. Can
> > we more generalize allocation algorithms with a set of interfaces that
> > describe the service guarantees provided by a resource?
> >
> > >
> > > (3) It is a good question. Maybe it is better to commit Mike's patch
> > after
> > > disk_io_throttling as Mike needs to consider the limitation in
> hypervisor
> > > type, I think.
> >
> > I will expand on my thoughts in a later response to Mike regarding the
> > touch points between these two features. I think that disk_io_throttling
> > will need to be merged before SolidFire, but I think we need closer
> > coordination between the branches (possibly have have solidfire track
> > disk_io_throttling) to coordinate on this issue.
> >
> > >
> > > - Wei
> > >
> > >
> > > 2013/6/3 John Burwell <jb...@basho.com>
> > >
> > >> Mike,
> > >>
> > >> The things I want to understand are the following:
> > >>
> > >> 1) Is there value in capturing IOPS policies be captured in a common
> > >> data model (e.g. for billing/usage purposes, expressing offerings).
> > >> 2) Should there be a common interface model for reasoning about IOP
> > >> provisioning at runtime?
> > >> 3) How are conflicting provisioned IOPS configurations between a
> > >> hypervisor and storage device reconciled? In particular, a scenario
> > where
> > >> is lead to believe (and billed) for more IOPS configured for a VM
> than a
> > >> storage device has been configured to deliver. Another scenario
> could a
> > >> consistent configuration between a VM and a storage device at creation
> > >> time, but a later modification to storage device introduces logical
> > >> inconsistency.
> > >>
> > >> Thanks,
> > >> -John
> > >>
> > >> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com>
> > >> wrote:
> > >>
> > >> Hi John,
> > >>
> > >> I believe Wei's feature deals with controlling the max number of IOPS
> > from
> > >> the hypervisor side.
> > >>
> > >> My feature is focused on controlling IOPS from the storage system
> side.
> > >>
> > >> I hope that helps. :)
> > >>
> > >>
> > >> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
> > wrote:
> > >>
> > >>> Wei,
> > >>>
> > >>> My opinion is that no features should be merged until all functional
> > >>> issues have been resolved and it is ready to turn over to test.
> Until
> > >> the
> > >>> total Ops vs discrete read/write ops issue is addressed and
> re-reviewed
> > >> by
> > >>> Wido, I don't think this criteria has been satisfied.
> > >>>
> > >>> Also, how does this work intersect/compliment the SolidFire patch (
> > >>> https://reviews.apache.org/r/11479/)? As I understand it that work
> is
> > >>> also involves provisioned IOPS. I would like to ensure we don't
> have a
> > >>> scenario where provisioned IOPS in KVM and SolidFire are
> unnecessarily
> > >>> incompatible.
> > >>>
> > >>> Thanks,
> > >>> -John
> > >>>
> > >>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
> > >>>
> > >>> Wido,
> > >>>
> > >>>
> > >>> Sure. I will change it next week.
> > >>>
> > >>>
> > >>> -Wei
> > >>>
> > >>>
> > >>>
> > >>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> > >>>
> > >>>
> > >>> Hi Wei,
> > >>>
> > >>>
> > >>>
> > >>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> > >>>
> > >>>
> > >>> Wido,
> > >>>
> > >>>
> > >>> Exactly. I have pushed the features into master.
> > >>>
> > >>>
> > >>> If anyone object thems for technical reason till Monday, I will
> revert
> > >>>
> > >>> them.
> > >>>
> > >>>
> > >>> For the sake of clarity I just want to mention again that we should
> > >> change
> > >>>
> > >>> the total IOps to R/W IOps asap so that we never release a version
> with
> > >>>
> > >>> only total IOps.
> > >>>
> > >>>
> > >>> You laid the groundwork for the I/O throttling and that's great! We
> > >> should
> > >>>
> > >>> however prevent that we create legacy from day #1.
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>> -Wei
> > >>>
> > >>>
> > >>>
> > >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > >>>
> > >>>
> > >>> On 05/31/2013 03:59 PM, John Burwell wrote:
> > >>>
> > >>>
> > >>> Wido,
> > >>>
> > >>>
> > >>> +1 -- this enhancement must to discretely support read and write
> IOPS.
> > >>>
> > >>> I
> > >>>
> > >>> don't see how it could be fixed later because I don't see how we
> > >>>
> > >>> correctly
> > >>>
> > >>> split total IOPS into read and write. Therefore, we would be stuck
> > >>>
> > >>> with a
> > >>>
> > >>> total unless/until we decided to break backwards compatibility.
> > >>>
> > >>>
> > >>>
> > >>> What Wei meant was merging it into master now so that it will go in
> the
> > >>>
> > >>> 4.2 branch and add Read / Write IOps before the 4.2 release so that
> 4.2
> > >>>
> > >>> will be released with Read and Write instead of Total IOps.
> > >>>
> > >>>
> > >>> This is to make the May 31st feature freeze date. But if the window
> > moves
> > >>>
> > >>> (see other threads) then it won't be necessary to do that.
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>>
> > >>> I also completely agree that there is no association between network
> > >>>
> > >>> and
> > >>>
> > >>>
> > >>> disk I/O.
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>> -John
> > >>>
> > >>>
> > >>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
> > wrote:
> > >>>
> > >>>
> > >>> Hi Wei,
> > >>>
> > >>>
> > >>>
> > >>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> > >>>
> > >>>
> > >>> Hi Wido,
> > >>>
> > >>>
> > >>> Thanks. Good question.
> > >>>
> > >>>
> > >>> I thought about at the beginning. Finally I decided to ignore the
> > >>>
> > >>> difference of read and write mainly because the network throttling
> did
> > >>>
> > >>> not
> > >>>
> > >>> care the difference of sent and received bytes as well.
> > >>>
> > >>> That reasoning seems odd. Networking and disk I/O completely
> different.
> > >>>
> > >>>
> > >>> Disk I/O is much more expensive in most situations then network
> > >>>
> > >>> bandwith.
> > >>>
> > >>>
> > >>> Implementing it will be some copy-paste work. It could be
> > >>>
> > >>> implemented in
> > >>>
> > >>>
> > >>> few days. For the deadline of feature freeze, I will implement it
> > >>>
> > >>> after
> > >>>
> > >>> that , if needed.
> > >>>
> > >>>
> > >>>
> > >>> It think it's a feature we can't miss. But if it goes into the 4.2
> > >>>
> > >>> window we have to make sure we don't release with only total IOps and
> > >>>
> > >>> fix
> > >>>
> > >>> it in 4.3, that will confuse users.
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>> -Wei
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> > >>>
> > >>>
> > >>> Hi Wei,
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> > >>>
> > >>>
> > >>> Hi,
> > >>>
> > >>>
> > >>> I would like to merge disk_io_throttling branch into master.
> > >>>
> > >>> If nobody object, I will merge into master in 48 hours.
> > >>>
> > >>> The purpose is :
> > >>>
> > >>>
> > >>> Virtual machines are running on the same storage device (local
> > >>>
> > >>> storage or
> > >>>
> > >>> share strage). Because of the rate limitation of device (such as
> > >>>
> > >>> iops), if
> > >>>
> > >>> one VM has large disk operation, it may affect the disk performance
> > >>>
> > >>> of
> > >>>
> > >>> other VMs running on the same storage device.
> > >>>
> > >>> It is neccesary to set the maximum rate and limit the disk I/O of
> > >>>
> > >>> VMs.
> > >>>
> > >>>
> > >>>
> > >>> Looking at the code I see you make no difference between Read and
> > >>>
> > >>> Write
> > >>>
> > >>> IOps.
> > >>>
> > >>>
> > >>> Qemu and libvirt support setting both a different rate for Read and
> > >>>
> > >>> Write
> > >>>
> > >>> IOps which could benefit a lot of users.
> > >>>
> > >>>
> > >>> It's also strange, in the polling side you collect both the Read and
> > >>>
> > >>> Write
> > >>>
> > >>> IOps, but on the throttling side you only go for a global value.
> > >>>
> > >>>
> > >>> Write IOps are usually much more expensive then Read IOps, so it
> > >>>
> > >>> seems
> > >>>
> > >>> like a valid use-case where that an admin would set a lower value for
> > >>>
> > >>> write
> > >>>
> > >>> IOps vs Read IOps.
> > >>>
> > >>>
> > >>> Since this only supports KVM at this point I think it would be of
> > >>>
> > >>> great
> > >>>
> > >>> value to at least have the mechanism in place to support both,
> > >>>
> > >>> implementing
> > >>>
> > >>> this later would be a lot of work.
> > >>>
> > >>>
> > >>> If a hypervisor doesn't support setting different values for read and
> > >>>
> > >>> write you can always sum both up and set that as the total limit.
> > >>>
> > >>>
> > >>> Can you explain why you implemented it this way?
> > >>>
> > >>>
> > >>> Wido
> > >>>
> > >>>
> > >>> The feature includes:
> > >>>
> > >>>
> > >>>
> > >>> (1) set the maximum rate of VMs (in disk_offering, and global
> > >>>
> > >>> configuration)
> > >>>
> > >>> (2) change the maximum rate of VMs
> > >>>
> > >>> (3) limit the disk rate (total bps and iops)
> > >>>
> > >>> JIRA ticket: https://issues.apache.org/****
> > >>>
> > >>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> > >>>
> > >>> jira/browse/CLOUDSTACK-1192<
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> > >>>
> > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> > >>>
> > >>> <**
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> > >>>
> > >>>
> > >>>
> > >>> FS (I will update later) :
> > >>>
> > >>>
> > >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > >>>
> > >>> <
> > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > >>>
> > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > >>> http://cwiki.apache.org/confluence/**>
> > >>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > >>>
> > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> Merge check list :-
> > >>>
> > >>>
> > >>> * Did you check the branch's RAT execution success?
> > >>>
> > >>> Yes
> > >>>
> > >>>
> > >>> * Are there new dependencies introduced?
> > >>>
> > >>> No
> > >>>
> > >>>
> > >>> * What automated testing (unit and integration) is included in the
> > >>>
> > >>> new
> > >>>
> > >>> feature?
> > >>>
> > >>> Unit tests are added.
> > >>>
> > >>>
> > >>> * What testing has been done to check for potential regressions?
> > >>>
> > >>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> > >>>
> > >>> (2) VM operations, including
> > >>>
> > >>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> > >>>
> > >>> (3) Volume operations, including
> > >>>
> > >>> Attach, Detach
> > >>>
> > >>>
> > >>> To review the code, you can try
> > >>>
> > >>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> > >>>
> > >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> > >>>
> > >>>
> > >>> Best regards,
> > >>>
> > >>> Wei
> > >>>
> > >>>
> > >>> [1]
> > >>>
> > >>>
> > >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> > >>>
> > >>> <
> > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> > >>>
> > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > >>> http://cwiki.apache.org/confluence/**>
> > >>>
> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> > >>>
> > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> [2] refs/heads/disk_io_throttling
> > >>>
> > >>> [3]
> > >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> > >> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> > >>>
> > >>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> > >>>
> > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> > >>>
> > >>> <**
> > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> > >>>
> > >>>
> > >>>
> > >>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> > >>>
> > >>> **<
> > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> > >> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> > >>>
> > >>> <**
> > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> > >> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> > >>>
> > >>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > >>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> > >>>
> > >>>
> > >>>
> > >>> (**CLOUDSTACK-1301
> > >>>
> > >>> - VM Disk I/O Throttling)
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >> --
> > >> *Mike Tutkowski*
> > >> *Senior CloudStack Developer, SolidFire Inc.*
> > >> e: mike.tutkowski@solidfire.com
> > >> o: 303.746.7302
> > >> Advancing the way the world uses the
> > >> cloud<http://solidfire.com/solution/overview/?video=play>
> > >> *â„¢*
> > >>
> >
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
John,
For the billing, as no one works on billing now, users need to calculate
the billing by themselves. They can get the service_offering and
disk_offering of a VMs and volumes for calculation. Of course it is better
to tell user the exact limitation value of individual volume, and network
rate limitation for nics as well. I can work on it later. Do you think it
is a part of I/O throttling?
Sorry my misunstand the second the question.
Agree with what you said about the two features.
-Wei
2013/6/3 John Burwell <jb...@basho.com>
> Wei,
>
>
> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
>
> > Hi John, Mike
> >
> > I hope Mike's aswer helps you. I am trying to adding more.
> >
> > (1) I think billing should depend on IO statistics rather than IOPS
> > limitation. Please review disk_io_stat if you have time. disk_io_stat
> can
> > get the IO statistics including bytes/iops read/write for an individual
> > virtual machine.
>
> Going by the AWS model, customers are billed more for volumes with
> provisioned IOPS, as well as, for those operations (
> http://aws.amazon.com/ebs/). I would imagine our users would like the
> option to employ similar cost models. Could an operator implement such a
> billing model in the current patch?
>
> >
> > (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
> > limitation for a running virtual machine through command line. However,
> > CloudStack does not support changing the parameters of a created offering
> > (computer offering or disk offering).
>
> I meant at the Java interface level. I apologize for being unclear. Can
> we more generalize allocation algorithms with a set of interfaces that
> describe the service guarantees provided by a resource?
>
> >
> > (3) It is a good question. Maybe it is better to commit Mike's patch
> after
> > disk_io_throttling as Mike needs to consider the limitation in hypervisor
> > type, I think.
>
> I will expand on my thoughts in a later response to Mike regarding the
> touch points between these two features. I think that disk_io_throttling
> will need to be merged before SolidFire, but I think we need closer
> coordination between the branches (possibly have have solidfire track
> disk_io_throttling) to coordinate on this issue.
>
> >
> > - Wei
> >
> >
> > 2013/6/3 John Burwell <jb...@basho.com>
> >
> >> Mike,
> >>
> >> The things I want to understand are the following:
> >>
> >> 1) Is there value in capturing IOPS policies be captured in a common
> >> data model (e.g. for billing/usage purposes, expressing offerings).
> >> 2) Should there be a common interface model for reasoning about IOP
> >> provisioning at runtime?
> >> 3) How are conflicting provisioned IOPS configurations between a
> >> hypervisor and storage device reconciled? In particular, a scenario
> where
> >> is lead to believe (and billed) for more IOPS configured for a VM than a
> >> storage device has been configured to deliver. Another scenario could a
> >> consistent configuration between a VM and a storage device at creation
> >> time, but a later modification to storage device introduces logical
> >> inconsistency.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >> Hi John,
> >>
> >> I believe Wei's feature deals with controlling the max number of IOPS
> from
> >> the hypervisor side.
> >>
> >> My feature is focused on controlling IOPS from the storage system side.
> >>
> >> I hope that helps. :)
> >>
> >>
> >> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com>
> wrote:
> >>
> >>> Wei,
> >>>
> >>> My opinion is that no features should be merged until all functional
> >>> issues have been resolved and it is ready to turn over to test. Until
> >> the
> >>> total Ops vs discrete read/write ops issue is addressed and re-reviewed
> >> by
> >>> Wido, I don't think this criteria has been satisfied.
> >>>
> >>> Also, how does this work intersect/compliment the SolidFire patch (
> >>> https://reviews.apache.org/r/11479/)? As I understand it that work is
> >>> also involves provisioned IOPS. I would like to ensure we don't have a
> >>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> >>> incompatible.
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
> >>>
> >>> Wido,
> >>>
> >>>
> >>> Sure. I will change it next week.
> >>>
> >>>
> >>> -Wei
> >>>
> >>>
> >>>
> >>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >>>
> >>>
> >>> Hi Wei,
> >>>
> >>>
> >>>
> >>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>
> >>>
> >>> Wido,
> >>>
> >>>
> >>> Exactly. I have pushed the features into master.
> >>>
> >>>
> >>> If anyone object thems for technical reason till Monday, I will revert
> >>>
> >>> them.
> >>>
> >>>
> >>> For the sake of clarity I just want to mention again that we should
> >> change
> >>>
> >>> the total IOps to R/W IOps asap so that we never release a version with
> >>>
> >>> only total IOps.
> >>>
> >>>
> >>> You laid the groundwork for the I/O throttling and that's great! We
> >> should
> >>>
> >>> however prevent that we create legacy from day #1.
> >>>
> >>>
> >>> Wido
> >>>
> >>>
> >>> -Wei
> >>>
> >>>
> >>>
> >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>
> >>>
> >>> On 05/31/2013 03:59 PM, John Burwell wrote:
> >>>
> >>>
> >>> Wido,
> >>>
> >>>
> >>> +1 -- this enhancement must to discretely support read and write IOPS.
> >>>
> >>> I
> >>>
> >>> don't see how it could be fixed later because I don't see how we
> >>>
> >>> correctly
> >>>
> >>> split total IOPS into read and write. Therefore, we would be stuck
> >>>
> >>> with a
> >>>
> >>> total unless/until we decided to break backwards compatibility.
> >>>
> >>>
> >>>
> >>> What Wei meant was merging it into master now so that it will go in the
> >>>
> >>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
> >>>
> >>> will be released with Read and Write instead of Total IOps.
> >>>
> >>>
> >>> This is to make the May 31st feature freeze date. But if the window
> moves
> >>>
> >>> (see other threads) then it won't be necessary to do that.
> >>>
> >>>
> >>> Wido
> >>>
> >>>
> >>>
> >>> I also completely agree that there is no association between network
> >>>
> >>> and
> >>>
> >>>
> >>> disk I/O.
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> -John
> >>>
> >>>
> >>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >>>
> >>>
> >>> Hi Wei,
> >>>
> >>>
> >>>
> >>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>
> >>>
> >>> Hi Wido,
> >>>
> >>>
> >>> Thanks. Good question.
> >>>
> >>>
> >>> I thought about at the beginning. Finally I decided to ignore the
> >>>
> >>> difference of read and write mainly because the network throttling did
> >>>
> >>> not
> >>>
> >>> care the difference of sent and received bytes as well.
> >>>
> >>> That reasoning seems odd. Networking and disk I/O completely different.
> >>>
> >>>
> >>> Disk I/O is much more expensive in most situations then network
> >>>
> >>> bandwith.
> >>>
> >>>
> >>> Implementing it will be some copy-paste work. It could be
> >>>
> >>> implemented in
> >>>
> >>>
> >>> few days. For the deadline of feature freeze, I will implement it
> >>>
> >>> after
> >>>
> >>> that , if needed.
> >>>
> >>>
> >>>
> >>> It think it's a feature we can't miss. But if it goes into the 4.2
> >>>
> >>> window we have to make sure we don't release with only total IOps and
> >>>
> >>> fix
> >>>
> >>> it in 4.3, that will confuse users.
> >>>
> >>>
> >>> Wido
> >>>
> >>>
> >>> -Wei
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >>>
> >>>
> >>> Hi Wei,
> >>>
> >>>
> >>>
> >>>
> >>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>
> >>>
> >>> Hi,
> >>>
> >>>
> >>> I would like to merge disk_io_throttling branch into master.
> >>>
> >>> If nobody object, I will merge into master in 48 hours.
> >>>
> >>> The purpose is :
> >>>
> >>>
> >>> Virtual machines are running on the same storage device (local
> >>>
> >>> storage or
> >>>
> >>> share strage). Because of the rate limitation of device (such as
> >>>
> >>> iops), if
> >>>
> >>> one VM has large disk operation, it may affect the disk performance
> >>>
> >>> of
> >>>
> >>> other VMs running on the same storage device.
> >>>
> >>> It is neccesary to set the maximum rate and limit the disk I/O of
> >>>
> >>> VMs.
> >>>
> >>>
> >>>
> >>> Looking at the code I see you make no difference between Read and
> >>>
> >>> Write
> >>>
> >>> IOps.
> >>>
> >>>
> >>> Qemu and libvirt support setting both a different rate for Read and
> >>>
> >>> Write
> >>>
> >>> IOps which could benefit a lot of users.
> >>>
> >>>
> >>> It's also strange, in the polling side you collect both the Read and
> >>>
> >>> Write
> >>>
> >>> IOps, but on the throttling side you only go for a global value.
> >>>
> >>>
> >>> Write IOps are usually much more expensive then Read IOps, so it
> >>>
> >>> seems
> >>>
> >>> like a valid use-case where that an admin would set a lower value for
> >>>
> >>> write
> >>>
> >>> IOps vs Read IOps.
> >>>
> >>>
> >>> Since this only supports KVM at this point I think it would be of
> >>>
> >>> great
> >>>
> >>> value to at least have the mechanism in place to support both,
> >>>
> >>> implementing
> >>>
> >>> this later would be a lot of work.
> >>>
> >>>
> >>> If a hypervisor doesn't support setting different values for read and
> >>>
> >>> write you can always sum both up and set that as the total limit.
> >>>
> >>>
> >>> Can you explain why you implemented it this way?
> >>>
> >>>
> >>> Wido
> >>>
> >>>
> >>> The feature includes:
> >>>
> >>>
> >>>
> >>> (1) set the maximum rate of VMs (in disk_offering, and global
> >>>
> >>> configuration)
> >>>
> >>> (2) change the maximum rate of VMs
> >>>
> >>> (3) limit the disk rate (total bps and iops)
> >>>
> >>> JIRA ticket: https://issues.apache.org/****
> >>>
> >>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> >>>
> >>> jira/browse/CLOUDSTACK-1192<
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>
> >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>
> >>> <**
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>
> >>>
> >>>
> >>> FS (I will update later) :
> >>>
> >>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>
> >>> <
> >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>
> >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> >>> http://cwiki.apache.org/confluence/**>
> >>>
> >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> >>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>
> >>>
> >>>
> >>>
> >>> Merge check list :-
> >>>
> >>>
> >>> * Did you check the branch's RAT execution success?
> >>>
> >>> Yes
> >>>
> >>>
> >>> * Are there new dependencies introduced?
> >>>
> >>> No
> >>>
> >>>
> >>> * What automated testing (unit and integration) is included in the
> >>>
> >>> new
> >>>
> >>> feature?
> >>>
> >>> Unit tests are added.
> >>>
> >>>
> >>> * What testing has been done to check for potential regressions?
> >>>
> >>> (1) set the bytes rate and IOPS rate on CloudStack UI.
> >>>
> >>> (2) VM operations, including
> >>>
> >>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >>>
> >>> (3) Volume operations, including
> >>>
> >>> Attach, Detach
> >>>
> >>>
> >>> To review the code, you can try
> >>>
> >>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >>>
> >>> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>
> >>>
> >>> Best regards,
> >>>
> >>> Wei
> >>>
> >>>
> >>> [1]
> >>>
> >>>
> >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>
> >>> <
> >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >>>
> >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> >>> http://cwiki.apache.org/confluence/**>
> >>>
> >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >>>
> >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> >>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>
> >>>
> >>>
> >>>
> >>> [2] refs/heads/disk_io_throttling
> >>>
> >>> [3]
> >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> >> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> >>>
> >>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>
> >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>
> >>> <**
> >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>
> >>>
> >>>
> >>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> >>>
> >>> **<
> >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> >> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>
> >>> <**
> >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> >> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>
> >>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>
> >>>
> >>>
> >>> (**CLOUDSTACK-1301
> >>>
> >>> - VM Disk I/O Throttling)
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=play>
> >> *â„¢*
> >>
>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Wei,
On Jun 3, 2013, at 2:13 AM, Wei ZHOU <us...@gmail.com> wrote:
> Hi John, Mike
>
> I hope Mike's aswer helps you. I am trying to adding more.
>
> (1) I think billing should depend on IO statistics rather than IOPS
> limitation. Please review disk_io_stat if you have time. disk_io_stat can
> get the IO statistics including bytes/iops read/write for an individual
> virtual machine.
Going by the AWS model, customers are billed more for volumes with provisioned IOPS, as well as, for those operations (http://aws.amazon.com/ebs/). I would imagine our users would like the option to employ similar cost models. Could an operator implement such a billing model in the current patch?
>
> (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
> limitation for a running virtual machine through command line. However,
> CloudStack does not support changing the parameters of a created offering
> (computer offering or disk offering).
I meant at the Java interface level. I apologize for being unclear. Can we more generalize allocation algorithms with a set of interfaces that describe the service guarantees provided by a resource?
>
> (3) It is a good question. Maybe it is better to commit Mike's patch after
> disk_io_throttling as Mike needs to consider the limitation in hypervisor
> type, I think.
I will expand on my thoughts in a later response to Mike regarding the touch points between these two features. I think that disk_io_throttling will need to be merged before SolidFire, but I think we need closer coordination between the branches (possibly have have solidfire track disk_io_throttling) to coordinate on this issue.
>
> - Wei
>
>
> 2013/6/3 John Burwell <jb...@basho.com>
>
>> Mike,
>>
>> The things I want to understand are the following:
>>
>> 1) Is there value in capturing IOPS policies be captured in a common
>> data model (e.g. for billing/usage purposes, expressing offerings).
>> 2) Should there be a common interface model for reasoning about IOP
>> provisioning at runtime?
>> 3) How are conflicting provisioned IOPS configurations between a
>> hypervisor and storage device reconciled? In particular, a scenario where
>> is lead to believe (and billed) for more IOPS configured for a VM than a
>> storage device has been configured to deliver. Another scenario could a
>> consistent configuration between a VM and a storage device at creation
>> time, but a later modification to storage device introduces logical
>> inconsistency.
>>
>> Thanks,
>> -John
>>
>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> Hi John,
>>
>> I believe Wei's feature deals with controlling the max number of IOPS from
>> the hypervisor side.
>>
>> My feature is focused on controlling IOPS from the storage system side.
>>
>> I hope that helps. :)
>>
>>
>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Wei,
>>>
>>> My opinion is that no features should be merged until all functional
>>> issues have been resolved and it is ready to turn over to test. Until
>> the
>>> total Ops vs discrete read/write ops issue is addressed and re-reviewed
>> by
>>> Wido, I don't think this criteria has been satisfied.
>>>
>>> Also, how does this work intersect/compliment the SolidFire patch (
>>> https://reviews.apache.org/r/11479/)? As I understand it that work is
>>> also involves provisioned IOPS. I would like to ensure we don't have a
>>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
>>> incompatible.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>>>
>>> Wido,
>>>
>>>
>>> Sure. I will change it next week.
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>
>>>
>>> Hi Wei,
>>>
>>>
>>>
>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>
>>>
>>> Wido,
>>>
>>>
>>> Exactly. I have pushed the features into master.
>>>
>>>
>>> If anyone object thems for technical reason till Monday, I will revert
>>>
>>> them.
>>>
>>>
>>> For the sake of clarity I just want to mention again that we should
>> change
>>>
>>> the total IOps to R/W IOps asap so that we never release a version with
>>>
>>> only total IOps.
>>>
>>>
>>> You laid the groundwork for the I/O throttling and that's great! We
>> should
>>>
>>> however prevent that we create legacy from day #1.
>>>
>>>
>>> Wido
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>
>>>
>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>
>>>
>>> Wido,
>>>
>>>
>>> +1 -- this enhancement must to discretely support read and write IOPS.
>>>
>>> I
>>>
>>> don't see how it could be fixed later because I don't see how we
>>>
>>> correctly
>>>
>>> split total IOPS into read and write. Therefore, we would be stuck
>>>
>>> with a
>>>
>>> total unless/until we decided to break backwards compatibility.
>>>
>>>
>>>
>>> What Wei meant was merging it into master now so that it will go in the
>>>
>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>>
>>> will be released with Read and Write instead of Total IOps.
>>>
>>>
>>> This is to make the May 31st feature freeze date. But if the window moves
>>>
>>> (see other threads) then it won't be necessary to do that.
>>>
>>>
>>> Wido
>>>
>>>
>>>
>>> I also completely agree that there is no association between network
>>>
>>> and
>>>
>>>
>>> disk I/O.
>>>
>>>
>>> Thanks,
>>>
>>> -John
>>>
>>>
>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>>
>>>
>>> Hi Wei,
>>>
>>>
>>>
>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>
>>>
>>> Hi Wido,
>>>
>>>
>>> Thanks. Good question.
>>>
>>>
>>> I thought about at the beginning. Finally I decided to ignore the
>>>
>>> difference of read and write mainly because the network throttling did
>>>
>>> not
>>>
>>> care the difference of sent and received bytes as well.
>>>
>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>
>>>
>>> Disk I/O is much more expensive in most situations then network
>>>
>>> bandwith.
>>>
>>>
>>> Implementing it will be some copy-paste work. It could be
>>>
>>> implemented in
>>>
>>>
>>> few days. For the deadline of feature freeze, I will implement it
>>>
>>> after
>>>
>>> that , if needed.
>>>
>>>
>>>
>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>>
>>> window we have to make sure we don't release with only total IOps and
>>>
>>> fix
>>>
>>> it in 4.3, that will confuse users.
>>>
>>>
>>> Wido
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>
>>>
>>> Hi Wei,
>>>
>>>
>>>
>>>
>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>
>>>
>>> Hi,
>>>
>>>
>>> I would like to merge disk_io_throttling branch into master.
>>>
>>> If nobody object, I will merge into master in 48 hours.
>>>
>>> The purpose is :
>>>
>>>
>>> Virtual machines are running on the same storage device (local
>>>
>>> storage or
>>>
>>> share strage). Because of the rate limitation of device (such as
>>>
>>> iops), if
>>>
>>> one VM has large disk operation, it may affect the disk performance
>>>
>>> of
>>>
>>> other VMs running on the same storage device.
>>>
>>> It is neccesary to set the maximum rate and limit the disk I/O of
>>>
>>> VMs.
>>>
>>>
>>>
>>> Looking at the code I see you make no difference between Read and
>>>
>>> Write
>>>
>>> IOps.
>>>
>>>
>>> Qemu and libvirt support setting both a different rate for Read and
>>>
>>> Write
>>>
>>> IOps which could benefit a lot of users.
>>>
>>>
>>> It's also strange, in the polling side you collect both the Read and
>>>
>>> Write
>>>
>>> IOps, but on the throttling side you only go for a global value.
>>>
>>>
>>> Write IOps are usually much more expensive then Read IOps, so it
>>>
>>> seems
>>>
>>> like a valid use-case where that an admin would set a lower value for
>>>
>>> write
>>>
>>> IOps vs Read IOps.
>>>
>>>
>>> Since this only supports KVM at this point I think it would be of
>>>
>>> great
>>>
>>> value to at least have the mechanism in place to support both,
>>>
>>> implementing
>>>
>>> this later would be a lot of work.
>>>
>>>
>>> If a hypervisor doesn't support setting different values for read and
>>>
>>> write you can always sum both up and set that as the total limit.
>>>
>>>
>>> Can you explain why you implemented it this way?
>>>
>>>
>>> Wido
>>>
>>>
>>> The feature includes:
>>>
>>>
>>>
>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>
>>> configuration)
>>>
>>> (2) change the maximum rate of VMs
>>>
>>> (3) limit the disk rate (total bps and iops)
>>>
>>> JIRA ticket: https://issues.apache.org/****
>>>
>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>>>
>>> jira/browse/CLOUDSTACK-1192<
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>
>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>
>>> <**
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>
>>>
>>>
>>> FS (I will update later) :
>>>
>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>
>>> <
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>
>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>> http://cwiki.apache.org/confluence/**>
>>>
>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>
>>>
>>>
>>>
>>> Merge check list :-
>>>
>>>
>>> * Did you check the branch's RAT execution success?
>>>
>>> Yes
>>>
>>>
>>> * Are there new dependencies introduced?
>>>
>>> No
>>>
>>>
>>> * What automated testing (unit and integration) is included in the
>>>
>>> new
>>>
>>> feature?
>>>
>>> Unit tests are added.
>>>
>>>
>>> * What testing has been done to check for potential regressions?
>>>
>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>
>>> (2) VM operations, including
>>>
>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>
>>> (3) Volume operations, including
>>>
>>> Attach, Detach
>>>
>>>
>>> To review the code, you can try
>>>
>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>
>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>
>>>
>>> Best regards,
>>>
>>> Wei
>>>
>>>
>>> [1]
>>>
>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>
>>> <
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>
>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>> http://cwiki.apache.org/confluence/**>
>>>
>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>
>>>
>>>
>>>
>>> [2] refs/heads/disk_io_throttling
>>>
>>> [3]
>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>
>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>
>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>
>>> <**
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>
>>>
>>>
>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>
>>> **<
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>
>>> <**
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>
>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>
>>>
>>>
>>> (**CLOUDSTACK-1301
>>>
>>> - VM Disk I/O Throttling)
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
Hi John, Mike
I hope Mike's aswer helps you. I am trying to adding more.
(1) I think billing should depend on IO statistics rather than IOPS
limitation. Please review disk_io_stat if you have time. disk_io_stat can
get the IO statistics including bytes/iops read/write for an individual
virtual machine.
(2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
limitation for a running virtual machine through command line. However,
CloudStack does not support changing the parameters of a created offering
(computer offering or disk offering).
(3) It is a good question. Maybe it is better to commit Mike's patch after
disk_io_throttling as Mike needs to consider the limitation in hypervisor
type, I think.
- Wei
2013/6/3 John Burwell <jb...@basho.com>
> Mike,
>
> The things I want to understand are the following:
>
> 1) Is there value in capturing IOPS policies be captured in a common
> data model (e.g. for billing/usage purposes, expressing offerings).
> 2) Should there be a common interface model for reasoning about IOP
> provisioning at runtime?
> 3) How are conflicting provisioned IOPS configurations between a
> hypervisor and storage device reconciled? In particular, a scenario where
> is lead to believe (and billed) for more IOPS configured for a VM than a
> storage device has been configured to deliver. Another scenario could a
> consistent configuration between a VM and a storage device at creation
> time, but a later modification to storage device introduces logical
> inconsistency.
>
> Thanks,
> -John
>
> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> Hi John,
>
> I believe Wei's feature deals with controlling the max number of IOPS from
> the hypervisor side.
>
> My feature is focused on controlling IOPS from the storage system side.
>
> I hope that helps. :)
>
>
> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com> wrote:
>
> > Wei,
> >
> > My opinion is that no features should be merged until all functional
> > issues have been resolved and it is ready to turn over to test. Until
> the
> > total Ops vs discrete read/write ops issue is addressed and re-reviewed
> by
> > Wido, I don't think this criteria has been satisfied.
> >
> > Also, how does this work intersect/compliment the SolidFire patch (
> > https://reviews.apache.org/r/11479/)? As I understand it that work is
> > also involves provisioned IOPS. I would like to ensure we don't have a
> > scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> > incompatible.
> >
> > Thanks,
> > -John
> >
> > On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
> >
> > Wido,
> >
> >
> > Sure. I will change it next week.
> >
> >
> > -Wei
> >
> >
> >
> > 2013/6/1 Wido den Hollander <wi...@widodh.nl>
> >
> >
> > Hi Wei,
> >
> >
> >
> > On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >
> >
> > Wido,
> >
> >
> > Exactly. I have pushed the features into master.
> >
> >
> > If anyone object thems for technical reason till Monday, I will revert
> >
> > them.
> >
> >
> > For the sake of clarity I just want to mention again that we should
> change
> >
> > the total IOps to R/W IOps asap so that we never release a version with
> >
> > only total IOps.
> >
> >
> > You laid the groundwork for the I/O throttling and that's great! We
> should
> >
> > however prevent that we create legacy from day #1.
> >
> >
> > Wido
> >
> >
> > -Wei
> >
> >
> >
> > 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >
> >
> > On 05/31/2013 03:59 PM, John Burwell wrote:
> >
> >
> > Wido,
> >
> >
> > +1 -- this enhancement must to discretely support read and write IOPS.
> >
> > I
> >
> > don't see how it could be fixed later because I don't see how we
> >
> > correctly
> >
> > split total IOPS into read and write. Therefore, we would be stuck
> >
> > with a
> >
> > total unless/until we decided to break backwards compatibility.
> >
> >
> >
> > What Wei meant was merging it into master now so that it will go in the
> >
> > 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
> >
> > will be released with Read and Write instead of Total IOps.
> >
> >
> > This is to make the May 31st feature freeze date. But if the window moves
> >
> > (see other threads) then it won't be necessary to do that.
> >
> >
> > Wido
> >
> >
> >
> > I also completely agree that there is no association between network
> >
> > and
> >
> >
> > disk I/O.
> >
> >
> > Thanks,
> >
> > -John
> >
> >
> > On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> >
> >
> > Hi Wei,
> >
> >
> >
> > On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >
> >
> > Hi Wido,
> >
> >
> > Thanks. Good question.
> >
> >
> > I thought about at the beginning. Finally I decided to ignore the
> >
> > difference of read and write mainly because the network throttling did
> >
> > not
> >
> > care the difference of sent and received bytes as well.
> >
> > That reasoning seems odd. Networking and disk I/O completely different.
> >
> >
> > Disk I/O is much more expensive in most situations then network
> >
> > bandwith.
> >
> >
> > Implementing it will be some copy-paste work. It could be
> >
> > implemented in
> >
> >
> > few days. For the deadline of feature freeze, I will implement it
> >
> > after
> >
> > that , if needed.
> >
> >
> >
> > It think it's a feature we can't miss. But if it goes into the 4.2
> >
> > window we have to make sure we don't release with only total IOps and
> >
> > fix
> >
> > it in 4.3, that will confuse users.
> >
> >
> > Wido
> >
> >
> > -Wei
> >
> >
> >
> >
> >
> >
> > 2013/5/31 Wido den Hollander <wi...@widodh.nl>
> >
> >
> > Hi Wei,
> >
> >
> >
> >
> > On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >
> >
> > Hi,
> >
> >
> > I would like to merge disk_io_throttling branch into master.
> >
> > If nobody object, I will merge into master in 48 hours.
> >
> > The purpose is :
> >
> >
> > Virtual machines are running on the same storage device (local
> >
> > storage or
> >
> > share strage). Because of the rate limitation of device (such as
> >
> > iops), if
> >
> > one VM has large disk operation, it may affect the disk performance
> >
> > of
> >
> > other VMs running on the same storage device.
> >
> > It is neccesary to set the maximum rate and limit the disk I/O of
> >
> > VMs.
> >
> >
> >
> > Looking at the code I see you make no difference between Read and
> >
> > Write
> >
> > IOps.
> >
> >
> > Qemu and libvirt support setting both a different rate for Read and
> >
> > Write
> >
> > IOps which could benefit a lot of users.
> >
> >
> > It's also strange, in the polling side you collect both the Read and
> >
> > Write
> >
> > IOps, but on the throttling side you only go for a global value.
> >
> >
> > Write IOps are usually much more expensive then Read IOps, so it
> >
> > seems
> >
> > like a valid use-case where that an admin would set a lower value for
> >
> > write
> >
> > IOps vs Read IOps.
> >
> >
> > Since this only supports KVM at this point I think it would be of
> >
> > great
> >
> > value to at least have the mechanism in place to support both,
> >
> > implementing
> >
> > this later would be a lot of work.
> >
> >
> > If a hypervisor doesn't support setting different values for read and
> >
> > write you can always sum both up and set that as the total limit.
> >
> >
> > Can you explain why you implemented it this way?
> >
> >
> > Wido
> >
> >
> > The feature includes:
> >
> >
> >
> > (1) set the maximum rate of VMs (in disk_offering, and global
> >
> > configuration)
> >
> > (2) change the maximum rate of VMs
> >
> > (3) limit the disk rate (total bps and iops)
> >
> > JIRA ticket: https://issues.apache.org/****
> >
> > jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> >
> > jira/browse/CLOUDSTACK-1192<
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >
> > <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >
> > <**
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >
> >
> >
> > FS (I will update later) :
> >
> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >
> > <
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >
> > VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > http://cwiki.apache.org/confluence/**>
> >
> > display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >
> >
> >
> >
> > Merge check list :-
> >
> >
> > * Did you check the branch's RAT execution success?
> >
> > Yes
> >
> >
> > * Are there new dependencies introduced?
> >
> > No
> >
> >
> > * What automated testing (unit and integration) is included in the
> >
> > new
> >
> > feature?
> >
> > Unit tests are added.
> >
> >
> > * What testing has been done to check for potential regressions?
> >
> > (1) set the bytes rate and IOPS rate on CloudStack UI.
> >
> > (2) VM operations, including
> >
> > deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >
> > (3) Volume operations, including
> >
> > Attach, Detach
> >
> >
> > To review the code, you can try
> >
> > git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >
> > f2e5591b710d04cc86815044f5823e******73a4a58944
> >
> >
> > Best regards,
> >
> > Wei
> >
> >
> > [1]
> >
> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >
> > <
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >
> > VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > http://cwiki.apache.org/confluence/**>
> >
> > display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >
> > apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >
> >
> >
> >
> > [2] refs/heads/disk_io_throttling
> >
> > [3]
> > https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> >
> > <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >
> > <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >
> > <**
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >
> >
> >
> > <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> > http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> >
> > **<
> > http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >
> > <**
> > https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >
> > <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >
> >
> >
> > (**CLOUDSTACK-1301
> >
> > - VM Disk I/O Throttling)
> >
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
I should also point out, John, that I am only supporting Disk Offerings for
4.2 (not Compute Offerings). I plan to support Compute Offerings in 4.3.
Also, the patch I submitted is for XenServer only. Depending on when we
freeze 4.2, I could submit code for VMware.
The XenServer and VMware code, though, is technically not a part of the
plug-in, but rather the storage framework (enhancing what Edison built). It
could be leveraged by any dynamic, zone-wide primary storage.
On Sun, Jun 2, 2013 at 7:14 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> Sending to the CS e-mail list (I accidentally only sent this to John
> before).
>
>
>
> On Sun, Jun 2, 2013 at 7:10 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Hi John,
>>
>> Let me see if I can answer your questions.
>>
>> 1) If we mean provide the ability to charge more for higher IOPS Disk
>> Offerings, then I would say 'yes.'
>>
>> 2) I might need more detail on what you're thinking here.
>>
>> 3) The IOPS in my patch code are from the storage system's point of view
>> only. If the hypervisor has a Max that is, say, below the Max of the
>> storage system, the storage system will never hit its Max. That being the
>> case, it probably makes sense to provide an error message to the user in
>> such a situation. I haven't actually seen Wei's code yet, but we should
>> coordinate on activities like this.
>>
>>
>> On Sun, Jun 2, 2013 at 6:54 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Mike,
>>>
>>> The things I want to understand are the following:
>>>
>>> 1) Is there value in capturing IOPS policies be captured in a common
>>> data model (e.g. for billing/usage purposes, expressing offerings).
>>> 2) Should there be a common interface model for reasoning about IOP
>>> provisioning at runtime?
>>> 3) How are conflicting provisioned IOPS configurations between a
>>> hypervisor and storage device reconciled? In particular, a scenario where
>>> is lead to believe (and billed) for more IOPS configured for a VM than a
>>> storage device has been configured to deliver. Another scenario could a
>>> consistent configuration between a VM and a storage device at creation
>>> time, but a later modification to storage device introduces logical
>>> inconsistency.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <mi...@solidfire.com>
>>> wrote:
>>>
>>> Hi John,
>>>
>>> I believe Wei's feature deals with controlling the max number of IOPS
>>> from the hypervisor side.
>>>
>>> My feature is focused on controlling IOPS from the storage system side.
>>>
>>> I hope that helps. :)
>>>
>>>
>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com> wrote:
>>>
>>>> Wei,
>>>>
>>>> My opinion is that no features should be merged until all functional
>>>> issues have been resolved and it is ready to turn over to test. Until the
>>>> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
>>>> Wido, I don't think this criteria has been satisfied.
>>>>
>>>> Also, how does this work intersect/compliment the SolidFire patch (
>>>> https://reviews.apache.org/r/11479/)? As I understand it that work is
>>>> also involves provisioned IOPS. I would like to ensure we don't have a
>>>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
>>>> incompatible.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>>>>
>>>> Wido,
>>>>
>>>>
>>>> Sure. I will change it next week.
>>>>
>>>>
>>>> -Wei
>>>>
>>>>
>>>>
>>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>>
>>>>
>>>> Hi Wei,
>>>>
>>>>
>>>>
>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>>
>>>>
>>>> Wido,
>>>>
>>>>
>>>> Exactly. I have pushed the features into master.
>>>>
>>>>
>>>> If anyone object thems for technical reason till Monday, I will revert
>>>>
>>>> them.
>>>>
>>>>
>>>> For the sake of clarity I just want to mention again that we should
>>>> change
>>>>
>>>> the total IOps to R/W IOps asap so that we never release a version with
>>>>
>>>> only total IOps.
>>>>
>>>>
>>>> You laid the groundwork for the I/O throttling and that's great! We
>>>> should
>>>>
>>>> however prevent that we create legacy from day #1.
>>>>
>>>>
>>>> Wido
>>>>
>>>>
>>>> -Wei
>>>>
>>>>
>>>>
>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>
>>>>
>>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>>
>>>>
>>>> Wido,
>>>>
>>>>
>>>> +1 -- this enhancement must to discretely support read and write IOPS.
>>>>
>>>> I
>>>>
>>>> don't see how it could be fixed later because I don't see how we
>>>>
>>>> correctly
>>>>
>>>> split total IOPS into read and write. Therefore, we would be stuck
>>>>
>>>> with a
>>>>
>>>> total unless/until we decided to break backwards compatibility.
>>>>
>>>>
>>>>
>>>> What Wei meant was merging it into master now so that it will go in the
>>>>
>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>>>
>>>> will be released with Read and Write instead of Total IOps.
>>>>
>>>>
>>>> This is to make the May 31st feature freeze date. But if the window
>>>> moves
>>>>
>>>> (see other threads) then it won't be necessary to do that.
>>>>
>>>>
>>>> Wido
>>>>
>>>>
>>>>
>>>> I also completely agree that there is no association between network
>>>>
>>>> and
>>>>
>>>>
>>>> disk I/O.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> -John
>>>>
>>>>
>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>>>
>>>>
>>>> Hi Wei,
>>>>
>>>>
>>>>
>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>
>>>>
>>>> Hi Wido,
>>>>
>>>>
>>>> Thanks. Good question.
>>>>
>>>>
>>>> I thought about at the beginning. Finally I decided to ignore the
>>>>
>>>> difference of read and write mainly because the network throttling did
>>>>
>>>> not
>>>>
>>>> care the difference of sent and received bytes as well.
>>>>
>>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>>
>>>>
>>>> Disk I/O is much more expensive in most situations then network
>>>>
>>>> bandwith.
>>>>
>>>>
>>>> Implementing it will be some copy-paste work. It could be
>>>>
>>>> implemented in
>>>>
>>>>
>>>> few days. For the deadline of feature freeze, I will implement it
>>>>
>>>> after
>>>>
>>>> that , if needed.
>>>>
>>>>
>>>>
>>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>>>
>>>> window we have to make sure we don't release with only total IOps and
>>>>
>>>> fix
>>>>
>>>> it in 4.3, that will confuse users.
>>>>
>>>>
>>>> Wido
>>>>
>>>>
>>>> -Wei
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>
>>>>
>>>> Hi Wei,
>>>>
>>>>
>>>>
>>>>
>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>> I would like to merge disk_io_throttling branch into master.
>>>>
>>>> If nobody object, I will merge into master in 48 hours.
>>>>
>>>> The purpose is :
>>>>
>>>>
>>>> Virtual machines are running on the same storage device (local
>>>>
>>>> storage or
>>>>
>>>> share strage). Because of the rate limitation of device (such as
>>>>
>>>> iops), if
>>>>
>>>> one VM has large disk operation, it may affect the disk performance
>>>>
>>>> of
>>>>
>>>> other VMs running on the same storage device.
>>>>
>>>> It is neccesary to set the maximum rate and limit the disk I/O of
>>>>
>>>> VMs.
>>>>
>>>>
>>>>
>>>> Looking at the code I see you make no difference between Read and
>>>>
>>>> Write
>>>>
>>>> IOps.
>>>>
>>>>
>>>> Qemu and libvirt support setting both a different rate for Read and
>>>>
>>>> Write
>>>>
>>>> IOps which could benefit a lot of users.
>>>>
>>>>
>>>> It's also strange, in the polling side you collect both the Read and
>>>>
>>>> Write
>>>>
>>>> IOps, but on the throttling side you only go for a global value.
>>>>
>>>>
>>>> Write IOps are usually much more expensive then Read IOps, so it
>>>>
>>>> seems
>>>>
>>>> like a valid use-case where that an admin would set a lower value for
>>>>
>>>> write
>>>>
>>>> IOps vs Read IOps.
>>>>
>>>>
>>>> Since this only supports KVM at this point I think it would be of
>>>>
>>>> great
>>>>
>>>> value to at least have the mechanism in place to support both,
>>>>
>>>> implementing
>>>>
>>>> this later would be a lot of work.
>>>>
>>>>
>>>> If a hypervisor doesn't support setting different values for read and
>>>>
>>>> write you can always sum both up and set that as the total limit.
>>>>
>>>>
>>>> Can you explain why you implemented it this way?
>>>>
>>>>
>>>> Wido
>>>>
>>>>
>>>> The feature includes:
>>>>
>>>>
>>>>
>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>
>>>> configuration)
>>>>
>>>> (2) change the maximum rate of VMs
>>>>
>>>> (3) limit the disk rate (total bps and iops)
>>>>
>>>> JIRA ticket: https://issues.apache.org/****
>>>>
>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>>>>
>>>> jira/browse/CLOUDSTACK-1192<
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>
>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>
>>>> <**
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>
>>>>
>>>>
>>>> FS (I will update later) :
>>>>
>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>
>>>> <
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>
>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>>> http://cwiki.apache.org/confluence/**>
>>>>
>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>
>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> >
>>>>
>>>>
>>>>
>>>> Merge check list :-
>>>>
>>>>
>>>> * Did you check the branch's RAT execution success?
>>>>
>>>> Yes
>>>>
>>>>
>>>> * Are there new dependencies introduced?
>>>>
>>>> No
>>>>
>>>>
>>>> * What automated testing (unit and integration) is included in the
>>>>
>>>> new
>>>>
>>>> feature?
>>>>
>>>> Unit tests are added.
>>>>
>>>>
>>>> * What testing has been done to check for potential regressions?
>>>>
>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>
>>>> (2) VM operations, including
>>>>
>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>
>>>> (3) Volume operations, including
>>>>
>>>> Attach, Detach
>>>>
>>>>
>>>> To review the code, you can try
>>>>
>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>>
>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Wei
>>>>
>>>>
>>>> [1]
>>>>
>>>>
>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>
>>>> <
>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>
>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>>> http://cwiki.apache.org/confluence/**>
>>>>
>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>
>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>> >
>>>>
>>>>
>>>>
>>>> [2] refs/heads/disk_io_throttling
>>>>
>>>> [3]
>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>>
>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>
>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>
>>>> <**
>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>
>>>>
>>>>
>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>>
>>>> **<
>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>
>>>> <**
>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>
>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>
>>>>
>>>>
>>>> (**CLOUDSTACK-1301
>>>>
>>>> - VM Disk I/O Throttling)
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *â„¢*
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Sending to the CS e-mail list (I accidentally only sent this to John
before).
On Sun, Jun 2, 2013 at 7:10 PM, Mike Tutkowski <mike.tutkowski@solidfire.com
> wrote:
> Hi John,
>
> Let me see if I can answer your questions.
>
> 1) If we mean provide the ability to charge more for higher IOPS Disk
> Offerings, then I would say 'yes.'
>
> 2) I might need more detail on what you're thinking here.
>
> 3) The IOPS in my patch code are from the storage system's point of view
> only. If the hypervisor has a Max that is, say, below the Max of the
> storage system, the storage system will never hit its Max. That being the
> case, it probably makes sense to provide an error message to the user in
> such a situation. I haven't actually seen Wei's code yet, but we should
> coordinate on activities like this.
>
>
> On Sun, Jun 2, 2013 at 6:54 PM, John Burwell <jb...@basho.com> wrote:
>
>> Mike,
>>
>> The things I want to understand are the following:
>>
>> 1) Is there value in capturing IOPS policies be captured in a common
>> data model (e.g. for billing/usage purposes, expressing offerings).
>> 2) Should there be a common interface model for reasoning about IOP
>> provisioning at runtime?
>> 3) How are conflicting provisioned IOPS configurations between a
>> hypervisor and storage device reconciled? In particular, a scenario where
>> is lead to believe (and billed) for more IOPS configured for a VM than a
>> storage device has been configured to deliver. Another scenario could a
>> consistent configuration between a VM and a storage device at creation
>> time, but a later modification to storage device introduces logical
>> inconsistency.
>>
>> Thanks,
>> -John
>>
>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> Hi John,
>>
>> I believe Wei's feature deals with controlling the max number of IOPS
>> from the hypervisor side.
>>
>> My feature is focused on controlling IOPS from the storage system side.
>>
>> I hope that helps. :)
>>
>>
>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com> wrote:
>>
>>> Wei,
>>>
>>> My opinion is that no features should be merged until all functional
>>> issues have been resolved and it is ready to turn over to test. Until the
>>> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
>>> Wido, I don't think this criteria has been satisfied.
>>>
>>> Also, how does this work intersect/compliment the SolidFire patch (
>>> https://reviews.apache.org/r/11479/)? As I understand it that work is
>>> also involves provisioned IOPS. I would like to ensure we don't have a
>>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
>>> incompatible.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>>>
>>> Wido,
>>>
>>>
>>> Sure. I will change it next week.
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>>>
>>>
>>> Hi Wei,
>>>
>>>
>>>
>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>
>>>
>>> Wido,
>>>
>>>
>>> Exactly. I have pushed the features into master.
>>>
>>>
>>> If anyone object thems for technical reason till Monday, I will revert
>>>
>>> them.
>>>
>>>
>>> For the sake of clarity I just want to mention again that we should
>>> change
>>>
>>> the total IOps to R/W IOps asap so that we never release a version with
>>>
>>> only total IOps.
>>>
>>>
>>> You laid the groundwork for the I/O throttling and that's great! We
>>> should
>>>
>>> however prevent that we create legacy from day #1.
>>>
>>>
>>> Wido
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>
>>>
>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>
>>>
>>> Wido,
>>>
>>>
>>> +1 -- this enhancement must to discretely support read and write IOPS.
>>>
>>> I
>>>
>>> don't see how it could be fixed later because I don't see how we
>>>
>>> correctly
>>>
>>> split total IOPS into read and write. Therefore, we would be stuck
>>>
>>> with a
>>>
>>> total unless/until we decided to break backwards compatibility.
>>>
>>>
>>>
>>> What Wei meant was merging it into master now so that it will go in the
>>>
>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>>
>>> will be released with Read and Write instead of Total IOps.
>>>
>>>
>>> This is to make the May 31st feature freeze date. But if the window moves
>>>
>>> (see other threads) then it won't be necessary to do that.
>>>
>>>
>>> Wido
>>>
>>>
>>>
>>> I also completely agree that there is no association between network
>>>
>>> and
>>>
>>>
>>> disk I/O.
>>>
>>>
>>> Thanks,
>>>
>>> -John
>>>
>>>
>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>>
>>>
>>> Hi Wei,
>>>
>>>
>>>
>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>
>>>
>>> Hi Wido,
>>>
>>>
>>> Thanks. Good question.
>>>
>>>
>>> I thought about at the beginning. Finally I decided to ignore the
>>>
>>> difference of read and write mainly because the network throttling did
>>>
>>> not
>>>
>>> care the difference of sent and received bytes as well.
>>>
>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>
>>>
>>> Disk I/O is much more expensive in most situations then network
>>>
>>> bandwith.
>>>
>>>
>>> Implementing it will be some copy-paste work. It could be
>>>
>>> implemented in
>>>
>>>
>>> few days. For the deadline of feature freeze, I will implement it
>>>
>>> after
>>>
>>> that , if needed.
>>>
>>>
>>>
>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>>
>>> window we have to make sure we don't release with only total IOps and
>>>
>>> fix
>>>
>>> it in 4.3, that will confuse users.
>>>
>>>
>>> Wido
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>
>>>
>>> Hi Wei,
>>>
>>>
>>>
>>>
>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>
>>>
>>> Hi,
>>>
>>>
>>> I would like to merge disk_io_throttling branch into master.
>>>
>>> If nobody object, I will merge into master in 48 hours.
>>>
>>> The purpose is :
>>>
>>>
>>> Virtual machines are running on the same storage device (local
>>>
>>> storage or
>>>
>>> share strage). Because of the rate limitation of device (such as
>>>
>>> iops), if
>>>
>>> one VM has large disk operation, it may affect the disk performance
>>>
>>> of
>>>
>>> other VMs running on the same storage device.
>>>
>>> It is neccesary to set the maximum rate and limit the disk I/O of
>>>
>>> VMs.
>>>
>>>
>>>
>>> Looking at the code I see you make no difference between Read and
>>>
>>> Write
>>>
>>> IOps.
>>>
>>>
>>> Qemu and libvirt support setting both a different rate for Read and
>>>
>>> Write
>>>
>>> IOps which could benefit a lot of users.
>>>
>>>
>>> It's also strange, in the polling side you collect both the Read and
>>>
>>> Write
>>>
>>> IOps, but on the throttling side you only go for a global value.
>>>
>>>
>>> Write IOps are usually much more expensive then Read IOps, so it
>>>
>>> seems
>>>
>>> like a valid use-case where that an admin would set a lower value for
>>>
>>> write
>>>
>>> IOps vs Read IOps.
>>>
>>>
>>> Since this only supports KVM at this point I think it would be of
>>>
>>> great
>>>
>>> value to at least have the mechanism in place to support both,
>>>
>>> implementing
>>>
>>> this later would be a lot of work.
>>>
>>>
>>> If a hypervisor doesn't support setting different values for read and
>>>
>>> write you can always sum both up and set that as the total limit.
>>>
>>>
>>> Can you explain why you implemented it this way?
>>>
>>>
>>> Wido
>>>
>>>
>>> The feature includes:
>>>
>>>
>>>
>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>
>>> configuration)
>>>
>>> (2) change the maximum rate of VMs
>>>
>>> (3) limit the disk rate (total bps and iops)
>>>
>>> JIRA ticket: https://issues.apache.org/****
>>>
>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>>>
>>> jira/browse/CLOUDSTACK-1192<
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>
>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>
>>> <**
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>
>>>
>>>
>>> FS (I will update later) :
>>>
>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>
>>> <
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>
>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>> http://cwiki.apache.org/confluence/**>
>>>
>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >
>>>
>>>
>>>
>>> Merge check list :-
>>>
>>>
>>> * Did you check the branch's RAT execution success?
>>>
>>> Yes
>>>
>>>
>>> * Are there new dependencies introduced?
>>>
>>> No
>>>
>>>
>>> * What automated testing (unit and integration) is included in the
>>>
>>> new
>>>
>>> feature?
>>>
>>> Unit tests are added.
>>>
>>>
>>> * What testing has been done to check for potential regressions?
>>>
>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>
>>> (2) VM operations, including
>>>
>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>
>>> (3) Volume operations, including
>>>
>>> Attach, Detach
>>>
>>>
>>> To review the code, you can try
>>>
>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>
>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>
>>>
>>> Best regards,
>>>
>>> Wei
>>>
>>>
>>> [1]
>>>
>>>
>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>
>>> <
>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>
>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
>>> http://cwiki.apache.org/confluence/**>
>>>
>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>
>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>> >
>>>
>>>
>>>
>>> [2] refs/heads/disk_io_throttling
>>>
>>> [3]
>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>
>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>
>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>
>>> <**
>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>
>>>
>>>
>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>
>>> **<
>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>
>>> <**
>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>
>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>
>>>
>>>
>>> (**CLOUDSTACK-1301
>>>
>>> - VM Disk I/O Throttling)
>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *â„¢*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *â„¢*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Mike,
The things I want to understand are the following:
1) Is there value in capturing IOPS policies be captured in a common
data model (e.g. for billing/usage purposes, expressing offerings).
2) Should there be a common interface model for reasoning about IOP
provisioning at runtime?
3) How are conflicting provisioned IOPS configurations between a
hypervisor and storage device reconciled? In particular, a scenario where
is lead to believe (and billed) for more IOPS configured for a VM than a
storage device has been configured to deliver. Another scenario could a
consistent configuration between a VM and a storage device at creation
time, but a later modification to storage device introduces logical
inconsistency.
Thanks,
-John
On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <mi...@solidfire.com>
wrote:
Hi John,
I believe Wei's feature deals with controlling the max number of IOPS from
the hypervisor side.
My feature is focused on controlling IOPS from the storage system side.
I hope that helps. :)
On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com> wrote:
> Wei,
>
> My opinion is that no features should be merged until all functional
> issues have been resolved and it is ready to turn over to test. Until the
> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
> Wido, I don't think this criteria has been satisfied.
>
> Also, how does this work intersect/compliment the SolidFire patch (
> https://reviews.apache.org/r/11479/)? As I understand it that work is
> also involves provisioned IOPS. I would like to ensure we don't have a
> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> incompatible.
>
> Thanks,
> -John
>
> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>
> Wido,
>
>
> Sure. I will change it next week.
>
>
> -Wei
>
>
>
> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>
>
> Hi Wei,
>
>
>
> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>
>
> Wido,
>
>
> Exactly. I have pushed the features into master.
>
>
> If anyone object thems for technical reason till Monday, I will revert
>
> them.
>
>
> For the sake of clarity I just want to mention again that we should change
>
> the total IOps to R/W IOps asap so that we never release a version with
>
> only total IOps.
>
>
> You laid the groundwork for the I/O throttling and that's great! We should
>
> however prevent that we create legacy from day #1.
>
>
> Wido
>
>
> -Wei
>
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>
> On 05/31/2013 03:59 PM, John Burwell wrote:
>
>
> Wido,
>
>
> +1 -- this enhancement must to discretely support read and write IOPS.
>
> I
>
> don't see how it could be fixed later because I don't see how we
>
> correctly
>
> split total IOPS into read and write. Therefore, we would be stuck
>
> with a
>
> total unless/until we decided to break backwards compatibility.
>
>
>
> What Wei meant was merging it into master now so that it will go in the
>
> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>
> will be released with Read and Write instead of Total IOps.
>
>
> This is to make the May 31st feature freeze date. But if the window moves
>
> (see other threads) then it won't be necessary to do that.
>
>
> Wido
>
>
>
> I also completely agree that there is no association between network
>
> and
>
>
> disk I/O.
>
>
> Thanks,
>
> -John
>
>
> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>
> Hi Wei,
>
>
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>
>
> Hi Wido,
>
>
> Thanks. Good question.
>
>
> I thought about at the beginning. Finally I decided to ignore the
>
> difference of read and write mainly because the network throttling did
>
> not
>
> care the difference of sent and received bytes as well.
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
>
> Disk I/O is much more expensive in most situations then network
>
> bandwith.
>
>
> Implementing it will be some copy-paste work. It could be
>
> implemented in
>
>
> few days. For the deadline of feature freeze, I will implement it
>
> after
>
> that , if needed.
>
>
>
> It think it's a feature we can't miss. But if it goes into the 4.2
>
> window we have to make sure we don't release with only total IOps and
>
> fix
>
> it in 4.3, that will confuse users.
>
>
> Wido
>
>
> -Wei
>
>
>
>
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>
> Hi Wei,
>
>
>
>
> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>
>
> Hi,
>
>
> I would like to merge disk_io_throttling branch into master.
>
> If nobody object, I will merge into master in 48 hours.
>
> The purpose is :
>
>
> Virtual machines are running on the same storage device (local
>
> storage or
>
> share strage). Because of the rate limitation of device (such as
>
> iops), if
>
> one VM has large disk operation, it may affect the disk performance
>
> of
>
> other VMs running on the same storage device.
>
> It is neccesary to set the maximum rate and limit the disk I/O of
>
> VMs.
>
>
>
> Looking at the code I see you make no difference between Read and
>
> Write
>
> IOps.
>
>
> Qemu and libvirt support setting both a different rate for Read and
>
> Write
>
> IOps which could benefit a lot of users.
>
>
> It's also strange, in the polling side you collect both the Read and
>
> Write
>
> IOps, but on the throttling side you only go for a global value.
>
>
> Write IOps are usually much more expensive then Read IOps, so it
>
> seems
>
> like a valid use-case where that an admin would set a lower value for
>
> write
>
> IOps vs Read IOps.
>
>
> Since this only supports KVM at this point I think it would be of
>
> great
>
> value to at least have the mechanism in place to support both,
>
> implementing
>
> this later would be a lot of work.
>
>
> If a hypervisor doesn't support setting different values for read and
>
> write you can always sum both up and set that as the total limit.
>
>
> Can you explain why you implemented it this way?
>
>
> Wido
>
>
> The feature includes:
>
>
>
> (1) set the maximum rate of VMs (in disk_offering, and global
>
> configuration)
>
> (2) change the maximum rate of VMs
>
> (3) limit the disk rate (total bps and iops)
>
> JIRA ticket: https://issues.apache.org/****
>
> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>
> jira/browse/CLOUDSTACK-1192<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>
> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>
> <**
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>
>
>
> FS (I will update later) :
>
>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>
> <
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>
> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> http://cwiki.apache.org/confluence/**>
>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >
>
>
>
> Merge check list :-
>
>
> * Did you check the branch's RAT execution success?
>
> Yes
>
>
> * Are there new dependencies introduced?
>
> No
>
>
> * What automated testing (unit and integration) is included in the
>
> new
>
> feature?
>
> Unit tests are added.
>
>
> * What testing has been done to check for potential regressions?
>
> (1) set the bytes rate and IOPS rate on CloudStack UI.
>
> (2) VM operations, including
>
> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>
> (3) Volume operations, including
>
> Attach, Detach
>
>
> To review the code, you can try
>
> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>
> f2e5591b710d04cc86815044f5823e******73a4a58944
>
>
> Best regards,
>
> Wei
>
>
> [1]
>
>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>
> <
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>
> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> http://cwiki.apache.org/confluence/**>
>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >
>
>
>
> [2] refs/heads/disk_io_throttling
>
> [3]
> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>
> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>
> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>
> <**
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>
>
>
> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>
> **<
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>
> <**
> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>
> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>
>
>
> (**CLOUDSTACK-1301
>
> - VM Disk I/O Throttling)
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi John,
I believe Wei's feature deals with controlling the max number of IOPS from
the hypervisor side.
My feature is focused on controlling IOPS from the storage system side.
I hope that helps. :)
On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jb...@basho.com> wrote:
> Wei,
>
> My opinion is that no features should be merged until all functional
> issues have been resolved and it is ready to turn over to test. Until the
> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
> Wido, I don't think this criteria has been satisfied.
>
> Also, how does this work intersect/compliment the SolidFire patch (
> https://reviews.apache.org/r/11479/)? As I understand it that work is
> also involves provisioned IOPS. I would like to ensure we don't have a
> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> incompatible.
>
> Thanks,
> -John
>
> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>
> Wido,
>
>
> Sure. I will change it next week.
>
>
> -Wei
>
>
>
> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>
>
> Hi Wei,
>
>
>
> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>
>
> Wido,
>
>
> Exactly. I have pushed the features into master.
>
>
> If anyone object thems for technical reason till Monday, I will revert
>
> them.
>
>
> For the sake of clarity I just want to mention again that we should change
>
> the total IOps to R/W IOps asap so that we never release a version with
>
> only total IOps.
>
>
> You laid the groundwork for the I/O throttling and that's great! We should
>
> however prevent that we create legacy from day #1.
>
>
> Wido
>
>
> -Wei
>
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>
> On 05/31/2013 03:59 PM, John Burwell wrote:
>
>
> Wido,
>
>
> +1 -- this enhancement must to discretely support read and write IOPS.
>
> I
>
> don't see how it could be fixed later because I don't see how we
>
> correctly
>
> split total IOPS into read and write. Therefore, we would be stuck
>
> with a
>
> total unless/until we decided to break backwards compatibility.
>
>
>
> What Wei meant was merging it into master now so that it will go in the
>
> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>
> will be released with Read and Write instead of Total IOps.
>
>
> This is to make the May 31st feature freeze date. But if the window moves
>
> (see other threads) then it won't be necessary to do that.
>
>
> Wido
>
>
>
> I also completely agree that there is no association between network
>
> and
>
>
> disk I/O.
>
>
> Thanks,
>
> -John
>
>
> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>
> Hi Wei,
>
>
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>
>
> Hi Wido,
>
>
> Thanks. Good question.
>
>
> I thought about at the beginning. Finally I decided to ignore the
>
> difference of read and write mainly because the network throttling did
>
> not
>
> care the difference of sent and received bytes as well.
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
>
> Disk I/O is much more expensive in most situations then network
>
> bandwith.
>
>
> Implementing it will be some copy-paste work. It could be
>
> implemented in
>
>
> few days. For the deadline of feature freeze, I will implement it
>
> after
>
> that , if needed.
>
>
>
> It think it's a feature we can't miss. But if it goes into the 4.2
>
> window we have to make sure we don't release with only total IOps and
>
> fix
>
> it in 4.3, that will confuse users.
>
>
> Wido
>
>
> -Wei
>
>
>
>
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>
> Hi Wei,
>
>
>
>
> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>
>
> Hi,
>
>
> I would like to merge disk_io_throttling branch into master.
>
> If nobody object, I will merge into master in 48 hours.
>
> The purpose is :
>
>
> Virtual machines are running on the same storage device (local
>
> storage or
>
> share strage). Because of the rate limitation of device (such as
>
> iops), if
>
> one VM has large disk operation, it may affect the disk performance
>
> of
>
> other VMs running on the same storage device.
>
> It is neccesary to set the maximum rate and limit the disk I/O of
>
> VMs.
>
>
>
> Looking at the code I see you make no difference between Read and
>
> Write
>
> IOps.
>
>
> Qemu and libvirt support setting both a different rate for Read and
>
> Write
>
> IOps which could benefit a lot of users.
>
>
> It's also strange, in the polling side you collect both the Read and
>
> Write
>
> IOps, but on the throttling side you only go for a global value.
>
>
> Write IOps are usually much more expensive then Read IOps, so it
>
> seems
>
> like a valid use-case where that an admin would set a lower value for
>
> write
>
> IOps vs Read IOps.
>
>
> Since this only supports KVM at this point I think it would be of
>
> great
>
> value to at least have the mechanism in place to support both,
>
> implementing
>
> this later would be a lot of work.
>
>
> If a hypervisor doesn't support setting different values for read and
>
> write you can always sum both up and set that as the total limit.
>
>
> Can you explain why you implemented it this way?
>
>
> Wido
>
>
> The feature includes:
>
>
>
> (1) set the maximum rate of VMs (in disk_offering, and global
>
> configuration)
>
> (2) change the maximum rate of VMs
>
> (3) limit the disk rate (total bps and iops)
>
> JIRA ticket: https://issues.apache.org/****
>
> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>
> jira/browse/CLOUDSTACK-1192<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>
> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>
> <**
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>
>
>
> FS (I will update later) :
>
>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>
> <
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>
> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> http://cwiki.apache.org/confluence/**>
>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >
>
>
>
> Merge check list :-
>
>
> * Did you check the branch's RAT execution success?
>
> Yes
>
>
> * Are there new dependencies introduced?
>
> No
>
>
> * What automated testing (unit and integration) is included in the
>
> new
>
> feature?
>
> Unit tests are added.
>
>
> * What testing has been done to check for potential regressions?
>
> (1) set the bytes rate and IOPS rate on CloudStack UI.
>
> (2) VM operations, including
>
> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>
> (3) Volume operations, including
>
> Attach, Detach
>
>
> To review the code, you can try
>
> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>
> f2e5591b710d04cc86815044f5823e******73a4a58944
>
>
> Best regards,
>
> Wei
>
>
> [1]
>
>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>
> <
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>
> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> http://cwiki.apache.org/confluence/**>
>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >
>
>
>
> [2] refs/heads/disk_io_throttling
>
> [3]
> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>
> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>
> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>
> <**
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>
>
>
> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>
> **<
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>
> <**
> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>
> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>
>
>
> (**CLOUDSTACK-1301
>
> - VM Disk I/O Throttling)
>
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*â„¢*
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
John,
Thanks for your reply.
I will revert the commit for the opnion from Wido and you, and ask you
reviewing it after modification.
-Wei
2013/6/3, John Burwell <jb...@basho.com>:
> Wei,
>
> My opinion is that no features should be merged until all functional issues
> have been resolved and it is ready to turn over to test. Until the total
> Ops vs discrete read/write ops issue is addressed and re-reviewed by Wido,
> I don't think this criteria has been satisfied.
>
> Also, how does this work intersect/compliment the SolidFire patch (
> https://reviews.apache.org/r/11479/)? As I understand it that work is also
> involves provisioned IOPS. I would like to ensure we don't have a scenario
> where provisioned IOPS in KVM and SolidFire are unnecessarily incompatible.
>
> Thanks,
> -John
>
> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
>
> Wido,
>
>
> Sure. I will change it next week.
>
>
> -Wei
>
>
>
> 2013/6/1 Wido den Hollander <wi...@widodh.nl>
>
>
> Hi Wei,
>
>
>
> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>
>
> Wido,
>
>
> Exactly. I have pushed the features into master.
>
>
> If anyone object thems for technical reason till Monday, I will revert
>
> them.
>
>
> For the sake of clarity I just want to mention again that we should change
>
> the total IOps to R/W IOps asap so that we never release a version with
>
> only total IOps.
>
>
> You laid the groundwork for the I/O throttling and that's great! We should
>
> however prevent that we create legacy from day #1.
>
>
> Wido
>
>
> -Wei
>
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>
> On 05/31/2013 03:59 PM, John Burwell wrote:
>
>
> Wido,
>
>
> +1 -- this enhancement must to discretely support read and write IOPS.
>
> I
>
> don't see how it could be fixed later because I don't see how we
>
> correctly
>
> split total IOPS into read and write. Therefore, we would be stuck
>
> with a
>
> total unless/until we decided to break backwards compatibility.
>
>
>
> What Wei meant was merging it into master now so that it will go in the
>
> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>
> will be released with Read and Write instead of Total IOps.
>
>
> This is to make the May 31st feature freeze date. But if the window moves
>
> (see other threads) then it won't be necessary to do that.
>
>
> Wido
>
>
>
> I also completely agree that there is no association between network
>
> and
>
>
> disk I/O.
>
>
> Thanks,
>
> -John
>
>
> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>
> Hi Wei,
>
>
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>
>
> Hi Wido,
>
>
> Thanks. Good question.
>
>
> I thought about at the beginning. Finally I decided to ignore the
>
> difference of read and write mainly because the network throttling did
>
> not
>
> care the difference of sent and received bytes as well.
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
>
> Disk I/O is much more expensive in most situations then network
>
> bandwith.
>
>
> Implementing it will be some copy-paste work. It could be
>
> implemented in
>
>
> few days. For the deadline of feature freeze, I will implement it
>
> after
>
> that , if needed.
>
>
>
> It think it's a feature we can't miss. But if it goes into the 4.2
>
> window we have to make sure we don't release with only total IOps and
>
> fix
>
> it in 4.3, that will confuse users.
>
>
> Wido
>
>
> -Wei
>
>
>
>
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>
> Hi Wei,
>
>
>
>
> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>
>
> Hi,
>
>
> I would like to merge disk_io_throttling branch into master.
>
> If nobody object, I will merge into master in 48 hours.
>
> The purpose is :
>
>
> Virtual machines are running on the same storage device (local
>
> storage or
>
> share strage). Because of the rate limitation of device (such as
>
> iops), if
>
> one VM has large disk operation, it may affect the disk performance
>
> of
>
> other VMs running on the same storage device.
>
> It is neccesary to set the maximum rate and limit the disk I/O of
>
> VMs.
>
>
>
> Looking at the code I see you make no difference between Read and
>
> Write
>
> IOps.
>
>
> Qemu and libvirt support setting both a different rate for Read and
>
> Write
>
> IOps which could benefit a lot of users.
>
>
> It's also strange, in the polling side you collect both the Read and
>
> Write
>
> IOps, but on the throttling side you only go for a global value.
>
>
> Write IOps are usually much more expensive then Read IOps, so it
>
> seems
>
> like a valid use-case where that an admin would set a lower value for
>
> write
>
> IOps vs Read IOps.
>
>
> Since this only supports KVM at this point I think it would be of
>
> great
>
> value to at least have the mechanism in place to support both,
>
> implementing
>
> this later would be a lot of work.
>
>
> If a hypervisor doesn't support setting different values for read and
>
> write you can always sum both up and set that as the total limit.
>
>
> Can you explain why you implemented it this way?
>
>
> Wido
>
>
> The feature includes:
>
>
>
> (1) set the maximum rate of VMs (in disk_offering, and global
>
> configuration)
>
> (2) change the maximum rate of VMs
>
> (3) limit the disk rate (total bps and iops)
>
> JIRA ticket: https://issues.apache.org/****
>
> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>
> jira/browse/CLOUDSTACK-1192<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>
> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>
> <**
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>
>
>
> FS (I will update later) :
>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>
> <
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>
> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> http://cwiki.apache.org/confluence/**>
>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>
>
>
>
> Merge check list :-
>
>
> * Did you check the branch's RAT execution success?
>
> Yes
>
>
> * Are there new dependencies introduced?
>
> No
>
>
> * What automated testing (unit and integration) is included in the
>
> new
>
> feature?
>
> Unit tests are added.
>
>
> * What testing has been done to check for potential regressions?
>
> (1) set the bytes rate and IOPS rate on CloudStack UI.
>
> (2) VM operations, including
>
> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>
> (3) Volume operations, including
>
> Attach, Detach
>
>
> To review the code, you can try
>
> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>
> f2e5591b710d04cc86815044f5823e******73a4a58944
>
>
> Best regards,
>
> Wei
>
>
> [1]
>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>
> <
> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>
> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> http://cwiki.apache.org/confluence/**>
>
> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>
>
>
>
> [2] refs/heads/disk_io_throttling
>
> [3]
> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>
> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>
> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>
> <**
> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>
>
>
> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>
> **<
> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>
> <**
> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>
> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>
>
>
> (**CLOUDSTACK-1301
>
> - VM Disk I/O Throttling)
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Wei,
My opinion is that no features should be merged until all functional issues
have been resolved and it is ready to turn over to test. Until the total
Ops vs discrete read/write ops issue is addressed and re-reviewed by Wido,
I don't think this criteria has been satisfied.
Also, how does this work intersect/compliment the SolidFire patch (
https://reviews.apache.org/r/11479/)? As I understand it that work is also
involves provisioned IOPS. I would like to ensure we don't have a scenario
where provisioned IOPS in KVM and SolidFire are unnecessarily incompatible.
Thanks,
-John
On Jun 1, 2013, at 6:47 AM, Wei ZHOU <us...@gmail.com> wrote:
Wido,
Sure. I will change it next week.
-Wei
2013/6/1 Wido den Hollander <wi...@widodh.nl>
Hi Wei,
On 06/01/2013 08:24 AM, Wei ZHOU wrote:
Wido,
Exactly. I have pushed the features into master.
If anyone object thems for technical reason till Monday, I will revert
them.
For the sake of clarity I just want to mention again that we should change
the total IOps to R/W IOps asap so that we never release a version with
only total IOps.
You laid the groundwork for the I/O throttling and that's great! We should
however prevent that we create legacy from day #1.
Wido
-Wei
2013/5/31 Wido den Hollander <wi...@widodh.nl>
On 05/31/2013 03:59 PM, John Burwell wrote:
Wido,
+1 -- this enhancement must to discretely support read and write IOPS.
I
don't see how it could be fixed later because I don't see how we
correctly
split total IOPS into read and write. Therefore, we would be stuck
with a
total unless/until we decided to break backwards compatibility.
What Wei meant was merging it into master now so that it will go in the
4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
will be released with Read and Write instead of Total IOps.
This is to make the May 31st feature freeze date. But if the window moves
(see other threads) then it won't be necessary to do that.
Wido
I also completely agree that there is no association between network
and
disk I/O.
Thanks,
-John
On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
Hi Wei,
On 05/31/2013 03:13 PM, Wei ZHOU wrote:
Hi Wido,
Thanks. Good question.
I thought about at the beginning. Finally I decided to ignore the
difference of read and write mainly because the network throttling did
not
care the difference of sent and received bytes as well.
That reasoning seems odd. Networking and disk I/O completely different.
Disk I/O is much more expensive in most situations then network
bandwith.
Implementing it will be some copy-paste work. It could be
implemented in
few days. For the deadline of feature freeze, I will implement it
after
that , if needed.
It think it's a feature we can't miss. But if it goes into the 4.2
window we have to make sure we don't release with only total IOps and
fix
it in 4.3, that will confuse users.
Wido
-Wei
2013/5/31 Wido den Hollander <wi...@widodh.nl>
Hi Wei,
On 05/30/2013 06:03 PM, Wei ZHOU wrote:
Hi,
I would like to merge disk_io_throttling branch into master.
If nobody object, I will merge into master in 48 hours.
The purpose is :
Virtual machines are running on the same storage device (local
storage or
share strage). Because of the rate limitation of device (such as
iops), if
one VM has large disk operation, it may affect the disk performance
of
other VMs running on the same storage device.
It is neccesary to set the maximum rate and limit the disk I/O of
VMs.
Looking at the code I see you make no difference between Read and
Write
IOps.
Qemu and libvirt support setting both a different rate for Read and
Write
IOps which could benefit a lot of users.
It's also strange, in the polling side you collect both the Read and
Write
IOps, but on the throttling side you only go for a global value.
Write IOps are usually much more expensive then Read IOps, so it
seems
like a valid use-case where that an admin would set a lower value for
write
IOps vs Read IOps.
Since this only supports KVM at this point I think it would be of
great
value to at least have the mechanism in place to support both,
implementing
this later would be a lot of work.
If a hypervisor doesn't support setting different values for read and
write you can always sum both up and set that as the total limit.
Can you explain why you implemented it this way?
Wido
The feature includes:
(1) set the maximum rate of VMs (in disk_offering, and global
configuration)
(2) change the maximum rate of VMs
(3) limit the disk rate (total bps and iops)
JIRA ticket: https://issues.apache.org/****
jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
jira/browse/CLOUDSTACK-1192<
https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
<ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
<**
https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
FS (I will update later) :
https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
<
https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
http://cwiki.apache.org/confluence/**>
display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>
Merge check list :-
* Did you check the branch's RAT execution success?
Yes
* Are there new dependencies introduced?
No
* What automated testing (unit and integration) is included in the
new
feature?
Unit tests are added.
* What testing has been done to check for potential regressions?
(1) set the bytes rate and IOPS rate on CloudStack UI.
(2) VM operations, including
deploy, stop, start, reboot, destroy, expunge. migrate, restore
(3) Volume operations, including
Attach, Detach
To review the code, you can try
git diff c30057635d04a2396f84c588127d7e******be42e503a7
f2e5591b710d04cc86815044f5823e******73a4a58944
Best regards,
Wei
[1]
https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
<
https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
http://cwiki.apache.org/confluence/**>
display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>
[2] refs/heads/disk_io_throttling
[3]
https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
<ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
<ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
<**
https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
<ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
**<
http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
<**
https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
<h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
(**CLOUDSTACK-1301
- VM Disk I/O Throttling)
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
Wido,
Sure. I will change it next week.
-Wei
2013/6/1 Wido den Hollander <wi...@widodh.nl>
> Hi Wei,
>
>
> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>
>> Wido,
>>
>> Exactly. I have pushed the features into master.
>>
>> If anyone object thems for technical reason till Monday, I will revert
>> them.
>>
>
> For the sake of clarity I just want to mention again that we should change
> the total IOps to R/W IOps asap so that we never release a version with
> only total IOps.
>
> You laid the groundwork for the I/O throttling and that's great! We should
> however prevent that we create legacy from day #1.
>
> Wido
>
> -Wei
>>
>>
>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>
>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>
>>> Wido,
>>>>
>>>> +1 -- this enhancement must to discretely support read and write IOPS.
>>>> I
>>>> don't see how it could be fixed later because I don't see how we
>>>> correctly
>>>> split total IOPS into read and write. Therefore, we would be stuck
>>>> with a
>>>> total unless/until we decided to break backwards compatibility.
>>>>
>>>>
>>>> What Wei meant was merging it into master now so that it will go in the
>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>> will be released with Read and Write instead of Total IOps.
>>>
>>> This is to make the May 31st feature freeze date. But if the window moves
>>> (see other threads) then it won't be necessary to do that.
>>>
>>> Wido
>>>
>>>
>>> I also completely agree that there is no association between network
>>> and
>>>
>>>> disk I/O.
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>>>
>>>> Hi Wei,
>>>>
>>>>>
>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>>
>>>>> Hi Wido,
>>>>>>
>>>>>> Thanks. Good question.
>>>>>>
>>>>>> I thought about at the beginning. Finally I decided to ignore the
>>>>>> difference of read and write mainly because the network throttling did
>>>>>> not
>>>>>> care the difference of sent and received bytes as well.
>>>>>>
>>>>>>
>>>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>>>
>>>>> Disk I/O is much more expensive in most situations then network
>>>>> bandwith.
>>>>>
>>>>> Implementing it will be some copy-paste work. It could be
>>>>> implemented in
>>>>>
>>>>>> few days. For the deadline of feature freeze, I will implement it
>>>>>> after
>>>>>> that , if needed.
>>>>>>
>>>>>>
>>>>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>>>> window we have to make sure we don't release with only total IOps and
>>>>> fix
>>>>> it in 4.3, that will confuse users.
>>>>>
>>>>> Wido
>>>>>
>>>>> -Wei
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>>
>>>>>> Hi Wei,
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>>> The purpose is :
>>>>>>>>
>>>>>>>> Virtual machines are running on the same storage device (local
>>>>>>>> storage or
>>>>>>>> share strage). Because of the rate limitation of device (such as
>>>>>>>> iops), if
>>>>>>>> one VM has large disk operation, it may affect the disk performance
>>>>>>>> of
>>>>>>>> other VMs running on the same storage device.
>>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of
>>>>>>>> VMs.
>>>>>>>>
>>>>>>>>
>>>>>>>> Looking at the code I see you make no difference between Read and
>>>>>>>>
>>>>>>> Write
>>>>>>> IOps.
>>>>>>>
>>>>>>> Qemu and libvirt support setting both a different rate for Read and
>>>>>>> Write
>>>>>>> IOps which could benefit a lot of users.
>>>>>>>
>>>>>>> It's also strange, in the polling side you collect both the Read and
>>>>>>> Write
>>>>>>> IOps, but on the throttling side you only go for a global value.
>>>>>>>
>>>>>>> Write IOps are usually much more expensive then Read IOps, so it
>>>>>>> seems
>>>>>>> like a valid use-case where that an admin would set a lower value for
>>>>>>> write
>>>>>>> IOps vs Read IOps.
>>>>>>>
>>>>>>> Since this only supports KVM at this point I think it would be of
>>>>>>> great
>>>>>>> value to at least have the mechanism in place to support both,
>>>>>>> implementing
>>>>>>> this later would be a lot of work.
>>>>>>>
>>>>>>> If a hypervisor doesn't support setting different values for read and
>>>>>>> write you can always sum both up and set that as the total limit.
>>>>>>>
>>>>>>> Can you explain why you implemented it this way?
>>>>>>>
>>>>>>> Wido
>>>>>>>
>>>>>>> The feature includes:
>>>>>>>
>>>>>>>
>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>>>> configuration)
>>>>>>>> (2) change the maximum rate of VMs
>>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
>>>>>>>> jira/browse/CLOUDSTACK-1192<https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>>> >
>>>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
>>>>>>>> <**https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>> FS (I will update later) :
>>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>> <https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>> >
>>>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<http://cwiki.apache.org/confluence/**>
>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>> Merge check list :-
>>>>>>>>
>>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>>> Yes
>>>>>>>>
>>>>>>>> * Are there new dependencies introduced?
>>>>>>>> No
>>>>>>>>
>>>>>>>> * What automated testing (unit and integration) is included in the
>>>>>>>> new
>>>>>>>> feature?
>>>>>>>> Unit tests are added.
>>>>>>>>
>>>>>>>> * What testing has been done to check for potential regressions?
>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>>> (2) VM operations, including
>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>>>> (3) Volume operations, including
>>>>>>>> Attach, Detach
>>>>>>>>
>>>>>>>> To review the code, you can try
>>>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7
>>>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Wei
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
>>>>>>>> <https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>>> >
>>>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<http://cwiki.apache.org/confluence/**>
>>>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
>>>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>>>>> >
>>>>>>>>
>>>>>>>>>
>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>> [3] https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
>>>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>>> >
>>>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
>>>>>>>> <**https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>> >
>>>>>>>>
>>>>>>>>>
>>>>>>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
>>>>>>>> **<http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>>> >
>>>>>>>> <**https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
>>>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>>
>>>>>>>>>
>>>>>>>> - VM Disk I/O Throttling)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wido den Hollander <wi...@widodh.nl>.
Hi Wei,
On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> Wido,
>
> Exactly. I have pushed the features into master.
>
> If anyone object thems for technical reason till Monday, I will revert them.
For the sake of clarity I just want to mention again that we should
change the total IOps to R/W IOps asap so that we never release a
version with only total IOps.
You laid the groundwork for the I/O throttling and that's great! We
should however prevent that we create legacy from day #1.
Wido
> -Wei
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>
>>> Wido,
>>>
>>> +1 -- this enhancement must to discretely support read and write IOPS. I
>>> don't see how it could be fixed later because I don't see how we correctly
>>> split total IOPS into read and write. Therefore, we would be stuck with a
>>> total unless/until we decided to break backwards compatibility.
>>>
>>>
>> What Wei meant was merging it into master now so that it will go in the
>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>> will be released with Read and Write instead of Total IOps.
>>
>> This is to make the May 31st feature freeze date. But if the window moves
>> (see other threads) then it won't be necessary to do that.
>>
>> Wido
>>
>>
>> I also completely agree that there is no association between network and
>>> disk I/O.
>>>
>>> Thanks,
>>> -John
>>>
>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>>
>>> Hi Wei,
>>>>
>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>>
>>>>> Hi Wido,
>>>>>
>>>>> Thanks. Good question.
>>>>>
>>>>> I thought about at the beginning. Finally I decided to ignore the
>>>>> difference of read and write mainly because the network throttling did
>>>>> not
>>>>> care the difference of sent and received bytes as well.
>>>>>
>>>>
>>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>>
>>>> Disk I/O is much more expensive in most situations then network bandwith.
>>>>
>>>> Implementing it will be some copy-paste work. It could be implemented in
>>>>> few days. For the deadline of feature freeze, I will implement it after
>>>>> that , if needed.
>>>>>
>>>>>
>>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>>> window we have to make sure we don't release with only total IOps and fix
>>>> it in 4.3, that will confuse users.
>>>>
>>>> Wido
>>>>
>>>> -Wei
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>>
>>>>> Hi Wei,
>>>>>>
>>>>>>
>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>>
>>>>>> Hi,
>>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>>> The purpose is :
>>>>>>>
>>>>>>> Virtual machines are running on the same storage device (local
>>>>>>> storage or
>>>>>>> share strage). Because of the rate limitation of device (such as
>>>>>>> iops), if
>>>>>>> one VM has large disk operation, it may affect the disk performance of
>>>>>>> other VMs running on the same storage device.
>>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>>>>>
>>>>>>>
>>>>>>> Looking at the code I see you make no difference between Read and
>>>>>> Write
>>>>>> IOps.
>>>>>>
>>>>>> Qemu and libvirt support setting both a different rate for Read and
>>>>>> Write
>>>>>> IOps which could benefit a lot of users.
>>>>>>
>>>>>> It's also strange, in the polling side you collect both the Read and
>>>>>> Write
>>>>>> IOps, but on the throttling side you only go for a global value.
>>>>>>
>>>>>> Write IOps are usually much more expensive then Read IOps, so it seems
>>>>>> like a valid use-case where that an admin would set a lower value for
>>>>>> write
>>>>>> IOps vs Read IOps.
>>>>>>
>>>>>> Since this only supports KVM at this point I think it would be of great
>>>>>> value to at least have the mechanism in place to support both,
>>>>>> implementing
>>>>>> this later would be a lot of work.
>>>>>>
>>>>>> If a hypervisor doesn't support setting different values for read and
>>>>>> write you can always sum both up and set that as the total limit.
>>>>>>
>>>>>> Can you explain why you implemented it this way?
>>>>>>
>>>>>> Wido
>>>>>>
>>>>>> The feature includes:
>>>>>>
>>>>>>>
>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>>> configuration)
>>>>>>> (2) change the maximum rate of VMs
>>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>>> jira/browse/CLOUDSTACK-1192<https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>>>>
>>>>>>> FS (I will update later) :
>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>> VM+Disk+IO+Throttling<https://**cwiki.apache.org/confluence/**
>>>>>>> display/CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>>>>>
>>>>>>> Merge check list :-
>>>>>>>
>>>>>>> * Did you check the branch's RAT execution success?
>>>>>>> Yes
>>>>>>>
>>>>>>> * Are there new dependencies introduced?
>>>>>>> No
>>>>>>>
>>>>>>> * What automated testing (unit and integration) is included in the new
>>>>>>> feature?
>>>>>>> Unit tests are added.
>>>>>>>
>>>>>>> * What testing has been done to check for potential regressions?
>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>>> (2) VM operations, including
>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>>> (3) Volume operations, including
>>>>>>> Attach, Detach
>>>>>>>
>>>>>>> To review the code, you can try
>>>>>>> git diff c30057635d04a2396f84c588127d7e****be42e503a7
>>>>>>> f2e5591b710d04cc86815044f5823e****73a4a58944
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Wei
>>>>>>>
>>>>>>> [1]
>>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>>> VM+Disk+IO+Throttling<https://**cwiki.apache.org/confluence/**
>>>>>>> display/CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>>>>>
>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>> [3] https://issues.apache.org/****jira/browse/CLOUDSTACK-1301<https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>>>>
>>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>>> <**https://issues.apache.org/**jira/browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>>>>
>>>>>>>
>>>>>>>> (**CLOUDSTACK-1301
>>>>>>>>
>>>>>>>
>>>>>>> - VM Disk I/O Throttling)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
Wido,
Exactly. I have pushed the features into master.
If anyone object thems for technical reason till Monday, I will revert them.
-Wei
2013/5/31 Wido den Hollander <wi...@widodh.nl>
> On 05/31/2013 03:59 PM, John Burwell wrote:
>
>> Wido,
>>
>> +1 -- this enhancement must to discretely support read and write IOPS. I
>> don't see how it could be fixed later because I don't see how we correctly
>> split total IOPS into read and write. Therefore, we would be stuck with a
>> total unless/until we decided to break backwards compatibility.
>>
>>
> What Wei meant was merging it into master now so that it will go in the
> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
> will be released with Read and Write instead of Total IOps.
>
> This is to make the May 31st feature freeze date. But if the window moves
> (see other threads) then it won't be necessary to do that.
>
> Wido
>
>
> I also completely agree that there is no association between network and
>> disk I/O.
>>
>> Thanks,
>> -John
>>
>> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>>
>> Hi Wei,
>>>
>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>
>>>> Hi Wido,
>>>>
>>>> Thanks. Good question.
>>>>
>>>> I thought about at the beginning. Finally I decided to ignore the
>>>> difference of read and write mainly because the network throttling did
>>>> not
>>>> care the difference of sent and received bytes as well.
>>>>
>>>
>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>
>>> Disk I/O is much more expensive in most situations then network bandwith.
>>>
>>> Implementing it will be some copy-paste work. It could be implemented in
>>>> few days. For the deadline of feature freeze, I will implement it after
>>>> that , if needed.
>>>>
>>>>
>>> It think it's a feature we can't miss. But if it goes into the 4.2
>>> window we have to make sure we don't release with only total IOps and fix
>>> it in 4.3, that will confuse users.
>>>
>>> Wido
>>>
>>> -Wei
>>>>
>>>>
>>>>
>>>>
>>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>>
>>>> Hi Wei,
>>>>>
>>>>>
>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>>
>>>>> Hi,
>>>>>> I would like to merge disk_io_throttling branch into master.
>>>>>> If nobody object, I will merge into master in 48 hours.
>>>>>> The purpose is :
>>>>>>
>>>>>> Virtual machines are running on the same storage device (local
>>>>>> storage or
>>>>>> share strage). Because of the rate limitation of device (such as
>>>>>> iops), if
>>>>>> one VM has large disk operation, it may affect the disk performance of
>>>>>> other VMs running on the same storage device.
>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>>>>
>>>>>>
>>>>>> Looking at the code I see you make no difference between Read and
>>>>> Write
>>>>> IOps.
>>>>>
>>>>> Qemu and libvirt support setting both a different rate for Read and
>>>>> Write
>>>>> IOps which could benefit a lot of users.
>>>>>
>>>>> It's also strange, in the polling side you collect both the Read and
>>>>> Write
>>>>> IOps, but on the throttling side you only go for a global value.
>>>>>
>>>>> Write IOps are usually much more expensive then Read IOps, so it seems
>>>>> like a valid use-case where that an admin would set a lower value for
>>>>> write
>>>>> IOps vs Read IOps.
>>>>>
>>>>> Since this only supports KVM at this point I think it would be of great
>>>>> value to at least have the mechanism in place to support both,
>>>>> implementing
>>>>> this later would be a lot of work.
>>>>>
>>>>> If a hypervisor doesn't support setting different values for read and
>>>>> write you can always sum both up and set that as the total limit.
>>>>>
>>>>> Can you explain why you implemented it this way?
>>>>>
>>>>> Wido
>>>>>
>>>>> The feature includes:
>>>>>
>>>>>>
>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>>> configuration)
>>>>>> (2) change the maximum rate of VMs
>>>>>> (3) limit the disk rate (total bps and iops)
>>>>>> JIRA ticket: https://issues.apache.org/****
>>>>>> jira/browse/CLOUDSTACK-1192<https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>>> >
>>>>>> FS (I will update later) :
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>> VM+Disk+IO+Throttling<https://**cwiki.apache.org/confluence/**
>>>>>> display/CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>>> >
>>>>>> Merge check list :-
>>>>>>
>>>>>> * Did you check the branch's RAT execution success?
>>>>>> Yes
>>>>>>
>>>>>> * Are there new dependencies introduced?
>>>>>> No
>>>>>>
>>>>>> * What automated testing (unit and integration) is included in the new
>>>>>> feature?
>>>>>> Unit tests are added.
>>>>>>
>>>>>> * What testing has been done to check for potential regressions?
>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>>> (2) VM operations, including
>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>>> (3) Volume operations, including
>>>>>> Attach, Detach
>>>>>>
>>>>>> To review the code, you can try
>>>>>> git diff c30057635d04a2396f84c588127d7e****be42e503a7
>>>>>> f2e5591b710d04cc86815044f5823e****73a4a58944
>>>>>>
>>>>>> Best regards,
>>>>>> Wei
>>>>>>
>>>>>> [1]
>>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>>>> VM+Disk+IO+Throttling<https://**cwiki.apache.org/confluence/**
>>>>>> display/CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>>> >
>>>>>> [2] refs/heads/disk_io_throttling
>>>>>> [3] https://issues.apache.org/****jira/browse/CLOUDSTACK-1301<https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>>> >
>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>>>> <**https://issues.apache.org/**jira/browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>> >
>>>>>>
>>>>>>> (**CLOUDSTACK-1301
>>>>>>>
>>>>>>
>>>>>> - VM Disk I/O Throttling)
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wido den Hollander <wi...@widodh.nl>.
On 05/31/2013 03:59 PM, John Burwell wrote:
> Wido,
>
> +1 -- this enhancement must to discretely support read and write IOPS. I don't see how it could be fixed later because I don't see how we correctly split total IOPS into read and write. Therefore, we would be stuck with a total unless/until we decided to break backwards compatibility.
>
What Wei meant was merging it into master now so that it will go in the
4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
will be released with Read and Write instead of Total IOps.
This is to make the May 31st feature freeze date. But if the window
moves (see other threads) then it won't be necessary to do that.
Wido
> I also completely agree that there is no association between network and disk I/O.
>
> Thanks,
> -John
>
> On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>> Hi Wei,
>>
>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>> Hi Wido,
>>>
>>> Thanks. Good question.
>>>
>>> I thought about at the beginning. Finally I decided to ignore the
>>> difference of read and write mainly because the network throttling did not
>>> care the difference of sent and received bytes as well.
>>
>> That reasoning seems odd. Networking and disk I/O completely different.
>>
>> Disk I/O is much more expensive in most situations then network bandwith.
>>
>>> Implementing it will be some copy-paste work. It could be implemented in
>>> few days. For the deadline of feature freeze, I will implement it after
>>> that , if needed.
>>>
>>
>> It think it's a feature we can't miss. But if it goes into the 4.2 window we have to make sure we don't release with only total IOps and fix it in 4.3, that will confuse users.
>>
>> Wido
>>
>>> -Wei
>>>
>>>
>>>
>>>
>>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>>
>>>> Hi Wei,
>>>>
>>>>
>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>>
>>>>> Hi,
>>>>> I would like to merge disk_io_throttling branch into master.
>>>>> If nobody object, I will merge into master in 48 hours.
>>>>> The purpose is :
>>>>>
>>>>> Virtual machines are running on the same storage device (local storage or
>>>>> share strage). Because of the rate limitation of device (such as iops), if
>>>>> one VM has large disk operation, it may affect the disk performance of
>>>>> other VMs running on the same storage device.
>>>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>>>
>>>>>
>>>> Looking at the code I see you make no difference between Read and Write
>>>> IOps.
>>>>
>>>> Qemu and libvirt support setting both a different rate for Read and Write
>>>> IOps which could benefit a lot of users.
>>>>
>>>> It's also strange, in the polling side you collect both the Read and Write
>>>> IOps, but on the throttling side you only go for a global value.
>>>>
>>>> Write IOps are usually much more expensive then Read IOps, so it seems
>>>> like a valid use-case where that an admin would set a lower value for write
>>>> IOps vs Read IOps.
>>>>
>>>> Since this only supports KVM at this point I think it would be of great
>>>> value to at least have the mechanism in place to support both, implementing
>>>> this later would be a lot of work.
>>>>
>>>> If a hypervisor doesn't support setting different values for read and
>>>> write you can always sum both up and set that as the total limit.
>>>>
>>>> Can you explain why you implemented it this way?
>>>>
>>>> Wido
>>>>
>>>> The feature includes:
>>>>>
>>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>>> configuration)
>>>>> (2) change the maximum rate of VMs
>>>>> (3) limit the disk rate (total bps and iops)
>>>>> JIRA ticket: https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>>> FS (I will update later) :
>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>> Merge check list :-
>>>>>
>>>>> * Did you check the branch's RAT execution success?
>>>>> Yes
>>>>>
>>>>> * Are there new dependencies introduced?
>>>>> No
>>>>>
>>>>> * What automated testing (unit and integration) is included in the new
>>>>> feature?
>>>>> Unit tests are added.
>>>>>
>>>>> * What testing has been done to check for potential regressions?
>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>>> (2) VM operations, including
>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>>> (3) Volume operations, including
>>>>> Attach, Detach
>>>>>
>>>>> To review the code, you can try
>>>>> git diff c30057635d04a2396f84c588127d7e**be42e503a7
>>>>> f2e5591b710d04cc86815044f5823e**73a4a58944
>>>>>
>>>>> Best regards,
>>>>> Wei
>>>>>
>>>>> [1]
>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>>> [2] refs/heads/disk_io_throttling
>>>>> [3] https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>>> (**CLOUDSTACK-1301
>>>>>
>>>>> - VM Disk I/O Throttling)
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by John Burwell <jb...@basho.com>.
Wido,
+1 -- this enhancement must to discretely support read and write IOPS. I don't see how it could be fixed later because I don't see how we correctly split total IOPS into read and write. Therefore, we would be stuck with a total unless/until we decided to break backwards compatibility.
I also completely agree that there is no association between network and disk I/O.
Thanks,
-John
On May 31, 2013, at 9:51 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> Hi Wei,
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>> Hi Wido,
>>
>> Thanks. Good question.
>>
>> I thought about at the beginning. Finally I decided to ignore the
>> difference of read and write mainly because the network throttling did not
>> care the difference of sent and received bytes as well.
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
> Disk I/O is much more expensive in most situations then network bandwith.
>
>> Implementing it will be some copy-paste work. It could be implemented in
>> few days. For the deadline of feature freeze, I will implement it after
>> that , if needed.
>>
>
> It think it's a feature we can't miss. But if it goes into the 4.2 window we have to make sure we don't release with only total IOps and fix it in 4.3, that will confuse users.
>
> Wido
>
>> -Wei
>>
>>
>>
>>
>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>
>>> Hi Wei,
>>>
>>>
>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>
>>>> Hi,
>>>> I would like to merge disk_io_throttling branch into master.
>>>> If nobody object, I will merge into master in 48 hours.
>>>> The purpose is :
>>>>
>>>> Virtual machines are running on the same storage device (local storage or
>>>> share strage). Because of the rate limitation of device (such as iops), if
>>>> one VM has large disk operation, it may affect the disk performance of
>>>> other VMs running on the same storage device.
>>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>>
>>>>
>>> Looking at the code I see you make no difference between Read and Write
>>> IOps.
>>>
>>> Qemu and libvirt support setting both a different rate for Read and Write
>>> IOps which could benefit a lot of users.
>>>
>>> It's also strange, in the polling side you collect both the Read and Write
>>> IOps, but on the throttling side you only go for a global value.
>>>
>>> Write IOps are usually much more expensive then Read IOps, so it seems
>>> like a valid use-case where that an admin would set a lower value for write
>>> IOps vs Read IOps.
>>>
>>> Since this only supports KVM at this point I think it would be of great
>>> value to at least have the mechanism in place to support both, implementing
>>> this later would be a lot of work.
>>>
>>> If a hypervisor doesn't support setting different values for read and
>>> write you can always sum both up and set that as the total limit.
>>>
>>> Can you explain why you implemented it this way?
>>>
>>> Wido
>>>
>>> The feature includes:
>>>>
>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>> configuration)
>>>> (2) change the maximum rate of VMs
>>>> (3) limit the disk rate (total bps and iops)
>>>> JIRA ticket: https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>> FS (I will update later) :
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>> Merge check list :-
>>>>
>>>> * Did you check the branch's RAT execution success?
>>>> Yes
>>>>
>>>> * Are there new dependencies introduced?
>>>> No
>>>>
>>>> * What automated testing (unit and integration) is included in the new
>>>> feature?
>>>> Unit tests are added.
>>>>
>>>> * What testing has been done to check for potential regressions?
>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>> (2) VM operations, including
>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>> (3) Volume operations, including
>>>> Attach, Detach
>>>>
>>>> To review the code, you can try
>>>> git diff c30057635d04a2396f84c588127d7e**be42e503a7
>>>> f2e5591b710d04cc86815044f5823e**73a4a58944
>>>>
>>>> Best regards,
>>>> Wei
>>>>
>>>> [1]
>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>> [2] refs/heads/disk_io_throttling
>>>> [3] https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>>> (**CLOUDSTACK-1301
>>>>
>>>> - VM Disk I/O Throttling)
>>>>
>>>>
>>>
>>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
Wido,
I agree with you.
Obviously we can not merge it into 4.2 while change it in 4.3.
I will change it next week.
If possible, I will investigate other hypervisors support in the
Testing/Bug Fixes window (3/6/2013 to 30/6/2013).
-Wei
2013/5/31 Wido den Hollander <wi...@widodh.nl>
> Hi Wei,
>
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>
>> Hi Wido,
>>
>> Thanks. Good question.
>>
>> I thought about at the beginning. Finally I decided to ignore the
>> difference of read and write mainly because the network throttling did not
>> care the difference of sent and received bytes as well.
>>
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
> Disk I/O is much more expensive in most situations then network bandwith.
>
>
> Implementing it will be some copy-paste work. It could be implemented in
>> few days. For the deadline of feature freeze, I will implement it after
>> that , if needed.
>>
>>
> It think it's a feature we can't miss. But if it goes into the 4.2 window
> we have to make sure we don't release with only total IOps and fix it in
> 4.3, that will confuse users.
>
> Wido
>
> -Wei
>>
>>
>>
>>
>> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>>
>> Hi Wei,
>>>
>>>
>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>>
>>> Hi,
>>>> I would like to merge disk_io_throttling branch into master.
>>>> If nobody object, I will merge into master in 48 hours.
>>>> The purpose is :
>>>>
>>>> Virtual machines are running on the same storage device (local storage
>>>> or
>>>> share strage). Because of the rate limitation of device (such as iops),
>>>> if
>>>> one VM has large disk operation, it may affect the disk performance of
>>>> other VMs running on the same storage device.
>>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>>
>>>>
>>>> Looking at the code I see you make no difference between Read and Write
>>> IOps.
>>>
>>> Qemu and libvirt support setting both a different rate for Read and Write
>>> IOps which could benefit a lot of users.
>>>
>>> It's also strange, in the polling side you collect both the Read and
>>> Write
>>> IOps, but on the throttling side you only go for a global value.
>>>
>>> Write IOps are usually much more expensive then Read IOps, so it seems
>>> like a valid use-case where that an admin would set a lower value for
>>> write
>>> IOps vs Read IOps.
>>>
>>> Since this only supports KVM at this point I think it would be of great
>>> value to at least have the mechanism in place to support both,
>>> implementing
>>> this later would be a lot of work.
>>>
>>> If a hypervisor doesn't support setting different values for read and
>>> write you can always sum both up and set that as the total limit.
>>>
>>> Can you explain why you implemented it this way?
>>>
>>> Wido
>>>
>>> The feature includes:
>>>
>>>>
>>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>>> configuration)
>>>> (2) change the maximum rate of VMs
>>>> (3) limit the disk rate (total bps and iops)
>>>> JIRA ticket: https://issues.apache.org/****jira/browse/CLOUDSTACK-1192<https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>>> >
>>>>
>>>> FS (I will update later) :
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>> VM+Disk+IO+Throttling<https://**cwiki.apache.org/confluence/**
>>>> display/CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>> >
>>>>
>>>> Merge check list :-
>>>>
>>>> * Did you check the branch's RAT execution success?
>>>> Yes
>>>>
>>>> * Are there new dependencies introduced?
>>>> No
>>>>
>>>> * What automated testing (unit and integration) is included in the new
>>>> feature?
>>>> Unit tests are added.
>>>>
>>>> * What testing has been done to check for potential regressions?
>>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>>> (2) VM operations, including
>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>>> (3) Volume operations, including
>>>> Attach, Detach
>>>>
>>>> To review the code, you can try
>>>> git diff c30057635d04a2396f84c588127d7e****be42e503a7
>>>> f2e5591b710d04cc86815044f5823e****73a4a58944
>>>>
>>>> Best regards,
>>>> Wei
>>>>
>>>> [1]
>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****<https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
>>>> VM+Disk+IO+Throttling<https://**cwiki.apache.org/confluence/**
>>>> display/CLOUDSTACK/VM+Disk+IO+**Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>>> >
>>>> [2] refs/heads/disk_io_throttling
>>>> [3] https://issues.apache.org/****jira/browse/CLOUDSTACK-1301<https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
>>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>>> >
>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
>>>> <**https://issues.apache.org/**jira/browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>> >
>>>>
>>>>> (**CLOUDSTACK-1301
>>>>>
>>>>
>>>> - VM Disk I/O Throttling)
>>>>
>>>>
>>>>
>>>
>>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wido den Hollander <wi...@widodh.nl>.
Hi Wei,
On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> Hi Wido,
>
> Thanks. Good question.
>
> I thought about at the beginning. Finally I decided to ignore the
> difference of read and write mainly because the network throttling did not
> care the difference of sent and received bytes as well.
That reasoning seems odd. Networking and disk I/O completely different.
Disk I/O is much more expensive in most situations then network bandwith.
> Implementing it will be some copy-paste work. It could be implemented in
> few days. For the deadline of feature freeze, I will implement it after
> that , if needed.
>
It think it's a feature we can't miss. But if it goes into the 4.2
window we have to make sure we don't release with only total IOps and
fix it in 4.3, that will confuse users.
Wido
> -Wei
>
>
>
>
> 2013/5/31 Wido den Hollander <wi...@widodh.nl>
>
>> Hi Wei,
>>
>>
>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>
>>> Hi,
>>> I would like to merge disk_io_throttling branch into master.
>>> If nobody object, I will merge into master in 48 hours.
>>> The purpose is :
>>>
>>> Virtual machines are running on the same storage device (local storage or
>>> share strage). Because of the rate limitation of device (such as iops), if
>>> one VM has large disk operation, it may affect the disk performance of
>>> other VMs running on the same storage device.
>>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>>
>>>
>> Looking at the code I see you make no difference between Read and Write
>> IOps.
>>
>> Qemu and libvirt support setting both a different rate for Read and Write
>> IOps which could benefit a lot of users.
>>
>> It's also strange, in the polling side you collect both the Read and Write
>> IOps, but on the throttling side you only go for a global value.
>>
>> Write IOps are usually much more expensive then Read IOps, so it seems
>> like a valid use-case where that an admin would set a lower value for write
>> IOps vs Read IOps.
>>
>> Since this only supports KVM at this point I think it would be of great
>> value to at least have the mechanism in place to support both, implementing
>> this later would be a lot of work.
>>
>> If a hypervisor doesn't support setting different values for read and
>> write you can always sum both up and set that as the total limit.
>>
>> Can you explain why you implemented it this way?
>>
>> Wido
>>
>> The feature includes:
>>>
>>> (1) set the maximum rate of VMs (in disk_offering, and global
>>> configuration)
>>> (2) change the maximum rate of VMs
>>> (3) limit the disk rate (total bps and iops)
>>> JIRA ticket: https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>>> FS (I will update later) :
>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>> Merge check list :-
>>>
>>> * Did you check the branch's RAT execution success?
>>> Yes
>>>
>>> * Are there new dependencies introduced?
>>> No
>>>
>>> * What automated testing (unit and integration) is included in the new
>>> feature?
>>> Unit tests are added.
>>>
>>> * What testing has been done to check for potential regressions?
>>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>>> (2) VM operations, including
>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>>> (3) Volume operations, including
>>> Attach, Detach
>>>
>>> To review the code, you can try
>>> git diff c30057635d04a2396f84c588127d7e**be42e503a7
>>> f2e5591b710d04cc86815044f5823e**73a4a58944
>>>
>>> Best regards,
>>> Wei
>>>
>>> [1]
>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>>> [2] refs/heads/disk_io_throttling
>>> [3] https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>>>> (**CLOUDSTACK-1301
>>>
>>> - VM Disk I/O Throttling)
>>>
>>>
>>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wei ZHOU <us...@gmail.com>.
Hi Wido,
Thanks. Good question.
I thought about at the beginning. Finally I decided to ignore the
difference of read and write mainly because the network throttling did not
care the difference of sent and received bytes as well.
Implementing it will be some copy-paste work. It could be implemented in
few days. For the deadline of feature freeze, I will implement it after
that , if needed.
-Wei
2013/5/31 Wido den Hollander <wi...@widodh.nl>
> Hi Wei,
>
>
> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>
>> Hi,
>> I would like to merge disk_io_throttling branch into master.
>> If nobody object, I will merge into master in 48 hours.
>> The purpose is :
>>
>> Virtual machines are running on the same storage device (local storage or
>> share strage). Because of the rate limitation of device (such as iops), if
>> one VM has large disk operation, it may affect the disk performance of
>> other VMs running on the same storage device.
>> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>>
>>
> Looking at the code I see you make no difference between Read and Write
> IOps.
>
> Qemu and libvirt support setting both a different rate for Read and Write
> IOps which could benefit a lot of users.
>
> It's also strange, in the polling side you collect both the Read and Write
> IOps, but on the throttling side you only go for a global value.
>
> Write IOps are usually much more expensive then Read IOps, so it seems
> like a valid use-case where that an admin would set a lower value for write
> IOps vs Read IOps.
>
> Since this only supports KVM at this point I think it would be of great
> value to at least have the mechanism in place to support both, implementing
> this later would be a lot of work.
>
> If a hypervisor doesn't support setting different values for read and
> write you can always sum both up and set that as the total limit.
>
> Can you explain why you implemented it this way?
>
> Wido
>
> The feature includes:
>>
>> (1) set the maximum rate of VMs (in disk_offering, and global
>> configuration)
>> (2) change the maximum rate of VMs
>> (3) limit the disk rate (total bps and iops)
>> JIRA ticket: https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
>> FS (I will update later) :
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>> Merge check list :-
>>
>> * Did you check the branch's RAT execution success?
>> Yes
>>
>> * Are there new dependencies introduced?
>> No
>>
>> * What automated testing (unit and integration) is included in the new
>> feature?
>> Unit tests are added.
>>
>> * What testing has been done to check for potential regressions?
>> (1) set the bytes rate and IOPS rate on CloudStack UI.
>> (2) VM operations, including
>> deploy, stop, start, reboot, destroy, expunge. migrate, restore
>> (3) Volume operations, including
>> Attach, Detach
>>
>> To review the code, you can try
>> git diff c30057635d04a2396f84c588127d7e**be42e503a7
>> f2e5591b710d04cc86815044f5823e**73a4a58944
>>
>> Best regards,
>> Wei
>>
>> [1]
>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
>> VM+Disk+IO+Throttling<https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling>
>> [2] refs/heads/disk_io_throttling
>> [3] https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
>> <ht**tps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
>> >(**CLOUDSTACK-1301
>>
>> - VM Disk I/O Throttling)
>>
>>
>
Re: [MERGE] disk_io_throttling to MASTER
Posted by Wido den Hollander <wi...@widodh.nl>.
Hi Wei,
On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> Hi,
> I would like to merge disk_io_throttling branch into master.
> If nobody object, I will merge into master in 48 hours.
> The purpose is :
>
> Virtual machines are running on the same storage device (local storage or
> share strage). Because of the rate limitation of device (such as iops), if
> one VM has large disk operation, it may affect the disk performance of
> other VMs running on the same storage device.
> It is neccesary to set the maximum rate and limit the disk I/O of VMs.
>
Looking at the code I see you make no difference between Read and Write
IOps.
Qemu and libvirt support setting both a different rate for Read and
Write IOps which could benefit a lot of users.
It's also strange, in the polling side you collect both the Read and
Write IOps, but on the throttling side you only go for a global value.
Write IOps are usually much more expensive then Read IOps, so it seems
like a valid use-case where that an admin would set a lower value for
write IOps vs Read IOps.
Since this only supports KVM at this point I think it would be of great
value to at least have the mechanism in place to support both,
implementing this later would be a lot of work.
If a hypervisor doesn't support setting different values for read and
write you can always sum both up and set that as the total limit.
Can you explain why you implemented it this way?
Wido
> The feature includes:
>
> (1) set the maximum rate of VMs (in disk_offering, and global configuration)
> (2) change the maximum rate of VMs
> (3) limit the disk rate (total bps and iops)
> JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-1192
> FS (I will update later) :
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> Merge check list :-
>
> * Did you check the branch's RAT execution success?
> Yes
>
> * Are there new dependencies introduced?
> No
>
> * What automated testing (unit and integration) is included in the new
> feature?
> Unit tests are added.
>
> * What testing has been done to check for potential regressions?
> (1) set the bytes rate and IOPS rate on CloudStack UI.
> (2) VM operations, including
> deploy, stop, start, reboot, destroy, expunge. migrate, restore
> (3) Volume operations, including
> Attach, Detach
>
> To review the code, you can try
> git diff c30057635d04a2396f84c588127d7ebe42e503a7
> f2e5591b710d04cc86815044f5823e73a4a58944
>
> Best regards,
> Wei
>
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> [2] refs/heads/disk_io_throttling
> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<https://issues.apache.org/jira/browse/CLOUDSTACK-2071>(CLOUDSTACK-1301
> - VM Disk I/O Throttling)
>